• Abstract

hunchentoot-recycling-taskmaster is a taskmaster implementation for Hunchentoot, aiming to improve connection establishment efficiency through thread-pooling and flexible thread count adjustment.

• Performance tl;dr

[[file:pics/benchmark-result-2025-12-26/sleep_1ms/keep-alive/100_connections_requests_sec__sleep_1ms__keep-alive__higher_is_better.svg]]

[[file:pics/benchmark-result-2025-12-26/sleep_1ms/no-keep-alive/100_connections_requests_sec__sleep_1ms__no_keep-alive__higher_is_better.svg]]

On this benchmark, my HTTP handler always responds after 1ms to simulate some workloads.

• Hunchentoot is an all-rounder. It works well on Keep-alive connections. I think it is good for typical use-cases.
• If your workload does not utilize Keep-alive, hunchentoot-recycling-taskmaster may be useful.
• Woo is very difficult to use. Woo seems fast only when your handlers work with very low latency. See [[#about-woo][About Woo]] below.

In detail, See [[#benchmark][Benchmark]] below.

• How to use

** Caution about Lispworks

On Lispworks, hunchentoot-recycling-taskmaster does not work because Hunchentoot on that does not handle a listen socket directly.

** Installation

*** Loading

Currently hunchentoot-recycling-taskmaster is in [[https://ultralisp.org/][Ultralisp]] and [[https://www.quicklisp.org/beta/][Quicklisp]] . Not in [[https://github.com/ocicl/ocicl][ocicl]].

When you use Ultralisp or Quicklisp, =ql:quickload= it :

#+begin_src common-lisp (ql:quickload "hunchentoot-recycling-taskmaster") #+end_src

Or download it by yourself.

#+begin_src sh cd ~/quicklisp/local-projects git clone https://github.com/y2q-actionman/hunchentoot-recycling-taskmaster.git #+end_src

#+begin_src common-lisp (ql:register-local-projects) ; Do if required (ql:quickload "hunchentoot-recycling-taskmaster") #+end_src

*** Running tests

#+begin_src common-lisp (ql:quickload "hunchentoot-recycling-taskmaster-test") (asdf:test-system '#:hunchentoot-recycling-taskmaster) #+end_src

** Starting/stopping server

You can use hunchentoot-recycling-taskmaster just by changing =hunchentoot:acceptor= to =hunchentoot-recycling-taskmaster:parallel-acceptor=, or =hunchentoot:easy-acceptor= to =hunchentoot-recycling-taskmaster:parallel-easy-acceptor=.

#+begin_src common-lisp (defparameter test-server (make-instance 'hunchentoot-recycling-taskmaster:parallel-easy-acceptor :port 4242)) (hunchentoot:start test-server) #+end_src

#+begin_src bash curl "http://127.0.0.1:4242/yo"

=> "Hey!"

#+end_src

To stop it, =hunchentoot:stop= can be used.

#+begin_src common-lisp (hunchentoot:stop test-server) #+end_src

See [[file:demo.lisp][demo.lisp]] for the sample codes above.

** API

These symbols are exported from =hunchentoot-recycling-taskmaster= package. Please see their docstring.

• [Class] =parallel-acceptor=

• [Class] =parallel-easy-acceptor=

• [Class] =parallel-ssl-acceptor=

• [Class] =parallel-easy-ssl-acceptor=

• [Class] =recycling-taskmaster=

• [Variable] =default-standby-thread-count=

• [Function] =abandon-acceptor=

• [Condition] =recycling-taskmaster-corrupted-error=

• How it works

This section shows how hunchentoot-recycling-taskmaster works comparing other implementations.

** Hunchentoot; make one thread per connection.

[[file:pics/architecture/hunchentoot-architecture.dot.png]]

Hunchentoot uses one thread per one connection. It means when a client use one keep-alive connection for multiple requests, Hunchentoot dedicate one thread to that connection. There is some delay for new connections, but it works well on keep-alive connections.

** quux-hunchentoot and cl-tbnl-gserver-tmgr; Thread pooling

[[file:pics/architecture/hunchentoot-thread-pooling.dot.png]]

These implementations utilize a thread pool around Hunchentoot. Instead of making a new thread for a new connection, they reuse threads kept in its thread pool, reducing latency for new connections.

However, their benchmarks don't show significant differences from the original Hunchentoot. I suspect this is for two reasons:

1. HTTP benchmarking tool, such as =wrk=, utilizes keep-alive connections. Effects of thread-pooling are limited at the beginning of benchmarking.
2. Their thread-pool's size may be fixed. They cannot increase threads like Hunchentoot even on high workloads.

** hunchentoot-recycling-taskmaster

[[file:pics/architecture/hunchentoot-recycling-taskmaster-architecture.dot.png]]

hunchentoot-recycling-taskmaster tries to take both, thread-pooling and changing the number of threads dynamically. In hunchentoot-recycling-taskmaster, threads don't only work on connected sockets but also some management tasks, such as accepting a new connection from the listen socket, creating a new thread, or terminating themselves.

For this management, all threads share the listen socket for synchronizing acceptance and track how many threads on it. Since using the listen socket, hunchentoot-recycling-taskmaster successfully implements these mechanisms without adding new dependencies to hunchentoot.

• Benchmark

** Running benchmarks To run benchmark by yourself, do below:

#+begin_src common-lisp (ql:quickload "hunchentoot-recycling-taskmaster-benchmark") (asdf:test-system '#:hunchentoot-recycling-taskmaster-benchmark) #+end_src

** My Environments

*** My machine

| cl function name | | |-----------------------------+-------------------------------------| | LISP-IMPLEMENTATION-TYPE | SBCL | | LISP-IMPLEMENTATION-VERSION | 2.2.9.debian | | MACHINE-TYPE | X86-64 | | MACHINE-VERSION | 13th Gen Intel(R) Core(TM) i7-1360P | | SOFTWARE-TYPE | Linux | | SOFTWARE-VERSION | 5.15.153.1-microsoft-standard-WSL2 |

*** Server library versions and parameters

| name | parameters | version | Git commit | |----------------------------------+------------------------+---------+------------------------------------------| | hunchentoot-recycling-taskmaster | standby-thread-count 8 | 0.0.1 | 82913edaf3f65afb189f0d72ddeb6339bf0499ae | | hunchentoot | | 1.3.1 | d1617e9d4eab6cb801c56cf36d9b0aab134fb7e6 | | quux-hunchentoot | | 1.0.2 | quux-hunchentoot-20211230-git | | cl-tbnl-gserver-tmgr | max-thread-count 8 | 0.1.1 | 1ae71c9324e876761cd1ee51768a34f0793e6879 | | wookie | | 0.3.15 | 1f74b6c24b463c1e6fff35377e477934f72bac20 | | woo | worker-num 8 | 0.12.0 | 7f5219c55d49190f5ae17b123a8729b31c5d706e |

*** benchmarking tool

I used =wrk= like below

#+begin_src bash

keep-alive

wrk -t 4 -c 100 -d 10 http://localhost:4242/yo

simulating no keep-alive

wrk "-H Connection: close" -t 4 -c 100 -d 10 http://localhost:4242/yo #+end_src

** Results

*** keep-alive, sleep 1ms

[[file:pics/benchmark-result-2025-12-26/sleep_1ms/keep-alive/100_connections_requests_sec__sleep_1ms__keep-alive__higher_is_better.svg]] [[file:pics/benchmark-result-2025-12-26/sleep_1ms/keep-alive/100_connections_latency(us)__sleep_1ms__keep-alive__lower_is_better.svg]]

On this benchmark, my HTTP handler always responds after 1ms to simulate some workloads.

Hunchentoot was quite fast. As mentioned above, Hunchentoot assigns one thread per connection and keeps using that thread until the connection is closed. Therefore, if a connection is kept alive, there's no delay from thread creation, and it doesn't become Hunchentoot slow. =wrk= uses keep-alive by default, so this result benefited from this.

cl-tbnl-gserver-tmgr was not very fast. This is presumably because it only assigns 8 threads as workers.

Woo was not fast even set to 8 threads. See [[#about-woo][About Woo]] below.

*** no keep-alive, sleep 1ms

[[file:pics/benchmark-result-2025-12-26/sleep_1ms/no-keep-alive/100_connections_requests_sec__sleep_1ms__no_keep-alive__higher_is_better.svg]] [[file:pics/benchmark-result-2025-12-26/sleep_1ms/no-keep-alive/100_connections_latency(us)__sleep_1ms__no_keep-alive__lower_is_better.svg]]

On this benchmark, HTTP handler still causes 1ms latency, and I simulated "no keep-alive" by adding =wrk= option ="-H Connection: close"= .

Hunchentoot became slow. In this test, Hunchentoot works like "one thread per one request" so latencies caused by creating a new thread affected the result.

hunchentoot-recycling-taskmaster is designed to work well in this situation.

*** keep-alive, sleep 0ms

[[file:pics/benchmark-result-2025-12-26/sleep_0ms/keep-alive/100_connections_requests_sec__sleep_0ms__keep-alive__higher_is_better.svg]] [[file:pics/benchmark-result-2025-12-26/sleep_0ms/keep-alive/100_connections_latency(us)__sleep_0ms__keep-alive__lower_is_better.svg]]

On this benchmark, I set HTTP handler's latency to 0ms. (However some latencies caused by small computations still exist. See [[#the-delay-that-i-overlooked][the section about this]] below.)

hunchentoot-recycling-taskmaster became fast. I think this is because threads are accepting connections in parallel.

*** no keep-alive, sleep 0ms

[[file:pics/benchmark-result-2025-12-26/sleep_0ms/no-keep-alive/100_connections_requests_sec__sleep_0ms__no_keep-alive__higher_is_better.svg]] [[file:pics/benchmark-result-2025-12-26/sleep_0ms/no-keep-alive/100_connections_latency(us)__sleep_0ms__no_keep-alive__lower_is_better.svg]]

On this benchmark, I set HTTP handler's latency to 0ms like above, and set =wrk= option ="-H Connection:close"= .

Here, Woo finally takes first place, with hunchentoot-recycling-taskmaster coming in a close second.

*** Other results

• See [[pics/benchmark-result-2025-12-26/benchmark-result-2025-12-26.org][benchmark-result-2025-12-26.org]] for graphs with "400 connections" parameters.
• See [[benchmark/benchmark-result-2025-12-26][this directory]] for raw data.

** About Woo

*** How to sleep?

Woo becomes significantly slow if the handler is even slightly delayed. With the following setup for the "1ms sleep" benchmark, I observed poor results:"

#+begin_src lisp (defparameter handler-sleep-seconds 0)

(defun handler-small-sleep () (sleep handler-sleep-seconds))

(woo:run (lambda (env) (declare (ignore env)) (handler-small-sleep) '(200 (:content-type "text/plain") ("Hello, World"))) :worker-num 8) #+end_src

This is because Woo is an async server that handles multiple connections simultaneously in a thread. If a delay occurs in the processing of one connection, all other connections in the same thread will be delayed. Given its async architecture, =sleep= like above is obviously discouraged.

In cases like this, you generally don't =sleep= inside the async server's event loop. You run time-consuming processing outside the event loop, and when it's finished, you notify the event loop of the content to be sent and received, or set up a callback to be called. [[benchmark/wookie.lisp][My code for benchmarking Wookie]] does it.

But for some reason, Woo doesn't seem to have such a mechanism. I couldn't find it. [[https://github.com/quickdocs/quickdocs-api/tree/master][quickdocs-api]], which is said to use Woo, do not seem to take such considerations into account. [[https://github.com/TechEmpower/FrameworkBenchmarks/pull/8313][This code]] is also.

(It may be a one reason of not good benchmark linked by [[https://www.reddit.com/r/Common_Lisp/comments/1506kqb/comment/js220a6/][this Reddit thread]].)

another one says "Woo has no async foundation", too: https://news.ycombinator.com/item?id=29019217

https://lisp-journey.gitlab.io/blog/why-turtl-switched-from-lisp-to-js/

https://killtheradio.net/technology/cl-async-non-blocking-asynchronous-programming-for-common-lisp/

https://www.reddit.com/r/lisp/comments/q4s32u/asynchronous_web_programming_in_cl/

Some people on [[https://news.ycombinator.com/item?id=29019217][here]] or [[https://www.reddit.com/r/Common_Lisp/comments/1hmxv4k/comment/m4feb4b/][here]] have said that "offloading is possible with lparallel", but I have yet to find any code that actually does this. The following naive code, which creates a thread in the handler, will result in an error.

#+begin_src common-lisp ;; Making a thread like below don't works because `woo.ev:evloop' is NIL. (defparameter woo-callback-threads-app (lambda (_env) (declare (ignore _env)) (lambda (callback) (bt:make-thread (lambda () (funcall callback '(200 (:content-type "text/plain") ("Hello, World"))))))))

(clack:clackup woo-callback-threads-app :server :woo) #+end_src

#+begin_src common-lisp ;; Binding like that does not works also. (let ((evloop woo.ev:evloop)) (bt:make-thread (lambda (&aux (woo.ev:evloop evloop)) (funcall callback '(200 (:content-type "text/plain") ("Hello, World")))))) #+end_src

This is because the =woo.ev:evloop= variable, which is the actual event loop, cannot be referenced in a newly created thread with =(make-thread)= and become =NIL=.

By the way, using lparallel like above will be "one-thread-per-request" model.

I think that will poorly perform than "one-thread-per-connection" of Hunchentoot.

This problem can be solved by directly handling =libev=, which Woo depends on, directly handling the event loop. Gemini-CLI wrote [[benchmark/about_woo/woo-timer-cb--written-by-gemini--working.lisp][this code]] to do so. It certainly works, and the benchmarks aren't bad. However, I don't know how this code works. (Please don't ask me.)

So, to benchmark "sleep 1ms" in Woo, I have no choice but to use the bad code above. This is a huge disadvantage to Woo, but it's unavoidable due to the lack of async support in it. This is also the way quickdocs-api is written.

*** The delay that I overlooked

Some people might look at the above [[#keep-alive-sleep-0ms][keep-alive sleep 0ms result]] and wonder "Why is Woo not as fast as people say?" I thought the same thing, investigated and found that the cause was that the handler definition for this benchmark calls =(handler-small-sleep)= defined above. Let's see it again:

#+begin_src common-lisp (defparameter handler-sleep-seconds 0)

(defun handler-small-sleep () (sleep handler-sleep-seconds)) #+end_src

This code does the following:

1. Eval the special variable (which becomes 0).
2. Call =cl:sleep= with 0 (which should return immediately).

These two processes should not take much time, but unfortunately they may cause performance problems in Woo. Changing =handler-small-sleep= as follows made it faster, as shown in the following graph.

#+begin_src common-lisp (defun handler-small-sleep () ) #+end_src

[[file:pics/benchmark-result-2025-12-26/sleep_0ms_no_special_vars/keep-alive/100_connections_requests_sec__sleep_commented_out__keep-alive__higher_is_better.svg]] [[file:pics/benchmark-result-2025-12-26/sleep_0ms_no_special_vars/keep-alive/100_connections_latency(us)__sleep_commented_out__keep-alive__lower_is_better.svg]]

I tried some tests after a day, resulted here (with =(defparameter handler-sleep-seconds 0)=) :

| | requests/sec | latency | |--------------------------------------------------------------------------+--------------+---------| | =(sleep handler-sleep-seconds)= | 44770.99 | 2.27ms | | =(sleep 0)= | 49158.22 | 2.06ms | | =handler-sleep-seconds= | 215447.83 | 0.95ms | | do nothing | 247789.97 | 1.00ms | | =(when (plusp handler-sleep-seconds) (sleep handler-sleep-seconds))= | 240050.33 | 1.26ms |

So, to get the most out of Woo's performance, I should not call =sleep= even with 0.

In other words, when using Woo I will be troubled by overlooking any little delays. I must examine handlers should work with very low latency. If Woo's situation doesn't change, it could be argued that Woo is very fast only when your handlers perform minimal computation. I think Woo is good for serving static pages

or microbenchmarking.

(but why did such a thing by Lisp? Use others, like nginx.)

• TODO TODO list

** register to systems

• [[http://blog.quicklisp.org/2026/01/january-2026-quicklisp-dist-update-now.html][Quicklisp added this]] before my request. Thanks!
• Ultralisp
• ocicl

** Support Lispworks

Because Hunchentoot on Lispworks does not hold a listen socket on its structure, hunchentoot-recycling-taskmaster cannot utilize it.

** Benchmark other servers

• conserv :: Causing =cl:type-error= when I used its sample code.
• house :: Very fragile. See [[benchmark/about_house/about_house.org][my memo]] .
• teepeedee2 :: cannot be loaded by my machine because of [[benchmark/loading.org]["heap exhausted" error]].

** Ideas

• Using atomic variables -- its impact is small on SBCL, large on Allegro CL. See [[/atomic-op-taskmaster]] in this repository.
• make the number of =standby-thread-count= to variadic.

• License

BSD 2-Clause, same as Hunchentoot. See [[file:LICENSE][LICENSE]].