Server Modes

Server Modes

CWIST has two request paths, tuned for opposite goals. The mode is chosen at cwist_app_listen() time by CWIST_C1M_MODE (or the c1m_override argument of cwist_app_listen_ex()); application code does not change.

	CWIST reactor (default)	CWIST Classic pool
Select with	CWIST_C1M_MODE unset, 1 or true	CWIST_C1M_MODE=0 or false
Connection model	event-driven: one reactor thread multiplexes many connections	one thread per accepted connection (on-demand, detached, 256 KiB stack)
Strength	connection density and throughput; idle connections cost a reactor slot, not a thread	lowest median latency for short bursts; no request waits behind another in a batch
Idle keep-alive cost	about 24 KB of server memory per held connection (measured at C1M)	a parked thread and its stack
Practical ceiling	C1M (1,000,000 held connections measured)	C100K with raised task limits; C1M exceeds threads-max
Blocking WebSocket API (cwist_app_ws)	yes	yes
Callback WebSocket API (cwist_app_ws_async)	yes	routes registered only this way answer 501

The reactor

On Linux the reactor uses io_uring through raw syscalls (no liburing). By default it arms one-shot IORING_OP_POLL_ADD requests and the woken worker does the I/O inline. On reactors with a real ring, the HTTP receive wait can instead be a single IORING_OP_RECV whose completion stages the bytes; a per-connection flag keeps clients that do not pipeline on the poll path. If io_uring setup fails, the reactor falls back to epoll; on macOS and BSD it uses kqueue. CWIST_REACTOR_BACKEND=epoll forces epoll for comparisons.

Reactor behavior worth knowing:

• Pipelined HTTP/1.1 requests are served in batches of CWIST_HTTP_BATCH (default 16, max 1024) per event-loop turn and yield every CWIST_HTTP_YIELD_BATCH requests; responses served in one turn are coalesced into one write (up to 256 KiB).
• A handler that writes faster than a slow client reads parks the unsent bytes on a one-shot POLLOUT slot instead of blocking the reactor thread (deferred completions); synchronous responses still wait inside the send path for at most CWIST_HTTP_TIMEOUT_MS (30 s).
• Continuations that cannot be posted because the reactor queue is full are shed and their connections closed; the count is exported as cwist_http_continuation_shed_total in /metrics.
• Deferred responses (Async Handlers) complete on the connection's owner reactor thread.

Design and measurement notes: HTTP Reactor Fairness, Cooperative Queuing, Reactor Wakeup Regression.

The classic pool

Every accepted connection gets its own detached thread, created on demand, so the number of held connections scales with the task budget rather than a fixed pool. Requests run to completion on that thread. CWIST_POOL_PREWARM starts threads ahead of a connection ramp and CWIST_POOL_IDLE_TIMEOUT_MS (default 2000) controls how long an idle pool thread lingers. See Classic Pool Starvation.

C100K on the classic path needs task headroom (pids.max / TasksMax above 100K) and roughly 26 GB of virtual commit for 100K x 256 KiB stacks.

Worker processes

Independently of the mode, cwist_app_listen() forks CWIST_WORKERS worker processes before creating any threads. The default is one worker per online CPU core; CWIST_WORKERS=1 serves in the calling process without forking. Workers share the listening port through SO_REUSEPORT, and TLS workers share session-ticket keys so a session resumes on any worker.

cwist_app_listen_ex(app, port, workers, c1m) sets both explicitly: workers > 0 overrides CWIST_WORKERS and c1m is 1 (reactor), 0 (classic) or negative (read CWIST_C1M_MODE). With the reactor and one worker, every handler thread is joined before the call returns.

Profiles

CWIST_PROFILE fills in defaults for the tuning variables before the server starts. Variables you set explicitly always win.

CWIST_PROFILE	Sets
unset, empty or default	CWIST_C1M_MODE=1, CWIST_REACTOR_DRAIN_CHUNK=8
performance	CWIST_C1M_MODE=1, CWIST_MALLOC_ARENA_MAX=1, CWIST_REACTOR_DRAIN_CHUNK=8, CWIST_HTTP_BATCH=64
lowmem	CWIST_C1M_MODE=1, CWIST_MALLOC_ARENA_MAX=1, CWIST_WORKERS=2
lowlat	CWIST_C1M_MODE=0

An unknown value prints a warning and uses the defaults. cwist_apply_profile() applies the same logic and is called by cwist_app_listen().

Measured scale

Loopback, AMD Ryzen 5 5600X, CWIST_WORKERS=12, measured 2026-09-29:

Scale	Reactor	Classic pool
C10K	10,000 / 10,000 responded, ~0.9 s, ~301 MiB peak RSS	10,000 / 10,000, ~1.0 s, ~976 MiB
C100K	100,000 / 100,000, ~12.1 s, ~2.35 GiB	100,000 / 100,000, ~12.8 s, ~9.0 GiB
C1M	1,000,000 / 1,000,000 (8 x 125K clients), ~27 s per client process, ~22.9 GiB	out of reach (thread limit)

Kernel settings for C100K and above are in C1M File Limits. CI throughput and latency figures are in the repository README and Benchmark Methodology.

Which to pick

Start with the reactor. It suits APIs behind a reverse proxy, web pages, webhooks, SSE fan-out, WebSocket-scale chat, long polling and anything that holds many mostly idle connections. Switch to the classic pool only when the workload is short request bursts where the lowest median latency matters more than connection density, and one thread per active connection is acceptable.