Skip to content

refactor(server): make dispatch routing and failure paths explicit - #4036

Open
hubcio wants to merge 2 commits into
masterfrom
server-ng-pr4-spine-mechanisms
Open

refactor(server): make dispatch routing and failure paths explicit#4036
hubcio wants to merge 2 commits into
masterfrom
server-ng-pr4-spine-mechanisms

Conversation

@hubcio

@hubcio hubcio commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

The TCP dispatch funnel hid its rules in control flow: seven
ordered if-return probes where the order was the only spec, two
rewrite chains written out inline in two places, twenty send
sites each picking a failure channel by whichever builder they
called, two client-request handlers on shard 0 with separate
queues, and a read gate that let unknown codes fall through to a
catch-all.

Each is now written down once. classify() returns a RequestClass
that handle_client_request matches on, so the order of checks
inside classify is the routing. rewrite.rs holds both rewrite
chains as pure functions. dispatch/failure.rs holds the failure
channel table and the one exit every host-built frame takes. Each
shard builds one client-request handler that shard 0's transports
share, dropping the duplicate queues, the double hook install and
a reference cycle that leaked the shard. The read gate now
decides every code in the command table instead of passing
unlisted ones to the builder's empty-ok. Tests pin the probe
order, each channel's bytes, and every table entry.

Removing the cycle exposed a shutdown bug it had hidden: shard 0
owns the only write handle to the metadata state machine and
could drop it while peers were still reading, panicking their
pumps. Peers now count themselves out once their runtime is gone,
and shard 0 waits for that, bounded by shutdown_join_timeout.

The TCP dispatch funnel hid its rules in control flow: seven
ordered if-return probes where the order was the only spec, two
rewrite chains written out inline in two places, twenty send
sites each picking a failure channel by whichever builder they
called, two client-request handlers on shard 0 with separate
queues, and a read gate that let unknown codes fall through to a
catch-all.

Each is now written down once. classify() returns a RequestClass
that handle_client_request matches on, so the order of checks
inside classify is the routing. rewrite.rs holds both rewrite
chains as pure functions. dispatch/failure.rs holds the failure
channel table and the one exit every host-built frame takes. Each
shard builds one client-request handler that shard 0's transports
share, dropping the duplicate queues, the double hook install and
a reference cycle that leaked the shard. The read gate now
decides every code in the command table instead of passing
unlisted ones to the builder's empty-ok. Tests pin the probe
order, each channel's bytes, and every table entry.

Removing the cycle exposed a shutdown bug it had hidden: shard 0
owns the only write handle to the metadata state machine and
could drop it while peers were still reading, panicking their
pumps. Peers now count themselves out once their runtime is gone,
and shard 0 waits for that, bounded by shutdown_join_timeout.
@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Sep 2, 2026
The consumer-group Join/Leave rewrite failed silently: the funnel
logged and returned, sending nothing. Its only error is an
undecodable body, so there is nothing for the client to replay and
the SDK's lockstep connection waits out its read timeout before it
learns anything. Send the typed pre-consensus deny the rest of the
funnel already uses.

Rename FailureChannel to FrameChannel while the type is still new.
It labels every host-built frame, success replies included, so the
old name contradicted most of its call sites.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYs1ovuT73HVJHXhW9b5at
@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.69231% with 70 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.21%. Comparing base (e6ec869) to head (f5da637).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
core/server/src/dispatch/session_ops.rs 70.96% 18 Missing ⚠️
core/server/src/dispatch/partition.rs 80.00% 16 Missing and 1 partial ⚠️
core/server/src/rewrite.rs 95.17% 12 Missing and 2 partials ⚠️
core/server/src/dispatch/mod.rs 97.40% 11 Missing and 1 partial ⚠️
core/server/src/boot/threads.rs 98.12% 2 Missing and 1 partial ⚠️
core/server/src/boot/mod.rs 90.90% 2 Missing ⚠️
core/server/src/dispatch/failure.rs 99.50% 1 Missing and 1 partial ⚠️
core/server/src/dispatch/authz.rs 98.94% 1 Missing ⚠️
core/server/src/http/submit.rs 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #4036      +/-   ##
============================================
+ Coverage     85.12%   85.21%   +0.09%     
  Complexity     1402     1402              
============================================
  Files          1236     1238       +2     
  Lines        181607   182489     +882     
  Branches     147900   148782     +882     
============================================
+ Hits         154596   155514     +918     
+ Misses        22957    22911      -46     
- Partials       4054     4064      +10     
Components Coverage Δ
Rust Core 86.15% <95.69%> (+0.10%) ⬆️
Java SDK 67.29% <ø> (ø)
C# SDK 75.46% <ø> (+0.07%) ⬆️
Python SDK 90.06% <ø> (ø)
PHP SDK 85.65% <ø> (ø)
Node SDK 96.15% <ø> (-0.09%) ⬇️
Go SDK 69.35% <ø> (+0.06%) ⬆️
Files with missing lines Coverage Δ
core/server/src/boot/recovery.rs 96.01% <100.00%> (+0.05%) ⬆️
core/server/src/dispatch/reads.rs 90.06% <100.00%> (+0.87%) ⬆️
core/server/src/dispatch/test_support.rs 83.33% <100.00%> (+0.87%) ⬆️
core/server/src/http/handlers.rs 79.53% <ø> (ø)
core/server/src/dispatch/authz.rs 98.40% <98.94%> (+3.63%) ⬆️
core/server/src/http/submit.rs 87.09% <50.00%> (-0.85%) ⬇️
core/server/src/boot/mod.rs 86.91% <90.90%> (+0.01%) ⬆️
core/server/src/dispatch/failure.rs 99.50% <99.50%> (ø)
core/server/src/boot/threads.rs 82.52% <98.12%> (+5.97%) ⬆️
core/server/src/dispatch/mod.rs 96.51% <97.40%> (+4.24%) ⬆️
... and 3 more

... and 47 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review PR is waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant