[Prudctual]

Overview:

QUIC (DYN_RESPONSE_PLANE=quic) could not carry a worker's ErrorType on a pre-stream refusal. The frontend therefore answered HTTP 500 and fault detection quarantined the worker that had correctly refused the request. TCP already maps the same refusal to 400 once the typed prologue exists; this PR extends that path to QUIC with a mixed-version Error-frame encoding.

Details:

QuicResponseSender sent FrameKind::Error as raw UTF-8, so ResponsePublisher::send_prologue_typed had nowhere to put a DynamoError. The requesting side re-synthesized ErrorType::CannotConnect with no cause, which the frontend maps to a sanitized 500 and is_inhibited treats as a worker fault.

This PR:

• Adds StreamPrologueError and an optional typed_error on ResponseStreamPrologue (skipped when absent so an old sender still handshakes).
• Encodes a typed QUIC error as JSON ResponseStreamPrologue. Untyped errors stay raw UTF-8. Receivers accept both: JSON with a present error field recovers the type; anything else, including a Python {"message","code"} envelope, is the legacy message. Protocol version stays 3 — a bump would force a lockstep upgrade for a payload the old decoder already accepts as UTF-8.
• Implements send_prologue_typed on both QUIC and TCP so the worker's type survives whichever response plane is configured.
• Attaches a non-migration-sensitive typed error as the cause of the existing outer CannotConnect, so HTTP classification sees InvalidArgument while retry classification of the outer type is unchanged.
• Excludes InvalidArgument / Backend(InvalidArgument) from is_inhibited, so a clean client refusal does not quarantine the worker. An untyped pre-stream failure still does.

TCP untyped behaviour is unchanged: an old worker that sends only display text still yields 500 and is still treated as a worker fault.

Public API: StreamProvider<T> widens its error from String to StreamPrologueError. StreamPrologueError derefs to str, so err.contains(..), err.len(), and &err[..] keep compiling. This overlaps the runtime plumbing in ai-dynamo#14467; this PR is self-contained against current main so QUIC can land without waiting on that review. If ai-dynamo#14467 merges first, this rebases onto the shared types.

Where should the reviewer start?

• lib/runtime/src/pipeline/network/quic_response.rs — encode_error_payload / decode_error_payload and send_prologue_typed
• lib/runtime/src/pipeline/network.rs — ResponseStreamPrologue.typed_error and StreamPrologueError
• lib/runtime/src/pipeline/network/egress/addressed_router.rs — pre_stream_failure_error
• lib/runtime/src/pipeline/network/egress/push_router.rs — is_inhibited client-refusal exclude

Validation

• cargo fmt --all -- --check
• cargo clippy -p dynamo-runtime --lib --all-targets -- -D warnings
• cargo test -p dynamo-runtime --lib pipeline::network::quic_response::tests — 26 passed (includes typed Error-frame round trip, live send_prologue_typed → InvalidArgument, and legacy UTF-8)
• cargo test -p dynamo-runtime --lib pipeline::network::tcp::server::tests — 40 passed (TCP typed prologue survives; untyped/invalid-prologue paths unchanged)
• cargo test -p dynamo-llm --lib test_pre_stream_refusal_surfaces_as_400 test_pre_stream_connect_failure_still_surfaces_as_500 — typed pre-stream refusal → HTTP 400 with the worker's reason; untyped / engine-shutdown still 500
• typed_client_refusal_does_not_quarantine_the_worker — CannotConnect + InvalidArgument cause is not inhibited; untyped CannotConnect still is

Related Issues

🔗 This PR is linked to an issue:

• Fixes QUIC response plane cannot carry typed pre-stream errors, so refusals regress to 500 ai-dynamo/dynamo#14750
• Relates to fix(runtime): return 400, not 500, when a backend refuses a request it cannot serve ai-dynamo/dynamo#14467 (TCP typed prologue; this PR is the QUIC follow-up and is self-contained against current main)

AI disclosure

This change was drafted with assistance from Cursor Grok 4.6. I reviewed the wire-format compatibility, the quarantine/is_inhibited interaction, and the test results before opening the PR.

  

---

Summary by cubic

Carries the worker's typed error on the QUIC response plane so a clean client refusal gets HTTP 400 instead of HTTP 500 and is no longer treated as a worker fault. Untyped pre-stream failures still surface as HTTP 500 and quarantine the worker.

Note: this branch is far behind main and has not been rebased — the diff carries ~150 unrelated commits. Rebase before merging; only the QUIC typed-error work below belongs to this PR.

• Typed QUIC errors travel as JSON tagged with dynamo.quic.typed_error; anything untagged (legacy UTF-8, Python {"message","code"} envelopes, decoy {"error","code"} objects) stays raw UTF-8 so it can't be misparsed as a prologue. Protocol version stays 3.
• Client refusals (InvalidArgument / Backend(InvalidArgument)) are excluded from is_inhibited, and NON_MIGRATABLE is pinned to the same set so an attached cause never changes migration behavior.

Migration

• StreamProvider<T>'s error type widens from String to StreamPrologueError; it derefs to str, so .contains(), .len(), and indexing keep compiling.

^{Written for commit cfbd80c. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found across 10 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="lib/runtime/src/pipeline/network/egress/push_router.rs">

<violation number="1" location="lib/runtime/src/pipeline/network/egress/push_router.rs:64">
P2: The `CLIENT_REFUSAL` exclude set makes `is_inhibited` return `false` if *any* link in the chain is `InvalidArgument`, not just the outer pre-stream refusal. `is_inhibited` is also applied to mid-stream errors at lines 1948 and 1978, so any genuine worker fault (e.g. `CannotConnect`, `StreamIncomplete`, `ResponseTimeout`) whose cause chain happens to contain an `InvalidArgument` anywhere will now escape quarantine and keep being reselected. The pre-stream typed-refusal case is intended, but this suppresses inhibition for the whole chain regardless of the true outermost fault.</violation>
</file>

<file name="lib/runtime/src/pipeline/network/ingress/push_handler.rs">

<violation number="1" location="lib/runtime/src/pipeline/network/ingress/push_handler.rs:766">
P2: When the failure has no worker `DynamoError`, this always supplies `Some(ErrorType::Unknown)`, causing QUIC to emit typed JSON instead of the legacy raw UTF-8 error. Return an optional typed error and use `StreamPrologueError::from_message` when no worker error was recovered.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: When the failure has no worker DynamoError, this always supplies Some(ErrorType::Unknown), causing QUIC to emit typed JSON instead of the legacy raw UTF-8 error. Return an optional typed error and use StreamPrologueError::from_message when no worker error was recovered.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At lib/runtime/src/pipeline/network/ingress/push_handler.rs, line 766:

<comment>When the failure has no worker `DynamoError`, this always supplies `Some(ErrorType::Unknown)`, causing QUIC to emit typed JSON instead of the legacy raw UTF-8 error. Return an optional typed error and use `StreamPrologueError::from_message` when no worker error was recovered.</comment>

<file context>
@@ -729,7 +761,11 @@ where
-                    let _ = publisher.send_prologue(Some(error_string)).await;
+                    let prologue_error = StreamPrologueError::new(
+                        error_string,
+                        typed_error_from_pipeline_error(&error),
+                    );
+                    let _ = publisher.send_prologue_typed(Some(prologue_error)).await;
</file context>

[cubic-dev-ai]

P2: The CLIENT_REFUSAL exclude set makes is_inhibited return false if any link in the chain is InvalidArgument, not just the outer pre-stream refusal. is_inhibited is also applied to mid-stream errors at lines 1948 and 1978, so any genuine worker fault (e.g. CannotConnect, StreamIncomplete, ResponseTimeout) whose cause chain happens to contain an InvalidArgument anywhere will now escape quarantine and keep being reselected. The pre-stream typed-refusal case is intended, but this suppresses inhibition for the whole chain regardless of the true outermost fault.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At lib/runtime/src/pipeline/network/egress/push_router.rs, line 64:

<comment>The `CLIENT_REFUSAL` exclude set makes `is_inhibited` return `false` if *any* link in the chain is `InvalidArgument`, not just the outer pre-stream refusal. `is_inhibited` is also applied to mid-stream errors at lines 1948 and 1978, so any genuine worker fault (e.g. `CannotConnect`, `StreamIncomplete`, `ResponseTimeout`) whose cause chain happens to contain an `InvalidArgument` anywhere will now escape quarantine and keep being reselected. The pre-stream typed-refusal case is intended, but this suppresses inhibition for the whole chain regardless of the true outermost fault.</comment>

<file context>
@@ -54,7 +54,14 @@ fn is_inhibited(err: &(dyn std::error::Error + 'static)) -> bool {
+        ErrorType::InvalidArgument,
+        ErrorType::Backend(BackendError::InvalidArgument),
+    ];
+    match_error_chain(err, INHIBITED, CLIENT_REFUSAL)
 }
 
</file context>

[cubic-dev-ai]

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="lib/runtime/src/pipeline/network/quic_response.rs">

<violation number="1" location="lib/runtime/src/pipeline/network/quic_response.rs:300">
P2: An untyped JSON message containing the reserved `kind` and `error` fields is misclassified as the tagged envelope, even without a typed error. Require `typed_error` to be present before taking this branch so raw untyped messages remain byte-for-byte unchanged.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: An untyped JSON message containing the reserved kind and error fields is misclassified as the tagged envelope, even without a typed error. Require typed_error to be present before taking this branch so raw untyped messages remain byte-for-byte unchanged.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At lib/runtime/src/pipeline/network/quic_response.rs, line 300:

<comment>An untyped JSON message containing the reserved `kind` and `error` fields is misclassified as the tagged envelope, even without a typed error. Require `typed_error` to be present before taking this branch so raw untyped messages remain byte-for-byte unchanged.</comment>

<file context>
@@ -244,46 +244,65 @@ impl Frame {
-        && let Ok(prologue) = serde_json::from_slice::<ResponseStreamPrologue>(payload)
-        && let Some(message) = prologue.error
+        && let Ok(wire) = serde_json::from_slice::<QuicTypedErrorWire>(payload)
+        && wire.kind == QUIC_TYPED_ERROR_KIND
+        && let Some(message) = wire.prologue.error
     {
</file context>

Suggested change

	&& wire.kind == QUIC_TYPED_ERROR_KIND
	&& wire.kind == QUIC_TYPED_ERROR_KIND
	&& wire.prologue.typed_error.is_some()