[Prudctual]

Overview:

ThunderAgent was registering a synthetic public ModelDeploymentCard with a fresh ModelRuntimeConfig. Frontend admission therefore never saw the wrapped backend's runtime contract (token_budget, effective context length, engine capabilities), so over-limit prompts were admitted and then rejected after routing.

This PR copies the backend card's runtime_config (including all runtime_data) onto the proxy card before registration, then overlays CLI parser flags.

Details:

• After ThunderAgent subscribes to the wrapped worker's MDC stream, wait briefly for a backend card (preferring one that publishes token_budget).
• Inherit that card's runtime config wholesale: runtime_data via set_engine_specific, plus typed fields, structural-tag settings, and disagg bootstrap metadata.
• Overlay --dyn-tool-call-parser / --dyn-reasoning-parser / --publish-sglang-generate on top of the inherited contract.
• Forward the backend kv_cache_block_size on register_model.
• If no backend card appears, keep previous empty-config registration and log a warning.

This is the smallest fix for the reported admission hole. Removing ThunderAgent's public model registration entirely (the issue's longer-term direction) is left as follow-up; that would require invoking ThunderAgent as a post-preprocess router/flow-control policy.

Upstream compare (403 opening ai-dynamo/dynamo from this token):
https://github.com/ai-dynamo/dynamo/compare/main...Prudctual:dynamo:cursor/thunderagent-proxy-runtime-config-7cc2?expand=1

Where should the reviewer start?

• components/src/dynamo/thunderagent_router/proxy_card.py — inherit/select/wait helpers
• components/src/dynamo/thunderagent_router/__main__.py — _proxy_runtime_config() used at register_model
• components/src/dynamo/thunderagent_router/tests/test_proxy_card.py — token_budget preservation

Related Issues

🔗 This PR is linked to an issue:

• Closes [BUG]: ThunderAgent proxy registration drops backend model-card policy ai-dynamo/dynamo#14762

Summary

Proxy registration now inherits the backend ModelRuntimeConfig instead of publishing an incomplete synthetic card, so frontend token-budget admission matches the engine.

Validation

• pytest on components/src/dynamo/thunderagent_router/tests/test_proxy_card.py and test_capacity.py (22 passed; dynamo.llm stubbed in this environment).
• Overlay test in test_main.py asserts CLI parsers do not drop inherited token_budget.

  

---

Summary by cubic

Fixes ThunderAgent's public model card so frontend admission sees the backend's runtime contract (token_budget, context length, capabilities) instead of a synthetic empty config. Previously, over-limit prompts were admitted then rejected after routing. Closes ai-dynamo#14762.

Bug Fixes

• Inherits the wrapped worker's runtime config wholesale when registering the proxy model, including runtime_data and structural‑tag settings.
• Overlays --dyn-tool-call-parser, --dyn-reasoning-parser, and --publish-sglang-generate on top of the inherited contract.
• Forwards the backend kv_cache_block_size on register_model.
• Falls back to an empty config with a warning if no backend card appears within the wait window.

^{Written for commit e2e1479. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found across 6 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="components/src/dynamo/thunderagent_router/tests/test_main.py">

<violation number="1" location="components/src/dynamo/thunderagent_router/tests/test_main.py:303">
P2: This assertion does not verify that the inherited token budget is preserved. Compare the exact serialized value so parser overlays cannot silently replace it with an incorrect budget.</violation>
</file>

<file name="components/src/dynamo/thunderagent_router/proxy_card.py">

<violation number="1" location="components/src/dynamo/thunderagent_router/proxy_card.py:129">
P1: When a backend card contains a typed field without a writable Python binding, this assignment raises and the handler silently drops it. The proxy therefore does not inherit the full runtime contract; add support for these fields and carry LoRA capacity through registration instead of swallowing the failures.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: When a backend card contains a typed field without a writable Python binding, this assignment raises and the handler silently drops it. The proxy therefore does not inherit the full runtime contract; add support for these fields and carry LoRA capacity through registration instead of swallowing the failures.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At components/src/dynamo/thunderagent_router/proxy_card.py, line 129:

<comment>When a backend card contains a typed field without a writable Python binding, this assignment raises and the handler silently drops it. The proxy therefore does not inherit the full runtime contract; add support for these fields and carry LoRA capacity through registration instead of swallowing the failures.</comment>

<file context>
@@ -0,0 +1,159 @@
+        if key == "taints" and isinstance(value, (list, tuple, set)):
+            assigned = set(value)
+        try:
+            setattr(cfg, key, assigned)
+        except Exception as exc:
+            logger.warning(
</file context>

[cubic-dev-ai]

P2: This assertion does not verify that the inherited token budget is preserved. Compare the exact serialized value so parser overlays cannot silently replace it with an incorrect budget.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At components/src/dynamo/thunderagent_router/tests/test_main.py, line 303:

<comment>This assertion does not verify that the inherited token budget is preserved. Compare the exact serialized value so parser overlays cannot silently replace it with an incorrect budget.</comment>

<file context>
@@ -223,3 +227,78 @@ async def test_a_placed_program_does_not_consult_the_first_chunk():
+    assert runtime_cfg is overlay
+    assert runtime_cfg.tool_call_parser == "hermes"
+    assert runtime_cfg.reasoning_parser == "qwen"
+    assert runtime_cfg.get_engine_specific("token_budget") is not None
+    assert block_size == 64
</file context>

Suggested change

	assert runtime_cfg.get_engine_specific("token_budget") is not None
	assert runtime_cfg.get_engine_specific("token_budget") == (
	'{"combined_limit":131072,"reject_prompt_overflow":true,'
	'"reject_total_overflow":true}'
	)