You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Test Qwen3.8 27B and audit the native Forge handoff against a real Homebase task #111
Qwen3.8 27B Q4_K_M is installed with verified official layer digests. Evaluation complete; no production model switch, controller deployment or Homebase implementation was performed.
Native/generic trials under reasoning-off failed during source retrieval or model response. A separately labeled direct source-packet/structured-output diagnostic returned exact-source changes, but unchanged generated Python fails compilation. The final native low-reasoning trial239c2fd3df4b4f88bca5d465d9d45886 completed requested source reads, then hit the240-second deadline without a proposal. No usable model implementation accepted. These are bounded single-task trials, not a general coding benchmark.
The handoff audit found repeated JSON keys silently accepted in native results. Draft PR #113 fixes this;95targeted tests and both GitHub workflows pass, and restoring legacy parsing makes both new native regression cases fail. Issue #112 remains open for review/release. No fix deployed.
Foundry access returned after the earlier interruption. Host uptime exceeds4days; cause remains unknown. Final checks:SSH works,HomebaseHTTP200,forge_local ready,no models loaded,no unfinished live Forge tasks,deployed controller hashes and selected sources unchanged. Raw evidence recovered; no test remains blocked.
Recommendation:retain the current default. ReviewPR113 and simplify source delivery/task sizing/output contracts before further model promotion. Detailed results and all attempts are in comments and output/qwen38-handoff-20261117/REPORT.md.
Original scope
Requested by Mike on2026-11-17 after #110:install/testQwen3.8,audit the real MCP/native handoff, distinguish delivery/schema/source gates from independent correctness, and keep production/default routing unchanged. Real task:Homebase host-selection settings (kudama/homelab#14). No worker writes,command tools or credential grants.
2026-11-17 interim audit:86 existing handoff/native/receipt tests pass. Actual MCP bridge smoke against an isolated native controller rejects incorrect source hashes before inference, rejects duplicate orders and returns immutable snapshot provenance with review_required. Existing Qwen3-Coder native task65001ace8cfb41fe868dd9c9f80d2be2 failed tool_denied after requesting100 lines from the1-line test seed; no extra source returned. This is a protocol failure, not a reviewed code-quality result. Native duplicate-key acceptance was reproduced and a focused fix is now draft PR #113 / issue #112:95 checks pass, both new native cases fail under restored legacy parsing, and both GitHub workflows passed. Deployed controller/defaults remain unchanged. Qwen3.8 verified-layer download and model trials are still pending.
2026-11-17 model/handoff results saved; final diagnostic blocked by Foundry reachability.
Installed Qwen3.8 27B Q4_K_M with verified official model/projector and metadata digests. Production routing and Homebase deployment unchanged.
Native MCP trials on isolated controller code:
Current coder65001ace8cfb41fe868dd9c9f80d2be2: tool_denied, requested100 lines from1-line seed.
Qwen3.8 task92dde56c9c5c491aba4141df1cc18602: tool_limit_exceeded at10 calls after continued selected-source inspection.
Generic guided Qwen3.8 e819017b7a7248f8b4565c9ef00155ea: tool_denied on settings.py211..320 (>80lines).
-12-call native follow-up9d2d2a32b5714111a1d93c0bdfb1d4c1: invalid_model_response before usable tool output.
A separately labeled direct source-packet + structured-output coding diagnostic returned valid shape and exact-source replacements in106.06s, but unchanged generated Python fails compilation (unmatched parentheses in public_config). Static review also finds its host-selection expression always returns None. Both generated and independent tests fail at import; no model implementation accepted. This direct diagnostic is not a native handoff pass.
Handoff fix: draft PR #113 / issue #112;95 targeted checks pass, both new native cases fail with legacy parsing, and both GitHub workflows pass. No fix deployed.
Current adapter forces reasoning off. Installed metadata supports false/low/medium/xhigh, default medium. Final low-profile trial was not submitted: a runner-label guard prevented overwriting an existing record; label corrected locally, then Foundry stopped accepting SSH/Homebase connections and forge_local became unavailable. This harness/availability failure is not scored as a model failure. User was asked whether Foundry is restarting.
Recommendation so far: no model promotion; fix the verified handoff defect and improve source delivery/configuration before further model swaps. Remaining: restore host reachability, retrieve raw remote records, run corrected low-reasoning native check and verify host health. Evidence retained in output/qwen38-handoff-20261117/REPORT.md and review-summary.json; raw native records remain in the private Foundry evaluation directory.
2026-11-18: user confirmed Foundry should be online. Rechecked: authenticated Forge health and SSH now respond; model inventory retains Qwen3.8. Reconciled evaluation state: no existing low-profile attempt record, no unfinished live Forge tasks, no loaded model. Uploaded the corrected isolated runner and resumed the final low-reasoning native handoff check with unchanged source permissions/default routing. Prior outage cause remains unconfirmed.
2026-11-18 evaluation completed after connectivity returned. Foundry uptime exceeds4days; no recent reboot was observed and the earlier access-failure cause remains unconfirmed.
Final isolated low-reasoning native MCP trial239c2fd3df4b4f88bca5d465d9d45886 used the same pinned Homebase source,12read-only calls,16K context,4096output and240-second deadline. It completed the five requested source reads, then failed deadline_exceeded at240.57s without a final proposal. Bad-source admission was rejected before inference, duplicate order rejected, and source hashes remained unchanged. The profile changed think to the model-supported low setting in a private controller copy only.
Conclusion: Qwen3.8 is installed, but no usable uncorrected implementation was accepted across these trials. Do not promote it on this evidence. Current handoff/tool budgeting/configuration and independent code validation need improvement; results are limited single-task trials, not proof about every possible model profile.
The verified native duplicate-JSON-key defect has a concrete draft fix in PR #113 / issue #112:95targeted checks and both GitHub workflows pass, with legacy-mutation reproduction. No fix or model routing change deployed.
Final health:SSH reachable,HomebaseHTTP200,forge_local ready,no models left loaded,no unfinished live Forge tasks,deployed app.py/work_orders.py hashes unchanged. Raw remote evidence recovered to the private Forge workspace output/qwen38-handoff-20261117; REPORT.md and review-summary.json updated. No remaining test is blocked. Next action:reviewPR113 and improve/simplify the handoff before another default-model decision.
Activity
kudama commentedon Nov 18, 2026
2026-11-17 interim audit:86 existing handoff/native/receipt tests pass. Actual MCP bridge smoke against an isolated native controller rejects incorrect source hashes before inference, rejects duplicate orders and returns immutable snapshot provenance with review_required. Existing Qwen3-Coder native task65001ace8cfb41fe868dd9c9f80d2be2 failed tool_denied after requesting100 lines from the1-line test seed; no extra source returned. This is a protocol failure, not a reviewed code-quality result. Native duplicate-key acceptance was reproduced and a focused fix is now draft PR #113 / issue #112:95 checks pass, both new native cases fail under restored legacy parsing, and both GitHub workflows passed. Deployed controller/defaults remain unchanged. Qwen3.8 verified-layer download and model trials are still pending.
kudama commentedon Nov 18, 2026
2026-11-17 model/handoff results saved; final diagnostic blocked by Foundry reachability.
Installed Qwen3.8 27B Q4_K_M with verified official model/projector and metadata digests. Production routing and Homebase deployment unchanged.
Native MCP trials on isolated controller code:
-12-call native follow-up9d2d2a32b5714111a1d93c0bdfb1d4c1: invalid_model_response before usable tool output.
A separately labeled direct source-packet + structured-output coding diagnostic returned valid shape and exact-source replacements in106.06s, but unchanged generated Python fails compilation (unmatched parentheses in public_config). Static review also finds its host-selection expression always returns None. Both generated and independent tests fail at import; no model implementation accepted. This direct diagnostic is not a native handoff pass.
Handoff fix: draft PR #113 / issue #112;95 targeted checks pass, both new native cases fail with legacy parsing, and both GitHub workflows pass. No fix deployed.
Current adapter forces reasoning off. Installed metadata supports false/low/medium/xhigh, default medium. Final low-profile trial was not submitted: a runner-label guard prevented overwriting an existing record; label corrected locally, then Foundry stopped accepting SSH/Homebase connections and forge_local became unavailable. This harness/availability failure is not scored as a model failure. User was asked whether Foundry is restarting.
Recommendation so far: no model promotion; fix the verified handoff defect and improve source delivery/configuration before further model swaps. Remaining: restore host reachability, retrieve raw remote records, run corrected low-reasoning native check and verify host health. Evidence retained in output/qwen38-handoff-20261117/REPORT.md and review-summary.json; raw native records remain in the private Foundry evaluation directory.
kudama commentedon Nov 18, 2026
2026-11-18: user confirmed Foundry should be online. Rechecked: authenticated Forge health and SSH now respond; model inventory retains Qwen3.8. Reconciled evaluation state: no existing low-profile attempt record, no unfinished live Forge tasks, no loaded model. Uploaded the corrected isolated runner and resumed the final low-reasoning native handoff check with unchanged source permissions/default routing. Prior outage cause remains unconfirmed.
kudama commentedon Nov 18, 2026
2026-11-18 evaluation completed after connectivity returned. Foundry uptime exceeds4days; no recent reboot was observed and the earlier access-failure cause remains unconfirmed.
Final isolated low-reasoning native MCP trial239c2fd3df4b4f88bca5d465d9d45886 used the same pinned Homebase source,12read-only calls,16K context,4096output and240-second deadline. It completed the five requested source reads, then failed deadline_exceeded at240.57s without a final proposal. Bad-source admission was rejected before inference, duplicate order rejected, and source hashes remained unchanged. The profile changed think to the model-supported low setting in a private controller copy only.
Conclusion: Qwen3.8 is installed, but no usable uncorrected implementation was accepted across these trials. Do not promote it on this evidence. Current handoff/tool budgeting/configuration and independent code validation need improvement; results are limited single-task trials, not proof about every possible model profile.
The verified native duplicate-JSON-key defect has a concrete draft fix in PR #113 / issue #112:95targeted checks and both GitHub workflows pass, with legacy-mutation reproduction. No fix or model routing change deployed.
Final health:SSH reachable,HomebaseHTTP200,forge_local ready,no models left loaded,no unfinished live Forge tasks,deployed app.py/work_orders.py hashes unchanged. Raw remote evidence recovered to the private Forge workspace output/qwen38-handoff-20261117; REPORT.md and review-summary.json updated. No remaining test is blocked. Next action:reviewPR113 and improve/simplify the handoff before another default-model decision.