Skip to content

feat(orchestrator): PolicyLoop — SELT self-healing control layer over… - #1

Open
dfu925-ux wants to merge 113 commits into
mainfrom
feature/policy-loop
Open

feat(orchestrator): PolicyLoop — SELT self-healing control layer over…#1
dfu925-ux wants to merge 113 commits into
mainfrom
feature/policy-loop

Conversation

@dfu925-ux

Copy link
Copy Markdown
Owner

Co to robi

Warstwa PolicyLoop — samonaprawczy pipeline SELT nad niezmienionym Orchestrator.run_loop/4. Wpięcie przez huki, bez przepisywania jądra routingu TRINITY.

Wdrożone prace badawcze

  • SELT (arXiv:2506.07557) — tryby :learn/:think/:mimic/:recite jako struktury promptu (Mode).
    • Control-R (arXiv:2506.00189) — Reasoning Control Fields → budżet tokenów per stan + meta-prompt.
    • AIRL-S/PRM (arXiv:2508.14313) — agregacja PRM-Min (najsłabsze ogniwo).
    • LLaMA-Berry/PPRM (NAACL 2025) — relatywne ocenianie kandydatów (drift guard).

Bezpieczeństwo

  • AuditGate z 3 rozdzielnymi licznikami (trinity_turns / self_heal / citeclaw); twardy stop HITL przez {:shutdown, {:audit_gate, _}} pod supervisorem :transient.
    • Izolowany Executor: Docker + opcjonalny gVisor (--net none, --cap-drop=ALL, --read-only, --pids-limit).

Zmiany w jądrze (addytywne, wstecznie kompatybilne)

  • RoleInjector.inject_role/3 (stare /2 deleguje).
    • Orchestrator: prompt_directives w run_ctx + użycie /3.

Testy

  • 4 testy integracyjne PolicyLoop — zielone (sukces SELT, self-heal+regresja, trip safe-gate).
    • Pełny zestaw: 306 testów przechodzi. (3 lokalne failures w Parity.Check to artefakt CRLF z klonu na Windows — niezwiązane z tym PR, przechodzą na Linuksie/CI.)

dfu925-ux added 30 commits June 24, 2026 18:13
- do_turn: Keyword.split route_opts -> routing keys to CoordinationHead.route,
  generation keys (:max_tokens budget, :temperature, :reference_required)
  forwarded to agent dispatch. Fixes ArgumentError (unknown :max_tokens /
  temperature must be positive) surfaced by live run_loop.
- inference.ex: explicit per-call :inference_max_tokens/:inference_temperature
  now override the pool spec -> Control-R budget actually reaches the LLM API.
- scripts/phase6_scopeB_e2e.exs: full PolicyLoop E2E (host router + live Qwen,
  stubbed Executor). Validates SELT loop + AuditGate end-to-end.

mix test: no new regressions (306, 4 pre-existing env failures). Scope B: {:ok}.
…ct; <thought>/fence-safe extractor + unit test
…scoring; fix mock test for worker-response contract
…k) + fix B worker pool qwen-max->qwen-coder-plus + GLM discrimination test (3/3)
…— blind 47/54, oracle 48/54, zero false-accepts
…48->54/54 (87%->100%), A 42->52/54 (78%->96%). Sufit zniesiony - waskie gardlo bylo Workerem. B 44/54 (81,5%): petla<goly worker, GLM-verifier teraz waskim gardlem (+403 FreeTierOnly zanieczyszcza pomiar).
…e_fun). Napisane przez Qwen Worker, audyt Claude Core. 7 tests 0 failures. Marker [[cite:ID]], uchwyty nie payload (Konst. par.5), fail-closed (par.7). Zakres L1 (istnienie cytowania), NIE L2 (entailment).
…aw MCP client (Content-Length framing, fail-closed). E2E: real DOI verified, fake DOI rejected. 24 tests 0 failures.
…ugh reference gate; 8b regex rewritten pattern-engine-free (source policy); TRINITY_SANDBOX_RUNTIME moved to config/runtime.exs boundary; scripts/ excluded from source policy scan
…ter + injectable entailment seams, fail-closed, opt-in via retrieval_fun/entailment_fun. Written by Worker subagent, audited by Claude Core. 355 tests, 1 known env failure.
…eText retrieval_fun (fetch_pdf -> pdftotext, fail-closed, payload-free). Worker subagent, audit Claude Core. 378 tests, 1 known CUDA failure.
…n via DashScope, OpenAI-compatible). Injectable seams post_fun/api_key_fun, fail-closed on missing key/http/transport/bad output, errors carry only statuses+lengths, pattern-engine-free parsing. Worker subagent, audit Claude Core. 391 tests, 1 known CUDA failure.
…etch_pdf/pdftotext -> qwen3.7-max judge -> ClaimGate). ALL PASS: true claim SUPPORTED, false claim not_entailed, unregistered id rejected without LLM. Proven key/endpoint combo documented in-script.
…er (CiteClaw cite_fun + SourceText retrieval + EntailmentJudge judge), default model qwen3.7-max, DASHSCOPE key moved to runtime.exs boundary. 395 tests, 1 known CUDA failure.
…p 5, farthest-first kmeans, eigh orientation autodetect) on best-of-N, opt-in clustering/candidate_pool, injectable clustering_fun seam, fail-open fallback. 406 tests, 1 known CUDA failure. Worker subagent, audit + root-cause Claude Core.
… blind-Verifier termination to :recite; /ground cannot bypass ReferenceGate/ClaimGate. 3 new tests, 0 regressions, bypass closed E2E.
…terminal SELT stages skip sandbox execution (opt-in), inner run_loop cap injectable. /code E2E: 1439.18 in 85s, 1x run_loop (was 5-6x, 23min, AuditGate). Worker subagent, audit Claude Core. 415 tests, 1 known CUDA failure.
…er engine (Config/Coordinator/Runner/Formatter/History/CLI), seam contract mirrored 1:1 from assistant.exs (stage-aware /code, pass-through reference gate, empty-code guard, layer-0 depth classifier), contract-shape tests (opts kwlist, seam arities, slm_context). Qwen worker v2 after audit rejection v1, audit Claude Core. 424 tests, 1 known CUDA failure. Also versions scripts/assistant.exs (closes v18 debt).
…ract [[cite:ID]] in runner+mode :recite, injected entailment_fun (JUDGE_MODEL, max_tokens 1024), stage_aware_execution for /ground, StateManager.drop_empty (reasoning-burn poisoned history), worker max_tokens 512->2048, CiteClawClient: npx -y, non-sticky startup failure, timeout 15s->60s. Plus v20 web UI (web/, chat_ui, tests), README_ASSISTANT, judge 8->1024. NOT run through mix test (owner decision 08.07) - /ground E2E confirmed live 23:25-23:37.
…udge in ground contract, configurable ClaimGate :max_source_chars (default 4000 intact, /ground=60k: lexical window is cross-language blind, PL claim vs EN source picked random 2k slice), judge timeout 30s->120s for full-source verdicts, rejected-response snippet logging in citation gate (History only persists final result - 3 blind diagnoses in a row). claim_gate 14/14 green. Root cause found: worker refuses to cite arXiv 2506 as 'future paper' + thought-block truncation at 2048 - retrieval-first is next (HANDOFF v22).
… worker :recite prompt via base_opts seam, doc_id extraction arXiv/DOI from prompt (pattern-engine-free), SourceText fetch fail-open, shared 60k cap with L2 judge. 22 new tests green; remaining reds = pre-existing bc251eb policy debt + known CUDA. Worker subagent, audit + policy fixes Claude Core.
…ops labels (git=CMD, mix=WSL), retrieval-first E2E green
dfu925-ux added 30 commits July 17, 2026 17:24
…ement closes cross-sentence citation-span dilution; conservative abbrev/initial-safe boundary, fail-safe; default off (legacy); counters/judge untouched
…build path + one-shot multi_sentence_segment correction feedback; default off (legacy), counters untouched
…source path (Q4/D7d gap closed end-to-end); assertion added
…ec zdania odmowy; walidacja live sedzia (ABSENCE-true :ok, ABSENCE-false reject); projekt (c) w rezerwie
…t_fun

Default 8 is dead for reasoning judges; probe validated a different
instrument than the run (error class v54, extended lesson v58).
Bench runs on providers with concurrent-request caps (freeinference: 2)
need 1 without code edits. Default 4 = baseline behavior; unparsable or
<1 falls back to 4. Owner decision 21.07.
NoLibOsEnvTest guard: no direct System.get_env under lib/**. Read moved
to config/runtime.exs -> Config.l2_max_concurrency/0 (pattern: clustering).
Semantics unchanged: default 4, unparsable or <1 -> 4.
…z/L2 concurrency) + spec bramy asymetrycznej
… (b) korekta cytowana przed (c) [[absent]] — producent, testy 636/0, probe FP-EXPLICIT 3/3 grounded
…zystych [[absent]]; tor abstencji L2 osiagalny; testy 640/0
…]] tylko gdy seamy L2 wpiete (anty-fail-open), testy 647/0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant