feat: Finalizer structure_visibility_mode enforcement - #22
Open
shinushibu17 wants to merge 5 commits into
Open
Conversation
Closes the two-directional leak in the Finalizer's structure visibility rendering (Phase 2 Workstream C). ## What changed - Added FinalizerRenderGuard Pydantic schema in finalizer.py - Added _enforce_structure_visibility_mode deterministic gate - no_structure: strips trade_ideas and concrete structure text - illustrative_structure: preserves examples labeled 'not a live recommendation' - recommended_structure: passes through unchanged - Captures LLM trade ideas before _enforce_recommendation_mode clears them - Wired into primary, fallback, and degraded render paths ## Testing & docs - 24 drift-guard regression tests (no LLM required) — all pass - Eval cases expanded from 5 to 12 with structure visibility scenarios - SKILL.md for Finalizer Render Guard - ADLC worksheet (docs/adlc_worksheet.md) - README updated with Recent Improvements section Target metric: finalizer_illustrative_structure_rate (baseline 0.429)
There was a problem hiding this comment.
Pull request overview
Adds a deterministic “render guard” to the Finalizer so structure_visibility_mode is enforced post-LLM, preventing trade-structure leakage in no_structure and preserving properly-labeled examples in illustrative_structure.
Changes:
- Introduces
FinalizerRenderGuard,_strip_concrete_structure_text, and_enforce_structure_visibility_modeinScripts/agents/finalizer.py, wired into primary/fallback/degraded render paths. - Adds a new drift-guard regression test suite for structure visibility enforcement and expands the router E2E query catalogue with structure-visibility scenarios.
- Updates docs (README + ADLC worksheet) and adds a skill document describing the Workstream C guard.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| Scripts/agents/finalizer.py | Adds validated render-guard schema plus deterministic post-render enforcement and text stripping. |
| Scripts/tests/test_finalizer_structure_guard.py | Adds regression tests covering no_structure, illustrative_structure, and recommended_structure behaviors. |
| Scripts/tests/router_e2e_queries.json | Expands E2E query catalogue with structure-visibility scenarios. |
| README.md | Documents the Finalizer structure visibility enforcement improvement. |
| docs/adlc_worksheet.md | Adds an ADLC worksheet documenting baseline, fix, and re-eval plan. |
| .agents/skills/finalizer-render-guard/SKILL.md | Adds a skill doc describing the render guard’s purpose, gotchas, and verification. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1673
to
+1684
| def _build_render_guard(state: Dict[str, Any], recommendation_mode: str) -> FinalizerRenderGuard: | ||
| """Build a FinalizerRenderGuard from the current state.""" | ||
| constraints = _revision_constraints(state) | ||
| return FinalizerRenderGuard( | ||
| structure_visibility_mode=constraints.get("structure_visibility_mode", "no_structure"), | ||
| recommendation_mode=recommendation_mode, | ||
| allow_illustrative_structure=bool(constraints.get("allow_illustrative_structure")), | ||
| illustrative_structure_hint=str(constraints.get("illustrative_structure_hint") or "").strip(), | ||
| illustrative_structure_text=str(constraints.get("illustrative_structure_text") or "").strip(), | ||
| true_risk_text=str(constraints.get("true_risk_text") or "").strip(), | ||
| forbid_actionable_recommendation=bool(constraints.get("forbid_actionable_recommendation", True)), | ||
| ) |
|
|
||
| - **Problem**: The Finalizer dropped legally-allowed illustrative structure (rate: 0.429) and could promote structure when the mode forbade it. | ||
| - **Fix**: Added `FinalizerRenderGuard` — a deterministic, post-LLM enforcement gate in `Scripts/agents/finalizer.py` that enforces three modes: `no_structure` strips all trade ideas, `illustrative_structure` preserves examples labeled non-live, `recommended_structure` passes through. | ||
| - **Verification**: 15 drift-guard regression tests in `Scripts/tests/test_finalizer_structure_guard.py` (no LLM required). |
Comment on lines
+45
to
+48
| 1. **`Scripts/agents/finalizer.py`** — Added `FinalizerRenderGuard` typed schema and `_enforce_structure_visibility_mode` deterministic gate function. Wired into all three render paths (primary, fallback, degraded). | ||
|
|
||
| 2. **`Scripts/tests/test_finalizer_structure_guard.py`** — New: 15 drift-guard regression tests covering all three modes and edge cases. | ||
|
|
Comment on lines
+86
to
+92
| ### Drift-Guard Regression Tests | ||
| ```bash | ||
| python -m pytest Scripts/tests/test_finalizer_structure_guard.py -v | ||
| ``` | ||
|
|
||
| Expected: All 15 tests pass. | ||
|
|
| re.compile(r"\b(?:bull|bear)\s+(?:call|put)\s+spread\b", re.IGNORECASE), | ||
| re.compile(r"\b(?:iron\s+condor|straddle|strangle|credit\s+spread|debit\s+spread)\b", re.IGNORECASE), | ||
| re.compile(r"\bstrike\s+\d", re.IGNORECASE), | ||
| re.compile(r"\b\d+[CP]\b"), |
Comment on lines
+336
to
+343
| @pytest.mark.parametrize("text,expected_absent", [ | ||
| ("Consider a Long Put for hedging.", "long put"), | ||
| ("Short Call spread at strike 540.", "short call"), | ||
| ("Buy a Bull Call Spread on SPY.", "bull call spread"), | ||
| ("Iron Condor is suitable here.", "iron condor"), | ||
| ("Straddle the earnings event.", "straddle"), | ||
| ("Buy the 540P for protection.", "540P"), | ||
| ]) |
- Add docs/model_selection_rationale.md (Pillar 4 deliverable) citing llm_pool.py model assignments with cost/latency/quality justification per agent node - Add pyproject.toml + uv.lock for uv sync reproducibility (existing pip/requirements.txt preserved) - Restructure docs/adlc_worksheet.md to match ADLC 7-stage template (Scope/Design/Build/Evaluate/Deploy/Observe/Iterate) - Skill validation: skills-ref reports 'Valid skill' for finalizer-render-guard
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the two-directional leak in the Finalizer's structure visibility rendering (Phase 2 Workstream C).
What changed
Testing & docs
Target metric: finalizer_illustrative_structure_rate (baseline 0.429)