LLM Control Plane: Phase 0 (remediation merge) + Phase 1 (kernel)#16
Open
cryptoxdog wants to merge 10 commits into
Open
LLM Control Plane: Phase 0 (remediation merge) + Phase 1 (kernel)#16cryptoxdog wants to merge 10 commits into
cryptoxdog wants to merge 10 commits into
Conversation
Merges and remediates the overlapping findings from consolidated-findings-plan.md (PRD-*/ICA-*) and consolidated-findings-remediation-plan.md (AUD-*/SEC-*/RAA-*/DWA-*), both audited against base_ref 87075d8. 18 distinct defects closed; see src/docs/WIP/LLM-Router audits/unified-remediation-ledger.md for full deduplication rationale and per-ID disposition (not tracked in git — WIP planning doc). Phase 1 — CI/test safety net (TEST-CI-GAP: PRD-002/AUD-001/RAA-001) - Add eslint.config.js (flat config, ESLint v9) and a real Vitest suite - Wire npm test + npm run lint into ci.yml and publish.yml Phase 2 — Critical live-traffic bug (ICA-002/ICA-003) - getDowngradedModel() now branches on provider/model-family instead of always returning a GeneralModel - Replace the unchecked `as any` cast in execute()'s Perplexity branch with a fail-closed isSonarModel() guard Phase 3 — Pricing authority consolidation (PRICING-DRIFT: PRD-003/AUD-002/RAA-002) - Extract canonical src/pricing.ts; openrouter.ts and general-matrix.ts both consume it instead of maintaining independent rate tables Phase 4 — Dead-wiring activation & cleanup - Implement the disableSearch branch (ICA-001/DWA-001): web_search_options is now omitted when search is disabled - Add checkSurge()/resetGlobalMonthly() router wrappers (DWA-002/DWA-003) - Implement a real CircuitBreaker, wired into execute() (DWA-005) - Remove unused RoutingResult/ResolvedConfig types (DWA-004) - Accumulate per-attempt fallback errors instead of discarding them (AUD-005) Phase 5 — Security hardening - validateImageUrl(): https-only + private/loopback/link-local hostname allowlist for vision image URLs (SEC-001, candidate CWE-918) - src/schemas.ts: Zod validation of TaskDescriptor/RouterConfig at execute()/constructor entry (SEC-002) - toJSON() redaction allowlists on PerplexityError/OpenRouterError/ BudgetExhaustedError (SEC-003) - npm audit --audit-level=high --omit=dev gate in ci.yml/publish.yml (SEC-004) Phase 6 — Contract documentation (RAA-003) - ARCHITECTURE.md documents module boundaries and router-only-egress - no-restricted-imports ESLint rule enforces that only src/index.ts may import providers/* directly Phase 7 — Re-baseline - Ledger and prior-baseline handoff doc updated to close out all findings (WIP docs, not tracked in git) 82 tests across 10 files, 0 lint errors, 0 npm audit vulnerabilities (prod deps), clean tsc build/typecheck. Co-authored-by: Cursor <cursoragent@cursor.com>
Merges the 18-finding remediation branch as the baseline for the LLM Control Plane 7-phase build. Resolves 6 conflicts: - eslint.config.js: union of both configs (main's src/docs/** ignore + branch's @eslint/js base config, no-restricted-imports router-only-egress rule, and coverage/** ignore) - package.json: union of devDependencies (typescript, vitest, typescript-eslint) - src/index.ts: take branch's CircuitBreaker import (rest was already a superset on this side) - src/types.ts: take branch's per-member eslint-disable comments (cosmetic) - tests/budget.test.ts, tests/router.test.ts: take branch's versions in full (rewritten supersets covering family-safe downgrade, circuit breaker, and Zod validation paths) Validated post-merge: build, verify:types, lint (0 errors, 2 pre-existing any warnings), test (120/120 passing across 13 files), npm audit --audit-level=high --omit=dev (0 vulnerabilities). Co-authored-by: Cursor <cursoragent@cursor.com>
… route-plan, execution-record, feedback-signal) Implements the Phase 1 nuclear contract (v5.0.0) additively under src/control-plane/, deliberately NOT wired into the legacy L9LLMRouter public surface (src/index.ts). The legacy router remains the package's default API; cutover is explicit Phase 8 scope, not performed here. Adds: - src/control-plane/contracts.ts — TaskProfile, LLMRoutePlan, LLMExecutionRecord, LLMFeedbackSignal schemas + all referenced unions - src/control-plane/hashing.ts — contentHashOf(): canonical-JSON SHA-256 - src/control-plane/task-profiler.ts — buildTaskProfile() - src/control-plane/policy-engine.ts — PolicyEngine interface (no impl) - src/control-plane/provider-adapter.ts — LLMProviderAdapter interface (no impl) - src/control-plane/route-plan.ts — buildRoutePlan(); route_id derived from routing-relevant fields only (excludes request_id) per the Routing Determinism Law - src/control-plane/execution-record.ts — buildExecutionRecord() - src/control-plane/feedback-signal.ts — buildFeedbackSignal() - src/control-plane/index.ts — barrel export, reachable via the new "./control-plane" package.json subpath export - config/llm-route-matrix.v2.yaml — static defaults skeleton (not loaded by any Phase 1 code path yet) - docs/control-plane-architecture.md — Phase 1 boundary doc, explains the legacy/control-plane coexistence decision - tests/control-plane/*.test.ts — 17 tests covering all 10 contract-required assertions (determinism, content_hash presence, interface callability, Graphiti/Neo4j import boundary, no hardcoded node URLs) Validated: build, verify:types, lint (0 errors, 2 pre-existing warnings unrelated to this change), test (137/137 across 21 files), npm audit --audit-level=high --omit=dev (0 vulnerabilities). Full captured output in src/control-plane/validation-results.md. Not built in this pass: Phases 2-7 (Provider Adapter Expansion, Route Matrix v2.1, Evidence and Signals, Fitness Engine, Promotion Workflow, Frontier Lab) and Phase 8 (legacy cutover). See machine-summary.json. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
…le 1) PR #16's CI run surfaced 5-6 pre-existing failing governance workflows, confirmed (via `gh run list --branch main`) to already be failing on main since they were imported in PR #3 — unrelated to this PR's own diff, but blocking this PR's checks, so fixed here as part of the remediation loop. Root cause 1 (9 files): every `uses: Quantum-L9/l9-ci-core/.github/ workflows/*.yml@v1` reference targets a tag that does not exist in that repo — `gh api repos/Quantum-L9/l9-ci-core/tags` shows only `v0.1.0` is published. Re-pinned all 9 refs to `@v0.1.0`; verified each target workflow file exists at that tag via the GitHub API before committing. Root cause 2 (l9-node-ts-monorepo.yml, self-contained, no external repo involved): `install-and-typecheck` ran `npm run typecheck`, but package.json only defines `verify:types` — fixed to call the real script name. `test-by-segment` ran `npx jest --testPathPattern=...` across a non-existent unit/integration split, but this package has no Jest dependency and uses Vitest exclusively with a flat tests/ directory — replaced with a single `test` job running `npm test`. Known deferred item (not fixed, documented instead): `L9 Analysis` (.github/workflows/l9-analysis.yml) pins l9-ci-core to a direct commit SHA (not a tag) and still fails — captured logs show the failure is `ModuleNotFoundError: No module named 'yaml'` inside l9-ci-core's own `provision-sdk` action (missing `pyyaml` in that action's pip install step). This is a defect in the external Quantum-L9/l9-ci-core / l9-ci-sdk repositories, not in any file this repo owns — no local fix is possible; recommend filing against Quantum-L9/l9-ci-core. Local verify (blocking gate, re-run after these changes): build, verify:types, lint (0 errors), test (137/137), npm audit --audit-level=high --omit=dev (0 vulnerabilities) — all green. YAML syntax of every edited workflow validated with python3 -c "yaml.safe_load(...)". Co-authored-by: Cursor <cursoragent@cursor.com>
l9-ci-core's security.yml@v0.1.0 reusable workflow does not declare a run-npm-audit input (only python-version, source-dir, l9-ci-install-command). Passing it caused a workflow-call input validation startup_failure in both l9-security.yml and the embedded security job in l9-node-ts-monorepo.yml. npm audit already runs separately in ci.yml. Co-authored-by: Cursor <cursoragent@cursor.com>
#16 cycle 3) GitHub Actions rejects a reusable-workflow call at startup (0 jobs, opaque "workflow file issue" / startup_failure) when a called job's declared `permissions:` exceed what the caller grants. l9-ci-core's security.yml, scorecard.yml, and pr-pipeline.yml's internal security/sbom/scorecard jobs request security-events:write, id-token:write, and/or pull-requests:read, but none of our wrapper workflows declared elevated permissions (repo default is read-only), so every call to them failed at resolution time before any job ran. Also drop `src-directory` from l9-governance.yml: trio-governance.yml@ v0.1.0 does not declare that input (only python-version, changed-files, policy-path, baseline-path, classifier-path, the three trio token secret-name inputs, pr-labels, labels-known), so passing it caused the same class of startup_failure independent of the permissions issue. Fixed: - l9-security.yml: add job-level permissions (contents:read, security-events:write, pull-requests:read) - l9-scorecard.yml: add job-level permissions (security-events:write, id-token:write, contents:read, actions:read) - l9-node-ts-monorepo.yml: same permissions on its embedded security/scorecard jobs - l9-pr-pipeline.yml: add job-level permissions (contents:read, security-events:write, pull-requests:read, id-token:write) and remove undeclared run-security/working-directory inputs - l9-governance.yml: remove undeclared src-directory input Not fixed (documented as out of scope for this PR, tracked separately): l9-pr-pipeline.yml, l9-governance.yml, and l9-pre-commit.yml still require Python-only governance infrastructure that does not exist in this Node/TypeScript repo (.github/governance/ quality-thresholds.yaml, rule-modes.yaml, audit-policy.yml, audit-baseline.json, a Python classifier script, a .pre-commit-config.yaml, and the closed-source `l9-ci` pip package). Creating that infra is a repo-governance decision, not a Control Plane PR concern. Co-authored-by: Cursor <cursoragent@cursor.com>
…cycle 4) pr-pipeline.yml@v0.1.0's embedded scorecard job requires security-events:write, id-token:write, contents:read, AND actions:read. Cycle 3 granted the first three but missed actions:read, so the call still failed at startup (0 jobs) even after the other permission fixes. Co-authored-by: Cursor <cursoragent@cursor.com>
…le 5)
SonarCloud's PR analysis failed the Security Rating and Reliability
Rating gates (both required >= A) on two real findings introduced by
this branch's CI-workflow edits and Phase 1 code:
- .github/workflows/{l9-security,l9-scorecard,l9-node-ts-monorepo,
l9-pr-pipeline}.yml: pin the Quantum-L9/l9-ci-core reusable-workflow
reference to its resolved commit SHA instead of the floating v0.1.0
tag (supply-chain integrity — a moved/deleted tag can silently swap
in different code). Applied the same pin to the remaining l9-ci-core
callers (l9-governance, l9-nightly, l9-pre-commit, l9-release,
l9-sbom) for consistency, even though Sonar did not flag those files
as changed in this diff.
- src/control-plane/hashing.ts: canonicalize()'s Object.keys(...).sort()
relied on the default sort comparator. Sonar's suggested fix
(String.localeCompare) would make key ordering — and therefore the
content hash — depend on runtime locale/ICU data, which breaks the
cross-machine determinism this module exists to guarantee. Used an
explicit ordinal comparator instead and documented why
localeCompare is deliberately not used here.
Verified: npm run verify:types, npm run lint (0 errors), npm test
(137/137 passing) all still green after this change.
Co-authored-by: Cursor <cursoragent@cursor.com>
Collaborator
Author
CI Remediation Summary (l9-PR-remediation)Ran 5 fix/verify cycles against this PR's CI. Result: every Fixed (in scope, verified green)
Verified locally after every change:
Not fixable from this repo (confirmed external root causes — no code change here can resolve these)
All five clusters trace back to the same family of issue: Fixing these for real requires either (a) publishing the |
l9-pre-commit.yml calls l9-ci-core's pre-commit-ci.yml, which fails
validate-hook-config with no .pre-commit-config.yaml present, and would
fail pre-commit-run/hook-version-drift on this repo's default
install-command (pip install -e .[dev]) since there is no pyproject.toml.
- Add .pre-commit-config.yaml with pinned pre-commit-hooks v6.0.0
file-hygiene hooks only (trailing-whitespace, end-of-file-fixer,
check-yaml, check-json, check-merge-conflict, check-added-large-files,
check-case-conflict) — no fabricated ruff/mypy hooks for a repo with no
Python source.
- Override install-command to a no-op ("true") in l9-pre-commit.yml so
pre-commit-run/hook-version-drift don't try to pip install a
nonexistent Python package.
- Strip trailing whitespace from the 6 files the new trailing-whitespace
hook flagged (CODE_OF_CONDUCT.md, src/index.ts, src/budget/index.ts,
src/matrices/general-matrix.ts, src/matrices/perplexity-matrix.ts,
src/vision/index.ts) so the first CI run passes clean.
Verified locally: pre-commit validate-config, pre-commit run --all-files
(clean second pass), npm run verify:types, npm run lint, npm test
(137/137 passing).
Co-authored-by: Cursor <cursoragent@cursor.com>
…d-fail l9-ci-core's pre-commit-ci.yml hardcodes actions/setup-python@v6 with cache: pip in every job (not exposed as a configurable input from this repo's wrapper). With no requirements.txt or pyproject.toml anywhere in the tree, setup-python errors before pre-commit even runs: "No file ... matched to [**/requirements.txt or **/pyproject.toml]" This repo has no Python source (see ARCHITECTURE.md); the file is intentionally empty and exists solely to give the cache step a lockfile to hash. Co-authored-by: Cursor <cursoragent@cursor.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Executes the first two steps of the 7-phase LLM Control Plane build plan:
fix/unified-remediation-phases-1-7(18 closed audit findings: family-safe model downgrade, circuit breaker, Zod validation, pricing table, error-redaction, router-only-egress lint rule) into this branch as the baseline for the Control Plane build. Resolved 6 conflicts (eslint.config.js,package.json,src/index.ts,src/types.ts,tests/budget.test.ts,tests/router.test.ts) — all mechanical/cosmetic, no semantic disputes.src/control-plane/—TaskProfile,LLMRoutePlan,LLMExecutionRecord,LLMFeedbackSignalschemas (all SHA-256 content-hashed),PolicyEngine/LLMProviderAdapterinterfaces (no implementation — later phases), and deterministic builder functions for each schema.Architectural decision (documented in
docs/control-plane-architecture.md): the new kernel lives undersrc/control-plane/and is reachable only via a new./control-planepackage.json subpath export. It is not wired intosrc/index.ts— the legacyL9LLMRouterremains the package's default public API. The plan's own Phase 8 is the explicit cutover step; wiring the two together now would pre-empt that decision mid-build.What is NOT included in this PR
Phases 2-8 of the build plan are not built:
L9LLMRoutercutoverEach remaining phase is its own ~10-file nuclear contract (schemas + interfaces + implementation + 10 tests + config + docs) — out of scope for a single autonomous pass. This PR intentionally stops at a clean, fully-validated checkpoint (Phase 0 + Phase 1) rather than partially building later phases.
Test plan
All commands run locally against this branch, output captured in
src/control-plane/validation-results.md:npm cinpm run build— exit 0npm run verify:types— exit 0npm run lint— 0 errors, 2 pre-existing warnings (unrelatedas anycasts insrc/index.ts, predate this PR)npm test— 137/137 passing across 21 files (120 pre-existing + 17 newtests/control-plane/*, covering all 10 Phase 1 contract-required assertions)npm audit --audit-level=high --omit=dev— 0 vulnerabilitiesMade with Cursor