Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ ladder are superseded before implementation.

| Chunk | Purpose | Risk | State |
|---|---|---:|---|
| `WS-QUAL-001-PLAN2` | Reconcile current hosted baseline, retire obsolete machinery, and define the small closure sequence | L1 | Planning in progress |
| `WS-QUAL-001-02R` | Project/setup observable behavior coverage | L2 | Proposed after PLAN2 |
| `WS-QUAL-001-03R` | Checker observable behavior coverage | L2 | Proposed after 02R only if more headroom is needed |
| `WS-QUAL-001-PLAN2` | Reconcile current hosted baseline, retire obsolete machinery, and define the small closure sequence | L1 | Merged PR #260 |
| `WS-QUAL-001-02R` | Project/setup observable behavior coverage | L2 | Merged PR #265 |
| `WS-QUAL-001-03R` | Checker observable behavior coverage | L2 | Implementation in progress |
| `WS-QUAL-001-04R` | Change the exact global hosted CI floor from 78 to 90 after current-main proof | L1 | Proposed after measured >=90.25% proof |

One chunk maps to one PR. A test chunk may close early when its behavioral scope
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,27 @@

## Audited baseline

### Current-main refresh for 03R

Backend run `30921410531` on current-main merge commit `5b853d50` completed
3,068 tests, covered 21,453 of 23,938 statements (89.619016 percent), recorded
727.166 seconds total hosted wall time, and a 567.994-second slowest lane.
Reaching 90.25 percent on this denominator requires 21,605 covered statements,
a net gain of 152. The focused 03R test union covers 168 statements missing
from this hosted report: checker service 107, runner 45, and compiler 16. That
projects 21,621 / 23,938, or 90.320829 percent; hosted exact-head fan-in remains
authoritative.

Checker-owned gaps are sufficient and remain unchanged by ART: service 169,
runner 45, compiler 26, router 12, repository 11, gate queue 2, and pre-review
gate 1. Existing checker tests are integration-heavy. Direct fast coverage is
still missing for observable policy-shape rejection, registry ordering and
conflicts, routing priority, blocking-policy escalation, role-sensitive result
redaction, and bounded gate recovery outcomes. These are the preferred 03R
test seams; unrelated TASK or ART lifecycle paths remain out of scope.

### PLAN2 historical baseline

Hosted Backend run `30854931616` on the final PR #249 tested tree
`19d48f7ea4bf20cb29f03cbba54f98683ce52661` produced:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,37 @@

## Current state

`WS-QUAL-001-PLAN2` merged through PR #260. `WS-QUAL-001-02R` is now at its
external-review gate with a test-only implementation and passing required
internal reviews.

The latest complete current-main hosted baseline is Backend run `30891776021`
on `b47a7e64`: 88.603709 percent across 23,455 statements and 2,936 tests.
02R's first exact-head hosted run passed but measured 21,003 covered
statements, or 89.545939 percent, one statement below the chunk's 89.55-percent
target. A meaningful project-agent composition test now covers that remaining
factory statement and awaits exact-head hosted verification. The global CI
floor remains 78 percent; named protected subsystem checks remain blocking at
90 percent.
`WS-QUAL-001-PLAN2` merged through PR #260. `WS-QUAL-001-02R` merged through
PR #265 with all 2,996 exact-head tests passing and 21,004 / 23,455 statements
covered (89.550203 percent).

The current-main baseline after ART PR #268 is Backend run `30921410531` on
`5b853d50`: 3,068 tests completed, 21,453 / 23,938 statements covered
(89.619016 percent), 727.166 seconds hosted wall time, and a 567.994-second
slowest lane. The global CI floor remains 78 percent; named protected subsystem
checks remain blocking at 90 percent.

Historical QUAL work delivered the isolated database runner and test-integrity
guards through PRs #103, #105, and #108. The many stopped semantic-analysis
replacements remain historical evidence, not work to resume.

## Current gate

Require Agent Gates, CodeRabbit, all Backend semantic lanes, final coverage
fan-in, and human review for `WS-QUAL-001-02R`. Hosted coverage must be at least
89.55 percent, and an unexplained hosted runtime increase above 10 percent
stops merge readiness.
`WS-QUAL-001-03R` is the current implementation chunk. It must add meaningful
checker-owned behavior tests and gain at least 152 covered statements on the
current-main denominator to reach the 90.25-percent headroom target. The
focused test union measures
168 unique previously missing checker statements and projects 21,621 / 23,938,
or 90.320829 percent. Require Agent
Gates, CodeRabbit, all Backend semantic lanes, final coverage fan-in, six
internal reviewer tracks, and human review. An unexplained focused or hosted
runtime increase above 10 percent stops merge readiness.

The 94-case focused implementation selection passes in 37.76 seconds with
narrow checker-module coverage. The isolated full `test_checkers.py` run
reached the 1,200-second local ceiling after approximately 95 percent completion
with 163 passing tests and no failure output. It is not a complete pass; hosted
Backend remains mandatory.

## Stop condition

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ Parent initiative: `WS-QUAL-001`
Goal: if still necessary after 02R, add meaningful fast tests for current
checker service, repository, runner, compiler, and routing gaps.

Current-main basis: Backend run `30921410531` on merge commit `5b853d50`
completed 3,068 tests with 21,453 / 23,938 covered statements (89.619016
percent), 727.166 seconds hosted wall time, and a 567.994-second slowest lane.
The 90.25-percent target requires 21,605 covered statements, a net gain of 152
on this exact denominator.

Risk: L2 test-only; P2.

Allowed files:
Expand All @@ -16,17 +22,23 @@ Not allowed: production code, migrations, workflow/threshold changes, skips,
xfails, coverage exclusions, assertion deletion, or duplicate system flows.

Acceptance: observable outcomes, focused proof, recorded runtime impact,
complete hosted Backend, and exact global coverage of at least 90.25 percent.
complete hosted Backend, and exact global coverage of at least 90.25 percent
(21,605 / 23,938 on the starting denominator).
If meaningful checker gaps are exhausted first, stop and plan one explicit
owner-specific successor rather than add artificial tests.

Verification commands:

- `cd backend && ruff check tests/test_checkers.py`
- `cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres python scripts/run_isolated_tests.py --metadata-json .ci/qual-03r-database.json --timeout-seconds 1200 -- pytest -q tests/test_checkers.py`
- `cd backend && qual_tmp=$(mktemp -d /tmp/ws-qual-03r-db.XXXXXX) && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$qual_tmp/metadata.json" --timeout-seconds 1200 -- .venv/bin/pytest -q tests/test_checkers.py`
- Hosted Backend semantic lanes and final coverage fan-in.
- Test-delta scan for skips, xfails, deleted assertions, and coverage narrowing.

The 1,200-second local full-file timeout is a bounded diagnostic limit on the
constrained development machine and may be reached before collection completes.
Only the hosted Backend exact-head run is authoritative proof of the complete
suite pass.

Required reviewers: senior, QA, test-delta, CI integrity, product/ops, reuse.
Human focus: checker invariant ownership and fast-layer placement. This chunk
does not change or test TASK-owned lifecycle behavior merely for percentage.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# WS-QUAL-001-03R External Review Response

## Comments addressed

- CodeRabbit reported one trivial documentation ambiguity: the contract listed
the 1,200-second local isolated-run bound without explaining that this
constrained machine may reach the bound before the full file completes.
The contract now keeps the same command and explicitly names hosted Backend
on the exact PR head as the authoritative complete-pass proof.

## Comments deferred

None.

## Human decisions needed

None for the external finding. Human merge approval remains required after all
exact-head checks pass.

## Commands rerun

- `git diff --check`
- `python3 scripts/check_markdown_links.py`
- `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -m pytest -q scripts/test_lightweight_agent_gates.py`

## Remaining risks

- Backend semantic lanes and final fan-in must pass on the repaired exact head.
- Hosted coverage must reach at least 90.25 percent and runtime must be compared
with current-main run `30921410531`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# WS-QUAL-001-03R Internal Review Evidence

## Reviewed revision

- Code SHA: `23235d506d9f16cb000ba9e9219e4f3e28ddbb79`
- Reviewed base SHA: `cda59fc32e3a60e1e2c337cae7ebeaa94b95e12b`
- Reconciled current-main SHA: `5b853d505b55b6465a472d25aa33caec4be18073`
- Reviewed at: `2026-08-04`
- Open reviewer sessions: none
- Valid findings addressed: yes

## Circuit-breaker decision

PASS with a documented size exception. The diff is large because 94 focused
cases are added to one checker-owned test module, but it crosses one production
boundary, changes no production code, and adds no workflow, migration, public
contract, dependency, skip, xfail, or coverage exclusion. Splitting the same
checker behavior across dependent PRs would add coordination cost without
making the underlying boundary smaller.

## Reviewer results

| Reviewer | Result | Blocking findings | Notes |
|---|---:|---|---|
| senior engineering | PASS | None | Helpers and fakes remain reviewable and assert concrete service outcomes. |
| QA/test | PASS WITH LOW RISKS | None | Hosted exact-head Backend remains the authoritative acceptance proof. |
| test delta | PASS | None | Additive behavior tests; no weakening, skips, xfails, or exclusions. |
| CI integrity | PASS WITH LOW RISKS | None | No CI changes; coverage arithmetic is coherent and hosted proof remains required. |
| product/ops | PASS | None | Checker routing remains distinct from human product review decisions. |
| reuse/dedup | PASS | None | Added tests are fast boundary tests and do not duplicate the existing HTTP/PostgreSQL suffix. |

Security/auth, architecture, and docs review are not required for this
test-only chunk: it changes no production boundary, authorization behavior,
architecture, public contract, or contributor documentation.

## Findings reconciled

The first reuse pass attributed pre-existing broad API tests to this chunk by
using their post-insertion line numbers. Git history showed that
`test_chunk10_checker_trial_runs_sample_submissions_through_real_api` and the
other cited system tests already existed on `origin/main`; the 03R insertion
only shifted them downward. Re-review of the actual insertion hunk found no
duplicate system flow and withdrew the finding.

During review, ART PR #268 advanced `main`. It touched ART production, tests,
lane routing, and ART documentation but did not overlap the checker test or
QUAL initiative delta. The branch merged exact current main without conflict;
`git diff origin/main...HEAD` retains the reviewed 03R implementation. Main's
new exact hosted baseline is recorded below and the final reconciled head
receives focused reviewer confirmation before publication.

## Commands and evidence

- `cd backend && .venv/bin/ruff check tests/test_checkers.py` — pass.
- Focused new-test selection — 94 passed, 76 deselected in 37.76 seconds.
- `pytest --collect-only -q tests/test_checkers.py` — 170 tests collected.
- Narrow coverage comparison against current main — 168 uniquely covered
statements: checker service 107, runner 45, compiler 16.
- `git diff --check origin/main...HEAD` — pass.
- Lightweight Agent Gates — 10 passed.
- Allowed-path review — checker tests and the QUAL initiative tree only.
- Test-delta scan — no skip, xfail, deleted assertion, coverage exclusion, or
CI bypass.
- Full isolated `test_checkers.py` — the constrained local runner reached
approximately 95 percent (about 163 tests) without a failure before its
1,200-second bound. This is not recorded as a complete pass.

## Remaining risks

- Hosted Backend must pass every semantic lane and final fan-in on the exact PR
head.
- Hosted fan-in must prove at least 21,605 / 23,938 covered statements (90.25
percent). The local projection is 21,621 / 23,938 (90.320829 percent).
- Compare hosted wall and slowest-lane time with main run `30921410531`
(727.166 seconds wall; 567.994 seconds slowest lane). An unexplained increase
above 10 percent stops merge readiness.
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# WS-QUAL-001-03R PR Trust Bundle

## Chunk

`WS-QUAL-001-03R` — Checker Behavior Coverage.

## Goal

Raise global backend coverage from the exact current-main 89.619016-percent
baseline to at least 90.25 percent through meaningful checker-owned behavior
tests, without changing production behavior or CI policy.

## Human-approved intent

Continue the approved QUAL initiative one bounded chunk at a time, preserve
real PostgreSQL/system proof, and move fast business-rule coverage into the
smallest owning layer rather than adding arbitrary infrastructure or shards.

## What changed and why

- Added 94 focused cases in `backend/tests/test_checkers.py` for runner,
compiler, service, authorization, routing, redaction, policy-lock, automatic
gate, recovery, provenance, and fail-closed behavior.
- Updated only the QUAL initiative's discovery, contract, map, status, and
review evidence so durable memory matches the implementation.
- The tests cover 168 statements that current main does not cover: checker
service 107, runner 45, and compiler 16.

## Design chosen

Use direct runner/compiler/service tests and bounded repository/session fakes
for checker-owned decisions. Keep existing broad HTTP/PostgreSQL flows as the
small contract suite and leave production composition unchanged.

## Alternatives rejected

- More arbitrary CI shards: they move runtime rather than establish ownership.
- Duplicate end-to-end flows: slower and disallowed by the contract.
- Artificial execution or coverage exclusions: they would weaken trust.
- Production refactoring in this chunk: outside the reviewed test-only scope.

## Scope control

The implementation is limited to `backend/tests/test_checkers.py` and the
`WS-QUAL-001` initiative tree. It changes no production code, migration,
workflow, threshold, dependency, public API, skip, xfail, or coverage omit
rule. The circuit breaker passed with a size exception because this remains one
test file and one production boundary with no production behavior change.

## Product behavior

No product behavior changes. The tests preserve the distinction between
checker routing recommendations and the only stored human review decisions:
`accept`, `needs_revision`, and `reject`.

## Acceptance criteria proof

Current-main Backend run `30921410531` on `5b853d50` recorded 3,068 completed
tests, 21,453 / 23,938 covered statements (89.619016 percent), 727.166 seconds
hosted wall time, and a 567.994-second slowest lane. The target requires 21,605
covered statements. Local differential coverage projects 21,621 / 23,938
(90.320829 percent); hosted exact-head fan-in is authoritative.

## Tests and checks run

- Ruff: pass.
- Focused selection: 94 passed, 76 deselected in 37.76 seconds.
- Checker collection: 170 tests collected.
- Git whitespace and allowed-path checks: pass.
- Lightweight Agent Gates: 10 passed.
- Local full checker file: approximately 95 percent completed without failure
before the 1,200-second runner bound; not claimed as a complete pass.

## Test delta and CI integrity

Tests are additive. No assertion was deleted or weakened, and no test was
skipped, xfailed, deselected by CI, or excluded from coverage. No workflow,
runner, lane, service, coverage command, or threshold changed.

## Reviewer results

Senior engineering, QA/test, test delta, CI integrity, product/ops, and
reuse/dedup reviewed exact code SHA `23235d506d9f16cb000ba9e9219e4f3e28ddbb79`.
All pass. The first reuse finding was withdrawn after git history proved the
cited broad system tests were pre-existing main content shifted by the new
insertion.

## External review

Agent Gates passed on the published head. CodeRabbit's one trivial finding was
valid: the contract needed to distinguish the bounded local 1,200-second
diagnostic timeout from hosted complete-pass authority. The contract now says
that explicitly. Backend exact-head semantic lanes and final fan-in are still
running; human review remains pending.

## Remaining risks and follow-up work

- Hosted coverage may differ from the local differential projection.
- Hosted runtime must remain within 10 percent of the recorded baseline unless
the change is explained before merge.
- Raising the blocking global floor is a separate successor decision after
this exact-head coverage is proven; it is not part of 03R.

## Human review focus

- Confirm tests exercise observable checker invariants rather than percentage
padding.
- Inspect fake fidelity around policy provenance and gate state transitions.
- Confirm hosted coverage is at least 90.25 percent and runtime remains within
the contract bound.

## Human merge ownership

This chunk stops after exact-head external checks and human review. It will not
be merged without the user's explicit approval for this PR.
Loading
Loading