Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,14 @@ remain stopped historical experiments. Do not resume them.

| Chunk | Purpose | Risk | State |
|---|---|---:|---|
| `WS-QUAL-001-PLAN3` | Replace percentage-only closure with behavior/mutation assurance | L1 | Planning in progress |
| `WS-QUAL-001-04M` | Pilot pinned changed-scope mutation evidence without a score gate | L1 | Proposed after PLAN3 merge and explicit instruction |
| `WS-QUAL-001-PLAN3` | Replace percentage-only closure with behavior/mutation assurance | L1 | Merged PR #272; late review corrected by PLAN3R1 |
| `WS-QUAL-001-PLAN3R1` | Resolve five valid late CodeRabbit findings from PR #272 | L1 | Corrective planning chunk in progress |
| `WS-QUAL-001-04M` | Pilot pinned changed-scope mutation evidence without a score gate | L1 | Proposed after PLAN3R1 merge and explicit instruction |
| `WS-QUAL-001-05M` | Add calibrated blocking behavior-mutation policy | L1 | Proposed only after accepted 04M hosted evidence and explicit instruction |

## Dependency rule

`PLAN3 -> 04M -> human calibration checkpoint -> 05M`.
`PLAN3 -> PLAN3R1 -> 04M -> human calibration checkpoint -> 05M`.

Each chunk maps to one PR. `04M` may prove that the candidate engine or target
strategy is unsuitable and stop without `05M`. Planning does not pre-authorize
Expand Down
46 changes: 30 additions & 16 deletions .agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,10 @@ two independently reviewed implementation chunks.

Add one exactly pinned mutation engine and a Workstream-owned policy wrapper.
The wrapper derives a closed set of eligible production targets from the git
delta or from an explicit test-only behavior claim. It selects the smallest
owner test set, runs under a hard timeout, and emits machine-readable exact-head
evidence.
delta and always mutates those targets. It then adds any explicit test-only
behavior claims, each with owning test nodes; a claim cannot replace eligible
changed-production mutation. The wrapper selects the smallest owner test set,
runs under a hard timeout, and emits machine-readable exact-head evidence.

The pilot does not block on mutation score. It does block on infrastructure
failure, malformed evidence, target escape, missing claimed tests, ordinary
Expand All @@ -31,11 +32,12 @@ Only after pilot review, add a separate required check for eligible changed
production logic and test-only PRs that claim behavioral improvement. The gate
uses the pilot's deterministic target and evidence grammar.

There is no repository-wide mutation percentage. Every eligible survivor
blocks unless it has a narrow, typed classification accepted by policy (for
example, demonstrably equivalent or non-behavioral). Missing, stale, broad, or
free-form exclusions fail closed. Timeout and tool errors do not count as
killed mutants and cannot silently pass.
There is no repository-wide mutation percentage. The policy must enumerate
every engine status, including killed, survived, suspicious, timeout, error,
and excluded. Each status either blocks or maps to an independently
verified, typed classification accepted by policy (for example, demonstrably
equivalent or non-behavioral). Missing, stale, broad, free-form, or unrecognized
classifications fail closed. No status may pass implicitly.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

## Behavior ownership

Expand All @@ -57,9 +59,11 @@ observable outcomes. Test-only non-behavioral maintenance uses a narrow typed
classification defined by policy rather than free-form exemption text.

Test-only changes that claim coverage or stronger behavior must provide this
mapping. Documentation-only, fixture-only, generated-code, and non-behavioral
maintenance changes are outside mutation selection but remain subject to
ordinary tests and review.
mapping. Fixture-only changes are non-behavioral only when policy evidence
proves they cannot affect test selection, test inputs, or assertion behavior;
otherwise they require a bounded behavior claim. Documentation-only,
generated-code, and independently verified non-behavioral maintenance changes
remain outside mutation selection but subject to ordinary tests and review.

## Runtime and isolation strategy

Expand All @@ -77,9 +81,13 @@ ordinary tests and review.

- Pin the selected engine and its transitive dependency closure with hashes.
- Do not add the mutation engine to production dependencies.
- Install the engine only from `scripts/mutation-requirements.txt` with
`pip install --require-hashes`; `backend/pyproject.toml` may contain tool
configuration but cannot add the engine to ordinary dev extras.
- Obtain the approved mutation package, version, and hash authority from the
protected base revision, an equivalently protected allowlist, or a protected
prebuilt runtime—never from a manifest editable by the pull request. If
`scripts/mutation-requirements.txt` is used, install only from its trusted
base-revision copy with `pip install --require-hashes`;
`backend/pyproject.toml` may contain tool configuration but cannot add the
engine to ordinary dev extras.
- Never apply mutants to the contributor worktree in CI.
- Upload bounded result evidence without source secrets, environment values,
database contents, or artifact payloads.
Expand Down Expand Up @@ -112,11 +120,17 @@ errors, missing evidence, stale evidence, and target escape all stop the gate.

## Dependency order

`PLAN3 -> 04M pilot -> human calibration checkpoint -> 05M blocking gate`.
`PLAN3 -> PLAN3R1 -> 04M pilot -> human calibration checkpoint -> 05M blocking
gate`.
`05M` cannot begin from planning alone; it requires accepted exact hosted pilot
evidence and a new explicit human instruction.

## Stop

Planning does not install a mutation engine, change a workflow, or change a
coverage threshold. Stop after the PLAN3 PR and human checkpoint.
coverage threshold. Stop after the PLAN3R1 correction PR. Do not start `04M`
without separate explicit human instruction.

After `04M`, stop at the human calibration checkpoint. Do not start `05M`
without accepted exact hosted pilot evidence and a new explicit human
instruction.
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,11 @@ new or materially changed subsystem checks remain blocking at 90 percent.

## Current gate

`WS-QUAL-001-PLAN3` is planning only. It replaces the unstarted 04R global-floor
switch with a two-stage behavior/mutation assurance proposal:
`WS-QUAL-001-PLAN3` merged through PR #272. `WS-QUAL-001-PLAN3R1` is a
planning-only correction for five valid CodeRabbit findings that arrived before
the merge but were not addressed. It must merge before `04M` starts.

The corrected proposal remains two-stage:

1. `04M` — bounded, pinned, changed-scope mutation pilot with complete evidence
and no blocking score.
Expand All @@ -26,6 +29,6 @@ implemented.

## Stop condition

Stop after PLAN3 planning review and PR. Do not start 04M automatically. Do not
start 05M without accepted exact hosted pilot evidence and a new human
Stop after the PLAN3R1 correction review and PR. Do not start 04M automatically.
Do not start 05M without accepted exact hosted pilot evidence and a new human
instruction.
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Only these PLAN3 records describe possible current work:

- `WS-QUAL-001-PLAN3-behavior-mutation-assurance.md`
- `WS-QUAL-001-04M-changed-scope-mutation-pilot.md`
- `WS-QUAL-001-PLAN3R1-late-external-review-correction.md`
- `WS-QUAL-001-05M-blocking-behavior-mutation-gate.md`

Every other file in this directory is historical evidence from the original
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ scripts/git_delta.py
scripts/test_git_delta.py
scripts/workstream_agent_gate.py
scripts/behavior-claim.schema.json
scripts/mutation-requirements.txt
scripts/test_lightweight_agent_gates.py
.ci/behavior-claims/WS-QUAL-001-04M.json
.ci/behavior-claims/README.md
Expand All @@ -59,22 +58,28 @@ replacement, reduction, or bypass of Backend semantic lanes, API E2E, or fan-in
full-backend mutation on ordinary PRs
mutation pragmas or free-form exclusion lists
production dependency changes
creation or modification of the protected mutation-tool dependency authority
pull_request_target, privileged PR-code execution, writable workflow token,
checkout credentials, secrets in mutation execution, or unpinned Actions
```

## Acceptance criteria

- [ ] One engine and transitive closure are exactly pinned and hash locked as
development/CI-only dependencies, installed exclusively from
`scripts/mutation-requirements.txt` with `--require-hashes`.
development/CI-only dependencies. Pull-request checks obtain the
approved package, version, and hash authority from the protected base
revision (or an equivalently protected allowlist or prebuilt runtime),
never from a requirements manifest editable by the pull request.
- [ ] That protected dependency authority exists before the 04M branch starts;
04M cannot introduce or modify it. If no approved authority exists, 04M
stops for a separate bounded prerequisite rather than installing a tool.
- [ ] `backend/pyproject.toml` contains configuration only; the mutation engine
is absent from production dependencies and ordinary dev extras.
`scripts/mutation-requirements.txt` is the sole mutation-tool dependency
authority; `backend/uv.lock` remains unchanged and is not a second install
path.
- [ ] Deterministic policy selects eligible changed targets or validates a
bounded test-only behavior claim with explicit owning test nodes.
`backend/uv.lock` remains unchanged and is not a second install path.
- [ ] Deterministic policy always selects and mutates eligible changed
production targets, then adds any bounded test-only behavior claims with
explicit owning test nodes; a claim cannot replace eligible target
mutation.
- [ ] Git-delta discovery extracts one shared `scripts/git_delta.py` primitive
reused by `scripts/workstream_agent_gate.py` and mutation policy, and
mutation evidence mirrors the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,10 +56,12 @@ production behavior, migration, or dependency changes

- [ ] Eligibility and evidence grammar are unchanged from accepted pilot proof
unless a separately reviewed correction is explicit.
- [ ] Every eligible surviving mutant blocks by default.
- [ ] Policy enumerates every engine status, including killed, survived,
suspicious, timeout, error, and excluded. Each status blocks or maps
to an independently verified, typed policy classification.
- [ ] Any allowed classification is narrow, typed, evidence-bound, and tested;
missing, stale, broad, or free-form classifications fail closed.
- [ ] Timeout, suspicious, and error outcomes never count as killed.
missing, stale, broad, free-form, or unrecognized classifications fail
closed, and no status passes implicitly.
- [ ] Test-only behavior/coverage claims cannot bypass target mutation.
- [ ] `CONTRIBUTING.md` and the canonical claim README/schema/example explain
when a claim is required, the permitted typed non-behavioral cases, local
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Chunk Contract: WS-QUAL-001-PLAN3R1 — Late External Review Correction

## Parent initiative

`WS-QUAL-001` — Behavior And Mutation Assurance

## Goal

Correct the five valid CodeRabbit findings that arrived before PR #272 merged
but were not addressed before the branch was deleted.

## Why this chunk exists

PR #272 changed planning only, so no unsafe mutation runtime was deployed.
However, its merged plan leaves ambiguity in dependency custody, mutation
outcome handling, fixture-only classifications, target selection, and exact
Backend evidence binding. Those ambiguities must be removed before `04M` may
start.

## Risk class

L1 — CI/test policy planning correction.

## Machine scope

```chunk-scope-json
{
"schema_version": 1,
"chunk_id": "WS-QUAL-001-PLAN3R1",
"allowed_paths": [
".agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/CHUNK_MAP.md",
".agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/PLAN.md",
".agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/STATUS.md",
".agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/chunks/README.md",
".agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/chunks/WS-QUAL-001-04M-changed-scope-mutation-pilot.md",
".agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/chunks/WS-QUAL-001-05M-blocking-behavior-mutation-gate.md",
".agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/chunks/WS-QUAL-001-PLAN3R1-late-external-review-correction.md",
".agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/reviews/WS-QUAL-001-PLAN3-pr-trust-bundle.md",
".agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/reviews/WS-QUAL-001-PLAN3R1-external-review-response.md",
".agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/reviews/WS-QUAL-001-PLAN3R1-internal-review-evidence.md",
".agent-loop/initiatives/WS-QUAL-001-backend-coverage-floor/reviews/WS-QUAL-001-PLAN3R1-pr-trust-bundle.md",
".agent-loop/merge-intents/WS-QUAL-001-PLAN3R1.json"
],
"forbidden_paths": [
".github/**",
"backend/**",
"scripts/**"
],
"verification_commands": [
"markdown-links",
"stale-wording",
"lightweight-agent-gates",
"git-diff-check"
]
}
```

## Not allowed

```text
workflow, dependency, backend, test, or mutation implementation
coverage-threshold changes
automatic start of 04M or 05M
changes outside the WS-QUAL-001 correction evidence and merge intent
```

## Acceptance criteria

- [ ] PR code cannot choose or modify the mutation-tool dependency authority.
- [ ] Eligible changed production targets are always selected; bounded
test-only behavior claims are additive and name owning test nodes.
- [ ] Every engine outcome fails closed or has an independently verified typed
policy classification.
- [ ] Fixture-only changes require proof before receiving a non-behavioral
classification.
- [ ] PLAN3 Backend evidence names both exact run and commit.
- [ ] All five late comments are recorded and resolved before `04M` starts.

## Required reviewers

- senior engineering
- QA/test
- security/auth
- product/ops
- architecture
- CI integrity
- docs
- reuse/dedup
- test delta

## Human review focus

Confirm this correction closes the five late comments without implementing or
pre-authorizing mutation CI.

## Stop conditions

Stop if the correction requires executable workflow, dependency, backend, test,
or mutation changes.
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ quality before implementation.

## Design chosen

Mutation targets are eligible changed production logic or explicit test-only
behavior claims. Claims use schema-v1
Mutation selection always includes eligible changed production logic and then
adds any explicit test-only behavior claims with owning test nodes. Claims use schema-v1
`.ci/behavior-claims/<chunk-id>.json`; mutable PR prose cannot widen them.
`04M` runs independently under hard limits and records complete outcomes.
`05M` is not authorized until pilot evidence is accepted by a human.
Expand Down Expand Up @@ -57,9 +57,9 @@ mutation outcomes are engineering evidence and never product decisions.

## Acceptance criteria proof

- Current main is recorded from Backend run `30926337804`: 3,162 completed
tests, 21,620 / 23,938 coverage (90.316651 percent), 620.264 seconds wall,
and 464.471 seconds slowest lane.
- Current main is recorded from Backend run `30926337804` on commit
`5f2baf90`: 3,162 completed tests, 21,620 / 23,938 coverage (90.316651
percent), 620.264 seconds wall, and 464.471 seconds slowest lane.
- Global 78 and protected 90 floors are explicitly preserved.
- Changed-production and test-only behavior paths are both planned.
- Strong-vs-weak seeded mutant proof, complete outcome evidence, runtime bounds,
Expand All @@ -85,9 +85,9 @@ reviewer session remains open.

## External review

Agent Gates, CodeRabbit, and human review are pending publication. Backend is
not required by the planning diff unless GitHub policy schedules it; no Backend
file changes.
This section records the pre-publication state of PLAN3. PR #272 later received
five valid CodeRabbit findings before merge; PLAN3R1 records their disposition
in `WS-QUAL-001-PLAN3R1-external-review-response.md`. No Backend files changed.

## Remaining risks

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# WS-QUAL-001-PLAN3R1 External Review Response

## Comments addressed

- PR #272 discussion `3714978005`: PR-editable dependency manifests are no
longer trusted as mutation-tool authority; the contract requires authority
from a protected base revision or an equivalent protected runtime that
already exists before 04M and cannot be introduced or modified by 04M.
- PR #272 discussion `3714978015`: test-only behavior claims are additive and
cannot replace mutation of eligible changed production targets.
- PR #272 discussion `3714978020`: every engine status must block or map to an
independently verified typed classification; implicit passing is forbidden.
- PR #272 discussion `3714978023`: fixture-only changes are exempt only after
evidence proves they cannot influence selection, inputs, or assertions.
- PR #272 discussion `3714978026`: PLAN3 Backend evidence now binds run
`30926337804` to commit `5f2baf90`.

## PR #278 follow-up comments addressed

- Standardized the canonical mutation status label as `error` across PLAN,
04M evidence, INTENT, DISCOVERY, and the 05M gateway.
- Separated the PLAN3R1 stop, explicit 04M start, 04M calibration checkpoint,
and explicit evidence-bound 05M start.
- Clarified that dependency authority comes from a protected base revision.
- Corrected the trust-bundle scope evidence to twelve changed paths.

## Comments deferred

None.

## Human decisions needed

None. This chunk corrects merged planning and does not implement mutation CI.

## Commands rerun

- `python3 scripts/check_markdown_links.py`
- `python3 scripts/check_stale_workstream_wording.py`
- `python3 scripts/check_stale_authorization_docs.py`
- `python3 scripts/check_stale_artifact_contracts.py`
- `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -m pytest -q scripts/test_lightweight_agent_gates.py`
- `git diff --check`

## Remaining risks

The mutation engine and exact executable policy remain intentionally undecided
until separately authorized `WS-QUAL-001-04M` implementation and hosted pilot
evidence.
Loading
Loading