diff --git a/AGENTS.md b/AGENTS.md index c63a9d1..3079c7f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -135,7 +135,7 @@ Published on PyPI as `platecli`; the installed command is `plate`. | Gate | Command / note | |------|----------------| | Default tests | `uv run python -m pytest tests/ -q -m "not live"` — never contacts a printer | -| Coverage (CI) | `--cov-fail-under=86` (CI run `31044588411` on `5b08720`, 2026-08-05: Windows 88.8% / Linux 3.9 89.3%, 3.12 89.2%, 3.14 89.2% / macOS 89.1%; A+ target **92%** — see roadmap) | +| Coverage (CI) | `--cov-fail-under=86` (2026-08-13, PR #119: Linux 90.99% / Windows 90.68% / macOS passing; matrix 3.10/3.12/3.14; A+ target **92%** — see roadmap) | | Lint | `uvx ruff check bambu_cli` + `uvx ruff format --check bambu_cli` | | Types | `uvx mypy -p bambu_cli` | | Security lint | `uvx bandit -c pyproject.toml -r bambu_cli -ll` | diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c4e39d5..5042c37 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -120,7 +120,7 @@ architecture/contract gaps have since closed — the domain→`cli` helper extra (B.4), the single-sourced TLS pin verification (B.5), the remaining JSON schemas (now *generated* from `bambu_cli/contracts/`, one per `--json` subcommand), and the camera bind/pin-fallback hardenings. Main gaps to A+ / 1.0 are now coverage -(89.2% measured on CI's Linux legs, CI floor **86**, target 92) and the camera +(90.99% Linux / 90.68% Windows measured on CI, CI floor **86**, target 92) and the camera residuals still listed in SECURITY.md. Do not read "A−/A" as "A+" — see the scoreboard for what is actually ticked. diff --git a/docs/quality-roadmap.md b/docs/quality-roadmap.md index 7c6f09e..8178c05 100644 --- a/docs/quality-roadmap.md +++ b/docs/quality-roadmap.md @@ -15,13 +15,13 @@ Historical baseline (do not read as current), from the audit + full `pytest --cov=bambu_cli` on 2026-07-08: **368 tests**, **78%** line coverage (1105 / 4973 stmts missed), **130** `sys.exit` sites in `bambu_cli/`, **7** `@mockable` sites (def + 6 uses), **1** `BambuError` raise in production. -The "Baseline" column below is that snapshot. **Current measured (2026-08-05, the -released `0.5.0` commit `5b08720` on `main`): 1419 passed / 1 deselected, 89.1% -branch coverage over 8120 statements (local Linux, py3.12).** The full CI matrix -for that same commit (run `31044588411`; CI runs a clean checkout, so its numbers -differ slightly from a local run — do not reconcile one to the other): -Windows 3.14 **88.8%** (still the binding leg), macOS 3.14 89.1%, Linux 3.9 -89.3% / 3.12 89.2% / 3.14 89.2%. +The "Baseline" column below is that snapshot. **Current measured (2026-08-13, +PR #119): 1499 passed / 1 live deselected, 90.99% branch coverage over 8368 +statements (local Linux).** Full CI matrix: Linux 90.99% / Windows 90.68% (still +the binding leg) / macOS passing, on the 3.10 / 3.12 / 3.14 matrix (Python 3.9 +was dropped in #115). The prior `5b08720` matrix (run `31044588411`, 2026-08-05) +measured 88.8–89.3%; CI runs a clean checkout, so its numbers differ slightly +from a local run — do not reconcile one to the other. | Area | Baseline | Gate to A | Gate to A+ | Primary evidence | |------|----------|-----------|------------|------------------| @@ -67,16 +67,17 @@ items are **accepted 1.0 residuals** in [SECURITY.md](../SECURITY.md), not open **Overall:** **A** — none below A−. **Not A+ across the board.** Remaining: coverage 91.0→92, CI floor 92, Typing strict, Product `v1.0.0` tag. -**Coverage floor history:** 79 (honest post-Phase-1 gate) → **81** (2026-07-09) → **83** (2026-07-26) → **86** (2026-08-13; Windows 88.8% still the binding leg, ~2.8pt margin). -Measured branch total is **89.1%** on local Linux (2026-08-05, py3.12), 89.2% on CI's Linux legs; the floor is set +**Coverage floor history:** 79 (honest post-Phase-1 gate) → **81** (2026-07-09) → **83** (2026-07-26) → **86** (2026-08-13; Windows 90.68% the binding leg, ~4.7pt margin). +Measured branch total is **90.99%** on local Linux, 90.68% on CI's Windows leg; the floor is set at the multi-OS minimum so the matrix does not flake while still denying points of silent rot vs the old 79 gate. -**Ratchet headroom (measured 2026-08-05, run `31044588411` on `5b08720`):** every -leg now sits above 88 — Windows 88.8%, macOS 89.1%, Linux 3.9/3.12/3.14 -89.3/89.2/89.2% — against a gate of 86, so roughly three points of drift can pass +**Ratchet headroom (measured 2026-08-13, PR #119):** every +leg now sits above 90 — Windows 90.68%, Linux 90.99%, macOS passing — against a +gate of 86, so roughly four points of drift can pass unnoticed. Windows remains the binding leg, as it has at every ratchet. Raising -the gate to **88** now clears Windows by only 0.8pt, which is a thin margin for a +the gate to **88** now clears Windows by ~2.7pt; a **90** gate clears it by only +~0.7pt, which is a thin margin for a matrix that has to stay green on every PR. Ratcheting means moving `ci.yml`, the citations in this file, and `docs/test-backlog.md` together — `tests/test_docs_consistency.py` and `tests/ci_workflow_smoke.py` both enforce that. @@ -571,7 +572,7 @@ bandit + pip-audit blocking # Conceptual — implement incrementally across phases jobs: test: - matrix: [py39, py312, py314] x [ubuntu] + [macos/windows @ newest] + matrix: [py310, py312, py314] x [ubuntu] + [macos/windows @ newest] steps: - pytest -W error::ResourceWarning --cov=bambu_cli --cov-fail-under=92 @@ -633,7 +634,7 @@ If **full A+** is the goal, follow phases 0→A→B→C→D in order; skip ahead | 0 Trust & truth | **done** | local | 2026-07-08 | allow-private-ips, bare except, version single-source | | A Testing foundation | **done** | local | 2026-07-08 | TLS suite, markers, transport tests, cov~80% | | B Error model & seams | **done** | #11 | 2026-07-08 | abort/BambuError; sys.exit entry-only; mockable removed. **B.4** paths/jsonio/argutils extract done (domain no longer imports private cli helpers); **B.5** single pin helper done (PR #89) | -| C Coverage & typing | **in progress** | #18 | — | full-package mypy + `check_untyped_defs` done (#18, 2026-07-09); **C.4** hermetic fake OrcaSlicer done; cov 89.2% on CI's Linux legs with CI floor **86** (target 92); per-module floors not enforced, so C.5 is the open item | +| C Coverage & typing | **in progress** | #18 | — | full-package mypy + `check_untyped_defs` done (#18, 2026-07-09); **C.4** hermetic fake OrcaSlicer done; cov 90.99% Linux / 90.68% Windows with CI floor **86** (target 92); per-module floors not enforced, so C.5 is the open item | | D Contracts & 1.0 | **in progress** | #101 | — | schemas + contract harness + stability policy done; **schemas are now generated** from `bambu_cli/contracts/` and every `--json` subcommand has one (#101). **D.3 support matrix** published in `docs/api.md`. Open: optional structured logging (D.5) and the 1.0 prep itself (D.6) | | E Stretch | not started | | | fuzz job, SBOM, dependabot, scheduled live-printer | | Doc truth pass | **done** | local | 2026-07-24 | versions de-literalized, prerequisites stated, camera guidance corrected, test/coverage numbers re-measured | @@ -670,6 +671,14 @@ If **full A+** is the goal, follow phases 0→A→B→C→D in order; skip ahead > and CONTRIBUTING's "coverage ~82% / helper extraction / TLS pin / remaining > schemas" gap list, all four of which had closed. Still **not** A+: coverage is > 89, not 92; no per-module floors; camera residuals stand. +> +> **Re-verified 2026-08-13** (PR #119): 1499 non-live tests passing (1 live +> deselected); 90.99% branch coverage over 8368 statements on local Linux, +> 90.68% on Windows (the binding CI leg); floor raised to **86**. Closed in this +> pass: the last `check_layers` `ALLOWED` edge (`context → printer`, replaced by +> an injectable printer factory) and the `protocols/mqtt.py` monolith (split into +> `mqtt_tls` / `mqtt_cmd` / `mqtt_print` / `mqtt_monitor` / `mqtt_session`). Still +> **not** A+: coverage is 91, not 92; no per-module floors; camera residuals stand. ### mockable count (burn-down) diff --git a/docs/test-backlog.md b/docs/test-backlog.md index dca617a..c98d6ea 100644 --- a/docs/test-backlog.md +++ b/docs/test-backlog.md @@ -6,12 +6,12 @@ This file is a short **remaining-gaps** list only. Refresh after each phase or audit. Do not treat historical “≥98% coverage” claims as current — see the snapshot below. -## Snapshot (2026-08-05, release commit `5b08720`) +## Snapshot (2026-08-13, PR #119; measured on Linux) | Metric | Current (honest) | A+ / 1.0 target | |--------|------------------|-----------------| | Non-live tests collected | **1499** passing (measured 2026-08-13 on Linux; 1 live test deselected) | ≥550 with zero known flakes ✅ size | -| Line/branch coverage (CI) | **91.0%** branch coverage over 8368 statements (local Linux, 2026-08-13); prior CI matrix on `5b08720` was 88.8–89.3%; **floor 86** | **≥92%** total; optional module floors | +| Line/branch coverage (CI) | **91.0%** branch coverage over 8368 statements (local Linux, 2026-08-13; Windows 90.68% / macOS passing on CI); prior CI matrix on `5b08720` was 88.8–89.3%; **floor 86** | **≥92%** total; optional module floors | | Typing | Full package mypy + `check_untyped_defs` | keep; optional full `strict` later | | Error model | `sys.exit` only in `cli.py` | keep | | `@mockable` / test-awareness | **0** (CI greps) | keep |