Enable ruff security, logging and debugging rules (S, T20, LOG, G, DTZ, ERA, TD, FIX, EXE)

.github/bump_version.py

@@ -3,14 +3,20 @@
 import re
 import sys
 from pathlib import Path
+from typing import TextIO
+
+
+def _report(message: str, *, file: TextIO | None = None) -> None:
+    """Print one line of this command's output, to stdout unless ``file`` is given."""
+    print(message, file=file)
 
 
 def get_current_version(pyproject_path: Path) -> str:
     """Return the version in pyproject.toml, exiting with status 1 if it is absent."""
     text = pyproject_path.read_text()
     match = re.search(r'^version\s*=\s*"(\d+\.\d+\.\d+)"', text, re.MULTILINE)
     if not match:
-        print(
+        _report(
             "Could not find version in pyproject.toml",
             file=sys.stderr,
         )
@@ -28,7 +34,7 @@ def infer_bump(changelog_dir: Path) -> str:
         f for f in changelog_dir.iterdir() if f.is_file() and f.name != ".gitkeep"
     ]
     if not fragments:
-        print("No changelog fragments found", file=sys.stderr)
+        _report("No changelog fragments found", file=sys.stderr)
         sys.exit(1)
 
     categories = {f.suffix.lstrip(".") for f in fragments}
@@ -63,7 +69,7 @@ def update_file(path: Path, old_version: str, new_version: str) -> None:
     )
     if updated != text:
         path.write_text(updated)
-        print(f"  Updated {path}")
+        _report(f"  Updated {path}")
 
 
 def main() -> None:
@@ -76,7 +82,7 @@ def main() -> None:
     bump = infer_bump(changelog_dir)
     new = bump_version(current, bump)
 
-    print(f"Version: {current} -> {new} ({bump})")
+    _report(f"Version: {current} -> {new} ({bump})")
 
     update_file(pyproject, current, new)
 

AGENTS.md

@@ -70,63 +70,33 @@ To add or change a rule, read `docs/engineering/skills/rules-registry.md`.
 
 ### API v2 layer boundaries
 
+- Checked: `api.v2.connectors-mutations-no-reads`, `api.v2.connectors-reads-no-mutations`, `api.v2.crud-checks-cover-every-resource`, `api.v2.session-modules-no-queries`, `api.v2.user-services-no-sql`, `api.v2.validators-transformations-no-db`
 - `api.routes.shared-query-parameters`: Routes that accept query parameters define them through the shared typed query models, never with a second manual parser. (doc-only, not checked: follow it yourself)
-- `api.v2.connectors-mutations-no-reads`: v2 mutation connectors (creates, updates, deletes) never issue select or get reads. (blocking: pytest `tests/unit/v2/test_data_crud_boundaries.py`)
 - `api.v2.connectors-never-commit`: v2 connectors receive a session from the service and never open, commit, roll back or close it. (doc-only, not checked: follow it yourself)
 - `api.v2.connectors-no-cross-calls`: v2 connector modules never call one another to sequence a workflow; the service does the sequencing. (doc-only, not checked: follow it yourself)
-- `api.v2.connectors-reads-no-mutations`: v2 read connectors never insert, add or delete rows. (blocking: pytest `tests/unit/v2/test_data_crud_boundaries.py`)
-- `api.v2.crud-checks-cover-every-resource`: The v2 CRUD boundary checks find modules by filename, so every resource package is covered, and every connector module is named as a create, read, update or delete module. (blocking: pytest `tests/unit/v2/test_data_crud_boundaries.py`)
 - `api.v2.routes-no-sessions-or-sql`: v2 route modules never open database sessions or construct SQL; they call one service method and map typed failures to HTTP. (doc-only, not checked: follow it yourself)
 - `api.v2.services-no-sql`: v2 services sequence work but never construct SQL expressions, call Session.exec, or define HTTP response models. (doc-only, not checked: follow it yourself)
-- `api.v2.session-modules-no-queries`: v2 database_session modules own session lifetime only and never build select, insert, update, delete or exec calls. (blocking: pytest `tests/unit/v2/test_data_crud_boundaries.py`)
-- `api.v2.user-services-no-sql`: v2 user services (identity, accounts, legacy references and user associations) never build or run SQL themselves or import SQLAlchemy; their connectors do. (blocking: pytest `tests/unit/v2/test_data_crud_boundaries.py`)
-- `api.v2.validators-transformations-no-db`: v2 validators and transformations never import SQLAlchemy or SQLModel query or session APIs. (blocking: pytest `tests/unit/v2/test_data_crud_boundaries.py`)
 
 ### Runtime persistence and route boundaries
 
-- `api.data.runtime-sql-confined`: Raw SQL is confined to the data-access layer; runtime modules and report orchestration use sessions and mapped models. (blocking: pytest `tests/unit/data/test_runtime_sql_boundaries.py`)
-- `api.legacy.removed-layers-stay-removed`: Removed legacy persistence layers, the temporary DAO module and the legacy endpoints package stay removed. (blocking: pytest `tests/unit/data/test_runtime_sql_boundaries.py`, `tests/unit/routes/test_flask_route_boundaries.py`, `tests/unit/services/test_orm_service_boundaries.py`)
-- `api.routes.flask-blueprints-own-urls`: Legacy Flask URLs are owned by blueprints, and the Flask app assembles blueprints without legacy endpoint wiring. (blocking: pytest `tests/unit/routes/test_flask_route_boundaries.py`)
-- `api.routes.no-domain-logic-in-http`: Flask routes delegate domain processing (economy comparison, calculation) to services instead of doing it in the HTTP layer. (blocking: pytest `tests/unit/routes/test_flask_route_boundaries.py`)
-- `api.routes.no-inline-json-errors`: Flask routes never build JSON error responses inline; they use the shared error handling. (blocking: pytest `tests/unit/routes/test_flask_route_boundaries.py`)
-- `api.services.no-direct-queries`: Services never issue database queries or open database connections directly; they delegate persistence to the data-access layer. (blocking: pytest `tests/unit/services/test_local_data_service_boundaries.py`, `tests/unit/services/test_orchestration_service_database_boundaries.py`, `tests/unit/services/test_orm_service_boundaries.py`, `tests/unit/services/test_run_and_spec_service_database_boundaries.py`)
-- `api.services.session-ownership`: Migrated services use sessions and mapped models; their public methods never accept persistence objects, and routes and the economy service never manage sessions. (blocking: pytest `tests/unit/services/test_local_data_service_boundaries.py`, `tests/unit/services/test_orm_service_boundaries.py`)
+- Checked: `api.data.runtime-sql-confined`, `api.legacy.removed-layers-stay-removed`, `api.routes.flask-blueprints-own-urls`, `api.routes.no-domain-logic-in-http`, `api.routes.no-inline-json-errors`, `api.services.no-direct-queries`, `api.services.session-ownership`
 
 ### Import and startup side effects
 
-- `api.catalog.initializer-not-at-startup`: Application startup modules never import the metadata catalog initializer. (blocking: pytest `tests/unit/v2/test_catalog_initialization.py`)
-- `api.startup.fail-closed-configuration`: Default Cloud SQL startup needs no Supabase configuration, and a selected v2 runtime without its database URL fails closed. (blocking: pytest `tests/unit/v2/test_import_side_effects.py`)
-- `api.startup.no-import-side-effects`: Importing v2 modules, starting up and serving a request open no network connections and create no files or runtime SQLite databases. (blocking: pytest `tests/unit/v2/test_import_side_effects.py`)
-- `api.startup.no-runtime-sqlite-modules`: Runtime SQLite modules are absent from the production package. (blocking: pytest `tests/unit/v2/test_import_side_effects.py`)
+- Checked: `api.catalog.initializer-not-at-startup`, `api.startup.fail-closed-configuration`, `api.startup.no-import-side-effects`, `api.startup.no-runtime-sqlite-modules`
 
 ### Data source isolation
 
-- `api.catalog.extractor-import-isolation`: The catalog extractor never imports PolicyEngine core, country packages or v1 metadata directly. (blocking: pytest `tests/unit/v2/test_catalog_extraction.py`)
-- `api.metadata.read-modules-isolated`: v2 metadata read modules never import PolicyEngine packages or v1 metadata sources. (blocking: pytest `tests/unit/v2/test_metadata_service.py`)
-- `api.reports.v1-persistence-only`: The existing report route and service import only v1 persistence. (blocking: pytest `tests/unit/v2/test_report_v1_runtime_selection.py`)
-- `api.stage12.comparison-tables-isolated`: Serving routes and services never read the Stage 12 comparison tables, and production tables never depend on them. (blocking: pytest `tests/unit/v2/test_stage12_comparison_isolation.py`)
-- `api.stage12.retirement-owned`: Retiring the Stage 12 comparison schema stays owned by a named follow-up change in the Stage 12 worker contract. (blocking: pytest `tests/unit/v2/test_stage12_comparison_isolation.py`)
-- `api.v2.single-orm-layer`: v2 models never create a parallel SQLAlchemy ORM layer alongside SQLModel. (blocking: pytest `tests/unit/v2/test_model_persistence.py`)
+- Checked: `api.catalog.extractor-import-isolation`, `api.metadata.read-modules-isolated`, `api.reports.v1-persistence-only`, `api.stage12.comparison-tables-isolated`, `api.stage12.retirement-owned`, `api.v2.single-orm-layer`
 
 ### Migration and schema
 
+- Checked: `api.migration.alembic-v1-qualified`, `api.migration.alembic-v2-qualified`, `api.migration.bundle-supported-by-simulation-api`, `api.migration.contracts-consistent`, `api.migration.cross-database-integration`
 - `api.migration.alembic-autogenerated-only`: Alembic revisions are autogenerated against an explicitly selected v1 or v2 configuration; never hand-written DDL or data operations. (doc-only, not checked: follow it yourself)
-- `api.migration.alembic-v1-qualified`: v1 Alembic revisions qualify against a disposable MySQL database. (blocking: CI job `alembic-v1-check`)
-- `api.migration.alembic-v2-qualified`: v2 Alembic revisions qualify against a disposable Postgres database, and alembic check finds no schema drift. (blocking: CI job `alembic-v2-check`)
-- `api.migration.bundle-supported-by-simulation-api`: A PolicyEngine bundle version change is supported by the simulation API before it merges. (blocking: CI job `ensure-policyengine-bundle-supported-by-simulation-api`)
-- `api.migration.contracts-consistent`: Migration route groups and workflow contracts are internally consistent, and their generated docs are current. (blocking: quality guard `migration-contracts`)
-- `api.migration.cross-database-integration`: Cross-database integration tests pass against real MySQL and Postgres. (blocking: CI job `v2-integration-check`)
 
 ### Code quality and types
 
-- `api.quality.alembic-revisions-unused-imports`: Alembic revision files are exempt from ruff lint rules, because they must stay exactly as Alembic generates them. (advisory: ruff per-file ignore of ALL for `migrations/*/versions/*.py`, not blocking)
-- `api.quality.environment-variables-load`: Required environment variables and credentials load in a deployed-like environment. (blocking: CI job `test_env_vars`)
-- `api.quality.mypy-v2`: Shared query models, FastAPI dependencies, v2 routes and services, and the user sign-in and account-deletion libraries pass mypy with untyped definitions disallowed. (blocking: mypy)
-- `api.quality.ruff-format`: All Python files are formatted with ruff format. (blocking: ruff format)
-- `api.quality.ruff-lint`: All Python code passes ruff check with every rule family selected in pyproject.toml, with no ratchet; families are added one themed group at a time until the selection is ALL. (blocking: ruff check (ANN, BLE, D, E4, E7, E9, EM, F, FLY, FURB, I, PIE, PYI, RET, RSE, TC, TRY, UP, W))
-- `api.quality.runtime-evaluated-models-resolve`: Every package model is one ruff treats as runtime-evaluated (a base resolves, through the import in its own module, to a name in runtime-evaluated-base-classes, or is a same-module class that does), and every Pydantic model and SQLAlchemy mapper resolves its annotations once the package is imported. (blocking: pytest `tests/unit/test_runtime_evaluated_annotations.py`)
-- `api.quality.tests-pass`: The unit, contract and to-refactor test suites pass. (blocking: CI job `test`)
-- `api.quality.tests-ruff-exemptions`: Test files may use pytest's bare assert and need no docstrings, annotations or named constants for magic numbers (S101, D1, ANN, PLR2004); every other ruff rule applies to tests. (blocking: ruff per-file ignore of ANN, D1, PLR2004, S101 for `**/tests/**`)
+- Checked: `api.quality.alembic-revisions-unused-imports`, `api.quality.ci-cd-scripts-ruff-exemptions`, `api.quality.command-line-modules-ruff-exemptions`, `api.quality.command-line-scripts-ruff-exemptions`, `api.quality.environment-variables-load`, `api.quality.mypy-v2`, `api.quality.ruff-format`, `api.quality.ruff-lint`, `api.quality.runtime-evaluated-models-resolve`, `api.quality.tests-pass`, `api.quality.tests-ruff-exemptions`
 
 ### Observability
 
@@ -135,21 +105,15 @@ To add or change a rule, read `docs/engineering/skills/rules-registry.md`.
 
 ### CI workflows and scripts
 
-- `api.ci.multiline-run-is-script`: A GitHub Actions run: block holds one line of code and no chained commands (&&, ||, ;); anything more lives in a tested script under .github/scripts/, called from a single-line step. (blocking: quality guard `ci-scripts`)
-- `api.ci.shell-scripts-pass-shellcheck`: Every shell script under .github/scripts/ passes shellcheck. (blocking: quality guard `ci-scripts`)
-- `api.ci.workflow-scripts-tested`: Every shell script a workflow calls is run by at least one test. (blocking: quality guard `ci-scripts`)
+- Checked: `api.ci.multiline-run-is-script`, `api.ci.shell-scripts-pass-shellcheck`, `api.ci.workflow-scripts-tested`
 
 ### Process and repository hygiene
 
-- `api.agents.adapters-point-to-agents-md`: CLAUDE.md and .github/copilot-instructions.md only point to AGENTS.md; skills and rules are listed in AGENTS.md alone. (blocking: quality guard `rules-registry`)
+- Checked: `api.agents.adapters-point-to-agents-md`, `api.process.changelog-fragment`, `api.process.cloud-run-image-builds`, `api.process.docker-image-builds`, `api.process.rules-registry`, `api.process.stacked-prs-full-ci-at-bottom`
 - `api.process.agents-never-bypass`: Agents never bypass branch rules, even with admin credentials: no gh pr merge --admin and no bypass option; a blocked merge is reported to a human. (doc-only, not checked: follow it yourself)
-- `api.process.changelog-fragment`: Every PR adds a changelog.d fragment. (blocking: CI job `check-changelog`)
-- `api.process.cloud-run-image-builds`: The Cloud Run container image builds. (blocking: CI job `test_cloud_run_container_builds`)
-- `api.process.docker-image-builds`: The API Docker image builds. (blocking: CI job `test_container_builds`)
+- `api.process.agents-suppress-as-last-resort`: Agents add an inline ruff suppression (noqa) only when neither a code change nor a ruff setting removes the finding, and a PR lists every inline suppression it adds, with the reason. (doc-only, not checked: follow it yourself)
 - `api.process.makefile-phony`: Command-only Makefile targets are declared in .PHONY, and .PHONY stays in sync when targets are renamed or removed. (doc-only, not checked: follow it yourself)
 - `api.process.pr-tracking-issue`: Every PR has a tracking issue in this repository, and the first line of its description is Fixes #ISSUE_NUMBER. (doc-only, not checked: follow it yourself)
 - `api.process.rule-machinery-code-owned`: Changes to the rule machinery (rules/, scripts/rules/, scripts/guards/, scripts/run_quality_guards.py and CODEOWNERS) need approval from a core developer other than the author. (doc-only, not checked: follow it yourself)
-- `api.process.rules-registry`: Every rule is registered in rules/registry.toml, is enforced where it claims to be, and its rendered views are current. (blocking: quality guard `rules-registry`)
-- `api.process.stacked-prs-full-ci-at-bottom`: In a stack of pull requests, only the lowest unmerged one runs the full PR workflow. Every pr.yml job is needed by the CI complete gate, which fails on the pull requests above it and otherwise passes only when every job passed, so a stack merges bottom-up and nothing merges untested. (blocking: pytest `tests/unit/test_ci_complete_gate.py`)
 
 <!-- END GENERATED: rules -->

changelog.d/ruff-security-logging.changed.md

@@ -0,0 +1 @@
+Enabled ruff's security, logging and debugging rules (S, T20, LOG, G, DTZ, ERA, TD, FIX, EXE): type-narrowing asserts became explicit checks, command-line scripts keep their printed output with reasons, commented-out code and TODO markers are gone, the remaining best-effort telemetry calls use a commented contextlib.suppress block, and every other deliberate exception carries a reasoned suppression; error responses and published schemas are unchanged. The unused GPT-4 dashboard experiment is removed, and a doc-only rule (api.process.agents-suppress-as-last-resort) has agents suppress a ruff finding only when no code change or ruff setting removes it, and list each new suppression in the PR description.

changelog.d/ruff-security-logging.fixed.md

@@ -0,0 +1 @@
+Removed the debug prints from the country validator and the report output and simulation routes, so requests no longer write context-free lines to stdout.