Enable ruff structure and import rules (INP, PLC, SLF, N, A, FAST, E)

[anth-volk]

Fixes #3919

Summary

Phase 1 step 1.8, group 7: structure and imports. The lowest PR in its stack; #3924 builds on it.

• Test exemption (decided for this group): test files now also ignore SLF001, INP001 and PLC0415, alongside S101, D1, ANN and PLR2004. Tests deliberately reach into private members, pytest test directories aren't packages, and tests often import inside a test after patching the environment. They also ignore N802, because test fakes name their methods after the interface they replace (a gRPC stub's Export). Application code still gets every rule. api.quality.tests-ruff-exemptions is updated to match.
• Rules: select gains A, E, FAST, INP, N, PLC and SLF. All 289 findings left after the exemption are fixed, or kept with an inline reason. The kept ones are listed below.
• FastAPI dependencies (FAST002, 21): ruff's own fix would break the v2 routes, which use from __future__ import annotations: FastAPI resolves string annotations against module globals, and the query dependencies were local closures. Each dependency is now a module-level alias instead, such as HouseholdCreateQueryDependency = Annotated[HouseholdCreateQuery, Depends(query_dependency(HouseholdCreateQuery))]. The OpenAPI is byte-identical.
• Imports inside functions (PLC0415):

  • Moved (8, plus 4 in the code Stage 12A: Sign-in identity and v1-anchored user associations #3891 added): imports that add no cycle, side effect or third-party load. A fresh-interpreter check of all 220 policyengine_api modules shows that policyengine_api.asgi, policyengine_api.api and 211 others load exactly the same modules. Seven route and dependency modules now also load the small, dependency-free readiness and fastapi_routes.dependencies modules.

  • Listed as lazily loaded (44 suppressions removed): pyproject.toml lists 23 packages and modules in ruff's banned-module-level-imports, with TID253 selected:

    • policyengine and spm_calculator, which load country models;
    • data.v2.database, which registers an exit hook at import;
    • the nine v2 database_session modules and the nine v2 services modules, which load persistence only when a default factory runs;
    • libs.auth0_management and google.cloud.secretmanager, which load only when a secret is read. data/orm.py imports a different submodule, google.cloud.sql.connector, at module level, so only the Secret Manager client is listed.

    No application module imports any of them at module level. Importing one there now fails, though TYPE_CHECKING imports are fine, and PLC0415 accepts their in-function imports without a suppression. Tests and .github scripts may still import them at module level, so TID253 joins their file-level exceptions; find-api-model-versions.py imports policyengine.

    Two v2 services imported sibling services at module level. Those imports now sit in the one function that uses each name: V2AccountService.export_account (four read_all_user_* readers) and mirror_legacy_user_policy_in_session (mirror_legacy_policy_in_session). Both modules already loaded only lazily, and no test patches those names.

  • Kept (16): see "Inline suppressions".

• Namespace packages (INP001, 45): policyengine_api has no __init__.py on purpose, and .github, migrations/v1, migrations/v2 and rules/fixtures aren't packages. They're declared in ruff's top-level namespace-packages setting, with a comment. Compared with the full ruff check output before the change, the only other difference is the hyphenated .github/find-api-model-versions.py (N999). A shell script runs it by that path, so N999 joins the .github/** file-level exceptions (api.quality.ci-cd-scripts-ruff-exemptions). No __init__.py was added, and the built wheel and sdist contain the same files.
• Names (N, A, SLF001): internal names are renamed: two constants in economy_comparison.py, the rules tool's BaseUnavailableError and assert_violation, SQLAlchemy's TimeoutError imported as PoolTimeoutError in tests, and 13 test function names that alone overflowed the line. Three reads of another object's private members now use public names, with no change in behaviour:
  • the in-memory cache pipeline reads its backend's lock through a new read-only operation_lock property;
  • ReportOutputService calls SimulationService.ensure_simulation_dual_write_state, formerly _ensure_simulation_dual_write_state, which now has a docstring;
  • worker_spm_release calls the new constants.get_bundle_manifest(), which reads the module attribute at call time, so tests that patch constants._policyengine_bundle still take effect.
• Line length (E501, 149): wrapped. Every split string has identical text, which a syntax-tree comparison confirms. The two published Field descriptions that were split are unchanged. Two are kept; see "Inline suppressions".
• Registry: api.quality.ruff-lint now lists the seven families and TID253. The test and .github exemption rules list the new codes. The rules tool changed (wrapped lines and the renames above), and sim takes the same changes in Enable ruff structure and import rules (INP, PLC, SLF, N, A, FAST, E) policyengine-sim-api#799, which has the same lazy-import mechanism.

Inline suppressions

This PR adds 19 and removes none (78 on master, 97 here). For each one, no code change, ruff setting or allowed file-level exception removes the finding without changing behaviour (api.process.agents-suppress-as-last-resort).

PLC0415, 16. Another application module imports each of these modules at module level, so it can't be listed in banned-module-level-imports. For country, metadata_service and specification, that importer is a single Flask module (services/metadata_service.py, routes/metadata_routes.py and routes/system_routes.py). Listing them would mean deferring imports in Flask code, which loads them at startup anyway.

Location	Imports	Reason
fastapi_routes/dependencies.py:704	libs.simulation_entrypoint	builds its client at import, which needs the gateway URL configured
fastapi_routes/dependencies.py:712	services.metadata_service	loads every country model; only the default reader needs it
fastapi_routes/dependencies.py:720	specification	reads and builds the OpenAPI document at import
fastapi_routes/dependencies.py:805	data.orm	looked up per call, so tests can replace the v1 session factory
fastapi_routes/v2/metadata/common.py:82	fastapi_routes.dependencies	looked up per request, so tests can replace the default factory
libs/simulation_entrypoint.py:260	constants	constants reads the installed bundle manifest at import
spm.py:460, spm.py:477	constants	constants reads the installed bundle manifest at import
services/household_calculation_service.py:134, :184	country	country imports this module and loads every country model
warmup.py:49	country	country models load during warmup, not at import
readiness.py:19, :25, :28, :34	data.v2.settings, libs.user_auth, migration_flags, spm	keeps importing readiness cheap; only validation needs configuration
services/v2/policies/validators.py:100	services.v2.policies.types	types imports this module (a cycle)

Others, 3:

Location	Code	Reason
country.py:699	N801	class reform: policyengine-core uses the class name as the reform key
services/v2/metadata/services.py:19	E501	a dotted module path can't be wrapped
scripts/measure_cloud_run_runtime.py:12	E501	the docstring is the script's --help text. Its 93-character first sentence can't be wrapped without breaking the docstring rules (D205, D400), and rewording it would change the help output

Testing

• pytest tests/unit at the head (before the changelog and comment-only commits): 2,686 passed.
• mypy over [tool.mypy].files is clean (167 source files).
• ruff check and ruff format --check (0.15.12) are clean repo-wide.
• Every Pydantic model schema, the FastAPI OpenAPI with every native router enabled, the /v2/openapi.json preview and the legacy spec were byte-identical to the base, with and without OLD_SIMULATION_GATEWAY_URL set. The later commits change no route, model or schema.
• The quality guards, the rules guard and the rules tool unittests pass.
• At each tip from Enable ruff structure and import rules (INP, PLC, SLF, N, A, FAST, E) #3920 to Register weighted-microdata rules, with AST guards and #767 proofs #3938 after this PR's last changes: ruff check (including RUF100 from Enable ruff argument, idiom and pytest-style rules (ARG, B, C4, FBT, PERF, PLW, PT, PTH, RUF, SIM and others) #3924 up, so no suppression is left unused) and ruff format --check are clean, and the quality guards and rules guard pass. The PRs above rebased with no conflicts outside pyproject.toml, the registry and the rendered rules. None of them refers to the three renamed private names.
• CI/CD: no workflow changed. .github/find-api-model-versions.py lost its suppression comment, and no environment variable was added, removed or renamed.
• CI: this PR is now the lowest in its stack, so it runs the full PR workflow. The run for the current head is queued.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 42.85714% with 60 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.29%. Comparing base (3f7deea) to head (94e0027).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
policyengine_api/fastapi_routes/dependencies.py	8.00%	23 Missing ⚠️
policyengine_api/services/economy_comparison.py	0.00%	8 Missing ⚠️
policyengine_api/readiness.py	0.00%	4 Missing ⚠️
policyengine_api/country.py	0.00%	2 Missing ⚠️
policyengine_api/services/economy_service.py	0.00%	2 Missing ⚠️
...gine_api/services/household_calculation_service.py	0.00%	2 Missing ⚠️
policyengine_api/services/household_mirroring.py	0.00%	2 Missing ⚠️
policyengine_api/services/policy_mirroring.py	0.00%	2 Missing ⚠️
policyengine_api/services/user_policy_mirroring.py	0.00%	2 Missing ⚠️
...gine_api/services/v2/comparison_runs/validators.py	0.00%	2 Missing ⚠️
... and 11 more

❗ There is a different number of reports uploaded between BASE (3f7deea) and HEAD (94e0027). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (3f7deea)	HEAD (94e0027)
	2	1

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #3920       +/-   ##
===========================================
- Coverage   84.08%   46.29%   -37.79%     
===========================================
  Files         255      239       -16     
  Lines       15368    13971     -1397     
  Branches     2555     2267      -288     
===========================================
- Hits        12922     6468     -6454     
- Misses       1686     6937     +5251     
+ Partials      760      566      -194     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.