Skip to content

Governance expansion and risk remediation#42

Merged
Polichinel merged 12 commits intodevelopmentfrom
feature/governance-expansion-and-risk-remediation
Apr 7, 2026
Merged

Governance expansion and risk remediation#42
Polichinel merged 12 commits intodevelopmentfrom
feature/governance-expansion-and-risk-remediation

Conversation

@Polichinel
Copy link
Copy Markdown
Collaborator

@Polichinel Polichinel commented Apr 7, 2026

Summary

  • Partition source-of-truth: meta/partitions.json + migration tool (scripts/update_partitions.py) + ADR-011. Override mechanism for declared deviations. Fixed ucdp_extractor copy-paste boundary error, deleted dead rude_boy root config.
  • Test expansion: 22 characterization tests for tooling scripts (including 6 for migration tool), 9 scaffold builder tests (AST-based injection seam verification), expanded partition tests covering models/ensembles/extractors/postprocessors. 2649 tests total.
  • Code hardening: Atomic file writes in create_catalogs.py, injection seams in scaffold builders (input_fn, get_version_fn, pipeline_config), CI set -e for catalog workflow.
  • Tech debt cleanup: Removed dead code in update_readme.py/generate_features_catalog.py, fixed rude_boy missing requirements.txt, fixed purple_alien pinned dependency.
  • Risk register: Expanded from initial assessment to 34 entries (30 concerns + 4 disagreements). Resolved 11, mitigated 7, accepted 3. C-05 (cross-repo ReproducibilityGate) tracked with upstream stories in views-stepshifter and views-baseline — both now implemented.

Risk register status

Status Count
Open 9
Mitigated 7
Resolved 11
Accepted 3
Disagreements 4

Test plan

  • pytest — 2649 passed, 35 skipped
  • ruff check — no new lint issues (pre-existing only in untouched files)
  • scripts/update_partitions.py --dry-run — 73 files scanned, 0 updates needed
  • Verify meta/partitions.json values match production boundaries

Note on diff size

~80 of the 112 changed files are mechanical bulk changes to model main.py files (removing warnings.filterwarnings("ignore") and unifying DARTS manager instantiation). The meaningful logic changes are in ~30 files.

Polichinel and others added 12 commits April 6, 2026 13:00
… risks

Expand governance framework and address 10 risk register items identified
during repo-assimilation, expert-code-review, and test-review audits.

Governance additions:
- ADR-004: activate evolution and stability rules (three-tier classification)
- ADR-010: technical risk register as governance artifact
- CICs: PackageScaffoldBuilder, IntegrationTestRunner
- Standard: physical architecture (heuristic, not strict)
- Risk register: 30 entries seeded from audits

Code changes:
- Remove warnings.filterwarnings("ignore") from all 74 main.py files (C-12)
- Migrate white_mustang to new CLI pattern, remove wandb.login() (C-25)
- Unify DARTS/HydraNet manager instantiation to single-variable pattern (C-24)
- Delete placeholder models thousand_miles and thrift_shop (C-09)
- Add ViewsMonth-to-date mapping comment in config_partitions.py (C-21)

Test additions:
- test_algorithm_coherence.py: validate algorithm-to-package mapping (C-04, C-08)
- Extend test_cli_pattern.py to cover ensembles + add warning suppression check
- Add 5 red-team tests to test_failure_modes.py (C-15, C-23)

Risk register status: Open 17 | Mitigated 4 | Resolved 3 | Accepted 3

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…workflow

Remove unused imports/variables in tooling scripts, fix PEP 8 None comparisons,
create missing rude_boy requirements.txt, normalize purple_alien version pinning,
and add set -e to catalog CI step to catch silent failures.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Create meta/partitions.json as single source of truth for partition
boundaries. Add scripts/update_partitions.py to rewrite all 73 config
files from it. Extend test_config_partitions.py to cover extractors and
postprocessors, read canonical values from source-of-truth, detect
undeclared overrides, and check for data leakage.

Fix ucdp_extractor's non-standard boundaries (copy-paste from stale
rude_boy root file). Delete dead rude_boy root-level config_partitions.py.
Add gitignore negation rules for requirements.txt and partitions.json.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 16 tests pinning the behavior of pure functions in create_catalogs.py,
update_readme.py, and generate_features_catalog.py. Covers table generation,
section replacement, repo structure rendering, and queryset Column regex
extraction. Scripts cannot be imported directly (views_pipeline_core
dependency) so logic is duplicated as characterization tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace direct open(path, "w") with tempfile.NamedTemporaryFile + os.replace
so a crash mid-write leaves only a temp file, not a corrupted README.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…del_scripts

Add optional input_fn and get_version_fn keyword args to ModelScaffoldBuilder
and pipeline_config to EnsembleScaffoldBuilder, enabling mocked testing without
stdin or network. Fix == False to not in package validation. Add 9 tests
(7 AST-based seam verification + 2 functional, skipped without views_pipeline_core).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Feature branches feature/reproducibilitygate pushed to views-stepshifter
and views-baseline with story docs proposing ReproducibilityGate contracts.
Patch approach (hardcoding keys in views-models) rejected as unsustainable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…stream

views-baseline now has a full ReproducibilityGate (ADR-014, CIC, 13 tests)
on feature/reproducibilitygate. views-stepshifter still needs implementation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… upstream

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…al tests

ModelScaffoldBuilder requires adjective_noun naming (validated by
ModelPathManager). Tests used "test_scaffold_model" which fails in CI
where views_pipeline_core is installed. Also adds error handling to
update_partitions.py, trailing newline to .gitignore, and sync guidance
to test_tooling_scripts.py docstring.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…in functional tests

build_model_scripts() references self.requirements_path which is set by
build_model_directory(). Functional tests skip directory creation, so the
attribute must be set manually in test setup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…oupling

Add 6 tests for scripts/update_partitions.py covering: already_current,
stale value update, dry-run, override skip, forecasting offset, and
missing file error. Document private attribute access in scaffold builder
functional tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Polichinel Polichinel merged commit 1fd3161 into development Apr 7, 2026
1 check passed
@Polichinel Polichinel deleted the feature/governance-expansion-and-risk-remediation branch April 7, 2026 09:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant