Skip to content

Require SQLAlchemy-only database connections and Alembic-only schema changes in every source directory #851

Closed

Description

@anth-volk

Problem

This repository's only relational database use today is the Stage 12 persistence library, which connects through a SQLAlchemy engine with the postgresql+psycopg driver name. Nothing keeps it that way, and the schema checks cover only part of the code.

  • Direct driver connections are not forbidden anywhere. No rule or guard stops code from importing psycopg, sqlite3 or another database driver and connecting with it directly. This has happened before. In 08ebe1c, the first Stage 12 persistence adapter, in policyengine-simulation-contract, imported psycopg and opened its connections with psycopg.connect, until 7e7fa00 replaced it with a SQLAlchemy engine.
  • The DDL check covers three directories. sim.stage12.no-schema-ddl matches its DDL pattern only against libs/policyengine-stage12-persistence/src, the entry project's src and the executor's src. The gateway, policyengine-fastapi, policyengine-simulation-contract, policyengine-simulation-observability and .github/scripts are not checked.
  • No check looks for Alembic imports, drop_all, Table.create, Table.drop or SQLAlchemy DDL statements. sim.stage12.no-migration-chain looks for Alembic configuration and revision directories. sim.stage12.persistence-typed-dml-only checks the persistence library for create_all( and drop_all( substrings only.

Proposal

Register sim.boundaries.sqlalchemy-and-alembic-only (enforced, mechanical): code connects to databases only through SQLAlchemy engines and connections, never by importing a database driver directly, and runtime code never changes the schema, which changes only through Alembic migrations.

  • The guard: a pytest guard in the entry project's tests. It parses every Python file under projects/*/src, libs/*/src and .github/scripts with ast, and rejects:
    • database driver imports;
    • alembic imports outside a migrations directory;
    • create_all and drop_all;
    • create() and drop() on objects it can identify as tables;
    • SQLAlchemy DDL statements;
    • DDL text in string literals.
  • Reuse: the guard uses the existing Stage 12 DDL_PATTERN rather than a new regex.
  • Proof: the psycopg connection from 08ebe1c, plus a runtime metadata.create_all(engine).
  • Docs: a database access skill doc.
  • Out of scope: non-relational stores (Modal Dicts, object storage and caches).

The wording doesn't say which repository's migrations own the schema, so it stays true now and after #844:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions