diff --git a/README.md b/README.md index 4ab325b4..ac08a570 100644 --- a/README.md +++ b/README.md @@ -45,6 +45,8 @@ The current templates cover standard-variable and list-heavy planning models, an - `README.md` is the user-facing entry point for the workspace and generated-project integration model. - `docs/extend-solver.md` and `docs/extend-domain.md` cover scaffold extension workflows. - `docs/lifecycle-pause-resume-contract.md` defines the retained lifecycle contract, including exact pause/resume semantics, snapshot identity, and terminal-state cleanup rules. +- `docs/python-model-ir.md` outlines the proposed Python-first declarative model IR and the intended lowering contract into typed Rust solver code. +- `docs/python-path2-postmortem.md` compares Path 2 against the removed `solverforge-py` experiment, records hard guardrails, and explains why any implementation should live outside this workspace. - `docs/typed-contract-audit.md` records the current neutral selector and extractor naming model, including the `EntityCollectionExtractor`, `ValueSelector`, and `MoveSelector` surface adopted in `0.7.0`. - `crates/*/WIREFRAME.md` files are the canonical public API maps for each crate. - `AGENTS.md` defines repository-level engineering and documentation expectations for coding agents. diff --git a/docs/python-model-ir.md b/docs/python-model-ir.md new file mode 100644 index 00000000..6d46b833 --- /dev/null +++ b/docs/python-model-ir.md @@ -0,0 +1,95 @@ +# Python Model IR (Path 2: Codegen + Compile) + +This document defines the proposed Python-first model surface that should lower into typed SolverForge Rust code, then compile as a Rust crate in a standalone integration repository. + +## Goals + +- Preserve SolverForge zero-erasure and monomorphized hot paths. +- Let Python users model the same planning constructs and lifecycle workflows. +- Keep the Python/Rust bridge thin: compile once, run in Rust, stream lifecycle events. + +## Historical Context + +Path 2 guardrails were derived from the removed `solverforge-py` experiment; see `docs/python-path2-postmortem.md`. + +This workspace intentionally keeps Path 2 at the documentation level. Any Python implementation should live outside the SolverForge Rust workspace and consume the public `solverforge` API as a client. + +## Planned Modules + +A standalone Python integration should roughly split into: + +- IR schema and validation +- Expression lowering +- Rust code generation and project writing +- Build/runtime bridge around compiled generated crates + +## Design + +The IR is declarative and typed: + +- Domain declarations (`FactDef`, `EntityDef`, `VariableDef`, `SolutionDef`) +- Constraint declarations (`ConstraintDef`, `JoinSpec`, `FilterSpec`, `ImpactSpec`) +- Runtime configuration (`TerminationDef`, `SolverDef`) +- Top-level container (`ModelDef`) + +Expressions are represented as an AST (not executable Python callbacks): + +- `RefExpr` +- `ConstExpr` +- `CompareExpr` +- `BoolExpr` +- `CallExpr` + +## Lambda Lowering + +A convenience helper such as `lambda_to_expr(fn, aliases)` can lower a restricted subset of Python lambda/function syntax into the expression AST: + +- Attribute references from known stream aliases +- `==`, `!=`, `<`, `<=`, `>`, `>=` +- Boolean `and`, `or`, `not` +- Whitelisted calls (`contains`, `overlaps`, `len`) + +Unsupported constructs fail fast with `LambdaLoweringError`. + +## Validation + +`validate_model(model)` should perform structural validation: + +- Unique entity/fact names +- Solution collection references target known entities/facts +- Constraint source and join collection references exist +- Join-specific required fields are present (`left_key/right_key` for keyed joins, predicate for predicate joins) + +## Code Generation (Path 2) + +A generator such as `generate_rust_module(model)` should emit Rust source with: + +- Domain structs annotated by `#[problem_fact]`, `#[planning_entity]`, `#[planning_solution]` +- Typed `define_constraints()` function using `ConstraintFactory` and fluent stream builders +- Join lowering for `self_equal`, `cross_keyed`, and `cross_predicate` +- Filter/impact/name lowering per constraint + +A project writer such as `write_rust_project(model, out_dir, crate_name)` should write a compilable crate: + +- `Cargo.toml` +- `src/lib.rs` + +Returned metadata should point to generated paths and build artifacts. + +## Intended Lowering Contract + +The IR lowers into the Rust stream API: + +- `source(collection)` -> `ConstraintFactory::::new().()` +- `join(self_equal|cross_keyed|cross_predicate)` -> `.join(...)` +- `filter(expr)` -> `.filter(...)` +- `impact` -> `.penalize_*()` / `.reward_*()` +- `name` -> `.named(...)` + +This keeps solving and scoring in Rust while preserving Python modeling ergonomics. + +## Current Limitations + +- The first production scope should target common standard-variable patterns only. +- Advanced list-variable selectors/phases should be designed as a separate lowering track, not implied by the initial IR. +- Packaging, native build/import flow, and lifecycle bridging should live in the standalone Python integration repo rather than this Rust workspace. diff --git a/docs/python-path2-postmortem.md b/docs/python-path2-postmortem.md new file mode 100644 index 00000000..4ee4cc3c --- /dev/null +++ b/docs/python-path2-postmortem.md @@ -0,0 +1,88 @@ +# Python Path 2 vs Historical `solverforge-py` (Postmortem) + +This note compares the proposed Python Path 2 direction (IR -> Rust codegen -> compile) with the removed `crates/solverforge-py` experiment. + +## Historical Reference Point + +The latest commit where `crates/solverforge-py` still existed was: + +- `be76aaf` (2026-02-06) `refactor(py): remove Solver pyclass and unify API under SolverManager` + +The deletion happened at: + +- `559c57d` (2026-03-08) `chore: delete dynamic, py + all cranelift stuff; delete stub dotfiles that were used with zoyd` + +## Why the Old Experiment Failed (Structural Issues) + +### 1) Dynamic runtime model instead of typed compile-time model + +Old `solverforge-py` built solutions with dynamic descriptors and dynamic values at runtime: + +- `DynamicDescriptor`, `DynamicEntity`, `DynamicSolution`, `DynamicConstraintSet` +- runtime-defined classes and value ranges + +This created a separate dynamic execution path that diverged from the typed core. + +### 2) String-expression constraints + +Old constraints were built from string expressions like: + +- `"A.row == B.row"` +- `"field + 1"` + +and parsed at runtime with ad-hoc parsing logic (`parse_expr`, `parse_simple_expr`). + +This was fragile, hard to validate statically, and not aligned with typed stream APIs. + +### 3) Python API drift from Rust public API + +The old interface (`entity_class`, `add_entities`, string joins/filters) did not match the typed Rust modeling surface and lifecycle contracts. + +### 4) Lifecycle/telemetry mismatch + +The old manager API exposed coarse status strings and custom async controls, which did not map to retained `job/snapshot/checkpoint` semantics used in modern SolverForge. + +## Path 2 Correctives + +Path 2 intentionally avoids the above failure modes: + +1. **Typed IR, not dynamic runtime objects** + - Python describes model structure and expressions as a typed AST. + +2. **Compile to Rust, do not interpret in Python** + - Emit Rust structs/macros/constraint streams and compile. + +3. **No string DSL at runtime** + - Expressions are AST nodes lowered into Rust source. + +4. **Keep Rust as the only execution path** + - Scoring/moves/phases run in generated Rust. + +5. **Align with retained lifecycle contracts** + - Future bindings should expose `job`/`snapshot`/`events` directly rather than inventing a parallel lifecycle model. + +## Non-Negotiable Guardrails + +- No runtime expression parser for user strings. +- No dynamic scoring/move engine fork for Python. +- No Python callback execution in hot scoring/move loops. +- Generated code must target the same public SolverForge contracts used by Rust users. + +## Repository Boundary + +This workspace should remain Rust-first and docs-first for Path 2. + +- Keep the design and guardrails in this repository. +- Build any Python implementation in a standalone repository that depends on the published/public `solverforge` surface. +- Do not reintroduce a `python/` implementation subtree or a second solver runtime inside this workspace. + +## Current Status + +This repository now documents the intended direction in `docs/python-model-ir.md` and records the architectural guardrails here. + +Remaining work includes: + +- creating the standalone Python integration repository, +- pyproject/maturin packaging for produced crates, +- list-variable parity, +- lifecycle bridge that forwards retained runtime events as web/SSE-friendly payloads.