Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/workflow-linter.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,10 +82,10 @@ jobs:
if: matrix.python-version == '3.10'
run: uv sync --python 3.10 --frozen --extra dev --extra prefect

# Optional managed imports resolve only on their supported Python range.
- name: "Install managed dependencies on Python 3.11+"
# Optional service imports resolve only on their supported Python range.
- name: "Install service dependencies on Python 3.11+"
if: matrix.python-version != '3.10'
run: uv sync --python ${{ matrix.python-version }} --frozen --extra dev --extra prefect --extra managed
run: uv sync --python ${{ matrix.python-version }} --frozen --extra dev --extra prefect --extra service

- name: "Linting: ruff check"
run: "uv run ruff check ."
Expand All @@ -95,9 +95,9 @@ jobs:

- name: "Linting: ty check (direct Prefect Python 3.10)"
if: matrix.python-version == '3.10'
run: "uv run ty check --exclude infrahub_sync/managed --exclude tests/managed ."
run: "uv run ty check --exclude infrahub_sync/service --exclude tests/service ."

- name: "Linting: ty check (managed Python 3.11+)"
- name: "Linting: ty check (service Python 3.11+)"
if: matrix.python-version != '3.10'
run: "uv run ty check ."

Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/workflow-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,11 @@ jobs:
if: matrix.python-version == '3.10'
run: uv sync --python 3.10 --frozen --extra dev --extra prefect

# Managed HTTP support is Python 3.11+; the separate 3.10 leg still proves
# The Sync HTTP service is Python 3.11+; the separate 3.10 leg still proves
# the direct Prefect orchestration profile.
- name: "Install managed dependencies on Python 3.11+"
- name: "Install service dependencies on Python 3.11+"
if: matrix.python-version != '3.10'
run: uv sync --python ${{ matrix.python-version }} --frozen --extra dev --extra prefect --extra managed
run: uv sync --python ${{ matrix.python-version }} --frozen --extra dev --extra prefect --extra service

# Runs only `-m "not integration"`. Integration tests need a live
# Infrahub instance and are opted into via `invoke tests-integration`
Expand All @@ -81,7 +81,7 @@ jobs:
# the base package neither imports nor needs Prefect; only an install where
# Prefect is genuinely absent proves the base install still imports and the CLI
# still runs. The flow suite skips here, by design.
name: "Unit Tests (base install, Python 3.10, no managed runtimes)"
name: "Unit Tests (base install, Python 3.10, no service runtimes)"
needs: ["files-changed"]
if: needs.files-changed.outputs.sync == 'true'
runs-on: "ubuntu-22.04"
Expand All @@ -106,9 +106,9 @@ jobs:

# opsmill_prefect_extras is vendored first-party code shipped with the package,
# so it is present on disk in every install; the base-package guarantee that
# nothing imports it without the managed extra is enforced by
# nothing imports it without the service extra is enforced by
# tests/test_no_prefect_import.py.
- name: "Assert managed runtime dependencies are absent from this environment"
- name: "Assert service runtime dependencies are absent from this environment"
run: |
for module in boto3 botocore fastapi prefect psycopg; do
if uv run --no-sync python -c "import ${module}" 2>/dev/null; then
Expand Down
43 changes: 22 additions & 21 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
## Agent Operating Principles

1. **Plan → Ask → Act → Verify → Record** — plan briefly, ask for missing context, act with the smallest change, verify locally, then record with a concise commit or PR note.
2. **Default to read-only and dry runs** — prefer `list`, `diff`, and `generate` before `sync`. Write/apply only with explicit instruction and human approval.
2. **Default to read-only and dry runs** — prefer `configs` inspection, `runs plan`, and `diff` before `sync`. Write/apply only with explicit instruction and human approval.
3. **Be specific and reversible** — small, scoped commits. Don't mix large refactors with behavior changes in one PR.
4. **Match existing patterns** — keep CLI, adapters, examples, and directory structure consistent with the codebase.
5. **Idempotency and safety** — favor operations safe to re-run. Never print or guess secrets. Handle timeouts, auth, and network errors explicitly.
Expand All @@ -18,10 +18,10 @@ Use Python 3.11–3.13 for the full development profile. The former private
(frozen; see its `VENDORED.md`), so no special repository access is required:

```bash
uv sync --extra dev --extra prefect --extra managed
uv sync --extra dev --extra prefect --extra service
```

On Python 3.10, install the direct Prefect profile instead. Managed Sync supports Python
On Python 3.10, install the direct Prefect profile instead. Sync supports Python
3.11–3.13 only:

```bash
Expand All @@ -47,25 +47,25 @@ documented in [`dev/knowledge/quality-gates.md`](dev/knowledge/quality-gates.md)

The `prefect` extra is not optional for development: without it `ty` cannot resolve
`infrahub_sync/orchestration/`'s imports and `tests/orchestration/test_flow.py` skips
itself whole. On Python 3.11–3.13, the `managed` extra is required too; otherwise `ty`
cannot resolve the managed service's FastAPI and Prefect imports. On Python 3.10,
`invoke linter.lint-ty` and `invoke linter.lint-pylint` exclude `infrahub_sync/managed`
(ty also excludes `tests/managed`), matching CI's direct Prefect gate. CI also runs a base-install job that keeps the Prefect-free
itself whole. On Python 3.11–3.13, the `service` extra is required too; otherwise `ty`
cannot resolve the Sync service's FastAPI and Prefect imports. On Python 3.10,
`invoke linter.lint-ty` and `invoke linter.lint-pylint` exclude `infrahub_sync/service`
(ty also excludes `tests/service`), matching CI's direct Prefect gate. CI also runs a base-install job that keeps the Prefect-free
guarantee honest.

**CLI sanity after changes:**

```bash
uv run infrahub-sync --help
uv run infrahub-sync list --directory examples/
uv run infrahub-sync generate --name from-netbox --directory examples/
uv run infrahub-sync configs --help
uv run infrahub-sync runs --help
```

The `from-netbox` generation check is integration-backed. Before running it,
The `from-netbox` example check is integration-backed. Before running it,
follow the [NetBox demo tutorial](docs/docs/tutorials/netbox-demo-to-infrahub.mdx)
through **Generate the sync code**. That setup uses a fresh Infrahub instance,
loads the matching schema library, creates a current `nbt_...` NetBox demo
token, and installs `pynetbox`. The public demo data changes over time; the
through **Register the configuration package**. That setup uses a fresh Infrahub
instance, loads the matching schema library, creates a current `nbt_...` NetBox
demo token, and installs `pynetbox`. The public demo data changes over time; the
bounded live acceptance test records its current data preconditions in
`tests/integration/test_saved_plan_apply_integration.py`.

Expand All @@ -79,7 +79,7 @@ uv run invoke docs.docusaurus
**Policy:**

- New or changed code is Ruff-clean and typed where touched (docstrings, specific exceptions).
- The codebase is clean under ty with no `[[tool.ty.overrides]]` blocks in `pyproject.toml`. Don't reintroduce overrides to mask type errors — fix the underlying issue, or use a targeted `# ty: ignore[<rule>]` with a short TODO at the call site. Run `uv run ty check .` in the full Python 3.11–3.13 profile (the frozen vendored upstream tests are excluded via `[tool.ty.src]` in `pyproject.toml`; the vendored package itself stays checked). On Python 3.10, run `uv run ty check --exclude infrahub_sync/managed --exclude tests/managed .`.
- The codebase is clean under ty with no `[[tool.ty.overrides]]` blocks in `pyproject.toml`. Don't reintroduce overrides to mask type errors — fix the underlying issue, or use a targeted `# ty: ignore[<rule>]` with a short TODO at the call site. Run `uv run ty check .` in the full Python 3.11–3.13 profile (the frozen vendored upstream tests are excluded via `[tool.ty.src]` in `pyproject.toml`; the vendored package itself stays checked). On Python 3.10, run `uv run ty check --exclude infrahub_sync/service --exclude tests/service .`.
- If you add tests, run `uv run pytest -q`.

## Repository Structure
Expand All @@ -104,10 +104,11 @@ Available adapters (`infrahub_sync/adapters/`): `infrahub`, `netbox`, `nautobot`

## CLI Commands

- `infrahub-sync list` — show available sync projects (safe).
- `infrahub-sync diff` — compute differences (safe).
- `infrahub-sync generate` — generate Python from YAML config (servers required).
- `infrahub-sync sync` — perform synchronization (servers and approval required).
- `infrahub-sync configs` — register and inspect configuration packages (safe).
- `infrahub-sync runs plan` — review a saved plan (safe).
- `infrahub-sync diff` — create a plan run and review its summary (Sync API required).
- `infrahub-sync sync` — perform synchronization (Sync API and approval required).
- `infrahub-sync apply` — apply a reviewed plan by checksum (Sync API and approval required).

## Configuration and Examples

Expand Down Expand Up @@ -176,7 +177,7 @@ uv run rumdl fmt . # fix
## Known Issues and Limitations

- Optional dependencies (e.g. `pynetbox`, `pynautobot`) may be missing, producing import warnings.
- `generate` and `sync` require running servers (Infrahub, NetBox, Nautobot).
- `diff`, `sync`, and `apply` require a running Sync API and its destination servers.
- Docs npm audit may flag dev-only vulnerabilities; they do not affect the Python package.

## Git and PR Process
Expand All @@ -192,7 +193,7 @@ uv run rumdl fmt . # fix

- [ ] Format and lint clean on changed areas.
- [ ] The type-check command for the active Python profile exits 0; new code typed.
- [ ] CLI behaviors validated (`--help`, `list`, targeted `generate`).
- [ ] CLI behaviors validated (`--help`, `configs --help`, `runs --help`).
- [ ] Docs updated if flags or config changed.
- [ ] Error handling uses specific exception types and clear messages.

Expand Down Expand Up @@ -223,7 +224,7 @@ step-by-step procedure. Supporting developer reference lives under `dev/`:
- [Adapter guidelines](dev/guidelines/README.md) — the rules for writing and testing an adapter.
- [Adapter guides](dev/guides/README.md) — adding and testing an adapter, step by step.

Core rule unchanged: provide read-only `list` / `diff` pathways and validate them before enabling `sync`.
Core rule unchanged: provide a read-only `diff` pathway and validate it before enabling `sync`.

## Beyond Adapters

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,11 +136,11 @@ may change.
| CLI | Configuration registration, plan review, run admission, and reviewed-plan apply | Base installation and Sync API access |
| Python client | Typed access to every shipped Sync API resource | Base installation and Sync API access |
| Direct Prefect deployment | Starting and observing one plan or confirmed sync through Prefect's API | `prefect` extra and a Prefect server |
| Managed Sync HTTP API | Authenticated remote runs, durable records and artifacts, reviewed apply, idempotency, and cancellation | `managed` extra, Prefect, a work pool, a worker, and shared durable storage |
| Sync HTTP API | Authenticated remote runs, durable records and artifacts, reviewed apply, idempotency, and cancellation | `service` extra, Prefect, a work pool, a worker, and shared durable storage |

See the [Python API](https://docs.infrahub.app/sync/reference/python-api),
[Prefect remote run](https://docs.infrahub.app/sync/reference/prefect-remote-run), and
[managed HTTP API](https://docs.infrahub.app/sync/reference/managed-http-api) references
[Sync HTTP API](https://docs.infrahub.app/sync/reference/sync-http-api) references
for their contracts and setup. For a bounded checkout-based live review, follow the
[`custom-example` plan and apply guide](examples/custom_adapter/README.md). Its source
fixture is deterministic; the review still uses a live, writable Infrahub destination.
Expand Down
1 change: 0 additions & 1 deletion changelog/+managed-cancellation-acknowledgement.fixed.md

This file was deleted.

1 change: 1 addition & 0 deletions changelog/+service-cancellation-acknowledgement.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fixed Sync API cancellation requests being reported as accepted when Prefect aborted, deferred, or returned an invalid state transition result.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Added an authenticated managed Sync HTTP API for creating and inspecting durable runs,
Added an authenticated Sync HTTP API for creating and inspecting durable runs,
reviewing retained plans, running read-only verification, applying an exact approved
checksum, retrieving results and artifacts, and requesting cancellation through Prefect.
Actor-scoped durable mutation receipts make exact retries converge on one Sync run and
Prefect flow run without storing the raw client key. Install the `managed` extra to run the
Prefect flow run without storing the raw client key. Install the `service` extra to run the
separate Prefect deployment and API; the base CLI and existing four-parameter direct
Prefect deployment remain unchanged.
2 changes: 0 additions & 2 deletions changelog/+standalone-product-projection.added.md

This file was deleted.

16 changes: 8 additions & 8 deletions dev/constitution.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ principles below put safety, reproducibility, and connector consistency ahead of

The non-mutating path is the default path, and applying changes is always a deliberate act.

- `list`, `diff`, and `generate` are non-applying and MUST stay safe to run at any time,
against any environment, without approval.
- `configs` inspection, `runs plan`, and `diff` are non-applying and MUST stay safe to run
at any time, against any environment, without approval.
- `sync` mutates a destination system and MUST require explicit user instruction,
confirmed target servers, and human approval. It MUST NOT run as an implicit side
effect of another command.
Expand Down Expand Up @@ -62,11 +62,11 @@ Adapters are the primary extension point; every connector MUST honor the same co

- A new adapter MUST live in `infrahub_sync/adapters/<name>.py` and follow the existing
adapter patterns rather than inventing new structure.
- It MUST provide `list` and `diff` pathways before `sync` is enabled.
- It MUST provide a `diff` pathway before `sync` is enabled.
- It MUST ship a connection config schema and a sanitized example under `examples/`.
- It MUST document required environment variables and expected error cases, and add a page
under `docs/docs/adapters/`.
- `list` / `diff` / `generate` / `sync` MUST flow through the core sync engine (`potenda`);
- `diff` / `sync` / `apply` MUST flow through the core sync engine (`potenda`);
no ad-hoc per-adapter sync logic that bypasses it.

**Rationale:** Consistent adapters keep the CLI predictable, make each new connector
Expand Down Expand Up @@ -124,8 +124,8 @@ Prefer the simplest solution that works and matches the patterns already in the
- YAGNI: build what the task needs, not speculative abstraction. A new abstraction needs
at least two real callers.
- New dependencies MUST be justified.
- Generated code (the Python that `generate` produces from YAML configs) MUST be regenerated
from its YAML source, never hand-edited.
- Generated code (the Python the internal generator produces from YAML configs) MUST be
regenerated from its YAML source, never hand-edited.
- Keep commits small and scoped; do not mix large refactors with behavior changes.

**Rationale:** A connector library accretes complexity quickly. Keeping each change small,
Expand Down Expand Up @@ -167,8 +167,8 @@ After changes, verify the CLI still behaves:

```bash
uv run infrahub-sync --help
uv run infrahub-sync list --directory examples/
uv run infrahub-sync generate --name from-netbox --directory examples/
uv run infrahub-sync configs --help
uv run infrahub-sync runs --help
```

### Logging
Expand Down
2 changes: 1 addition & 1 deletion dev/guidelines/secret-redaction.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> Part of: `dev/guidelines/` | Related: [The shared execution surface](../knowledge/execution-surface.md), [Writing an adapter](writing-an-adapter.md)

<!-- Extracted from dev/specs/archive/001-prefect-managed-remote-run on 2026-07-31 -->
<!-- Extracted from the archived prefect remote-run spec (dev/specs/archive/001, commit 33817cf) on 2026-07-31 -->

Rules for any code path that renders a failure across a process boundary — a served
deployment, an API response, a queued job. The adapter rule ("never log a secret") is about
Expand Down
2 changes: 1 addition & 1 deletion dev/guidelines/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> Part of: `dev/guidelines/` | Related: [Testing adapters](testing-adapters.md), [Quality gates](../knowledge/quality-gates.md)

<!-- Extracted from dev/specs/archive/001-prefect-managed-remote-run on 2026-07-31 -->
<!-- Extracted from the archived prefect remote-run spec (dev/specs/archive/001, commit 33817cf) on 2026-07-31 -->

Repository-wide rules for tests. [Testing adapters](testing-adapters.md) covers what an
adapter must ship; this document covers what makes any test worth having.
Expand Down
16 changes: 8 additions & 8 deletions dev/guides/adding-an-adapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,8 +133,8 @@ for this destination, or apply against a destination whose adapter implements th
planned-write surface.
```

Nothing else about the adapter degrades: `list`, `diff`, `sync` and plan *review*
(`diff --from-plan <run-id>`) all work unchanged. Only `apply` is unavailable. `infrahub` is
Nothing else about the adapter degrades: `diff`, `sync` and plan *review*
(`runs plan RUN_ID`) all work unchanged. Only `apply` is unavailable. `infrahub` is
the only one of the nine adapters shipped in this repository that implements the surface
today; the other eight refuse an `apply` exactly as described above.

Expand Down Expand Up @@ -243,13 +243,13 @@ Validate read-only paths before ever running `sync`:
uv run invoke format
uv run invoke lint

uv run infrahub-sync list --directory examples/
uv run infrahub-sync generate --name mysystem-example --directory examples/mysystem_to_infrahub/
uv run infrahub-sync diff --name mysystem-example --directory examples/mysystem_to_infrahub/
uv run infrahub-sync configs register --file examples/mysystem_to_infrahub/config.yml --reason "add mysystem example"
uv run infrahub-sync configs validate --config-id mysystem-example --version 1
uv run infrahub-sync diff --config-id mysystem-example --version 1 --reason "verify the new adapter"
```

`list` and `generate` need no live source; `diff` reads both sides but writes nothing. Run
`sync` only with explicit approval against a known-safe target.
`configs register` and `configs validate` read no source; `diff` plans against both sides but
writes nothing. Run `sync` only with explicit approval against a known-safe target.

## Quality checklist

Expand All @@ -258,7 +258,7 @@ uv run infrahub-sync diff --name mysystem-example --directory examples/mysystem_
- [ ] Decided whether the adapter implements the planned-write surface — **both** `new_peer_resolver` and `apply_planned_operation`; if it does not, confirmed that `apply` refuses cleanly and that `sync` is the documented path for it.
- [ ] Optional SDK imported with `# ty: ignore[unresolved-import]`; credentials from env vars; no secrets logged or committed.
- [ ] `uv run invoke format` and `uv run invoke lint` are clean; `uv run ty check .` exits 0.
- [ ] `list` / `generate` / `diff` succeed for the example.
- [ ] `configs register` / `configs validate` / `diff` succeed for the example.
- [ ] Unit tests added under `tests/adapters/`; `uv run pytest -q` passes offline.
- [ ] Example added under `examples/`; env vars documented.
- [ ] Documentation page added under `docs/docs/adapters/` and in the sidebar.
Expand Down
2 changes: 1 addition & 1 deletion dev/knowledge/execution-surface.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> Part of: `dev/knowledge/` | Related: [Sync architecture](sync-architecture.md), [Prefect orchestration](orchestration-prefect.md)

<!-- Extracted from dev/specs/archive/001-prefect-managed-remote-run on 2026-07-31 -->
<!-- Extracted from the archived prefect remote-run spec (dev/specs/archive/001, commit 33817cf) on 2026-07-31 -->

`infrahub_sync/execution.py` is the typed Python entry point to a single sync run. It exists
because the plan and serial-sync lifecycles need more than one caller: the CLI drives them
Expand Down
Loading
Loading