feat: host-driven runs, leases and adapters for v1.10.0

.github/workflows/ci.yml

@@ -73,6 +73,7 @@ jobs:
           - integrations/fabriq
           - integrations/sandbox
           - optimization
+          - mcpbridge
           - a2aremote
           - a2aremote/grpcbind
 

.github/workflows/release.yml

@@ -59,7 +59,7 @@ jobs:
 
       - name: Verify sub-modules
         run: |
-          for mod in api sentinel extension integrations/fabriq integrations/sandbox optimization; do
+          for mod in api sentinel extension integrations/fabriq integrations/sandbox optimization mcpbridge; do
             echo "::group::Verifying $mod"
             pushd "$mod" > /dev/null
             go mod download
@@ -103,10 +103,10 @@ jobs:
           # the sub-module tags alone, so main keeps pointing sub-modules at
           # the last release and stays buildable against the working tree.
           scripts/pin-submodules.sh "$TAG"
-          git add api/go.mod sentinel/go.mod extension/go.mod integrations/fabriq/go.mod integrations/sandbox/go.mod optimization/go.mod
+          git add api/go.mod sentinel/go.mod extension/go.mod integrations/fabriq/go.mod integrations/sandbox/go.mod optimization/go.mod mcpbridge/go.mod
           git commit -m "chore: pin sub-modules to $TAG"
 
-          for mod in api sentinel extension integrations/fabriq integrations/sandbox optimization; do
+          for mod in api sentinel extension integrations/fabriq integrations/sandbox optimization mcpbridge; do
             MOD_TAG="$mod/$TAG"
             if git rev-parse -q --verify "refs/tags/$MOD_TAG" > /dev/null; then
               echo "$MOD_TAG already exists, skipping"
@@ -128,7 +128,7 @@ jobs:
           PROBE=$(mktemp -d)
           cd "$PROBE"
           go mod init example.com/probe
-          for mod in api sentinel extension integrations/fabriq integrations/sandbox optimization; do
+          for mod in api sentinel extension integrations/fabriq integrations/sandbox optimization mcpbridge; do
             echo "::group::go get github.com/xraph/cortex/$mod@$TAG"
             GOFLAGS=-mod=mod GOPROXY=proxy.golang.org go get "github.com/xraph/cortex/$mod@$TAG"
             echo "::endgroup::"
@@ -176,5 +176,6 @@ jobs:
             go get github.com/xraph/cortex/integrations/fabriq@${{ steps.set-tag.outputs.tag }}
             go get github.com/xraph/cortex/integrations/sandbox@${{ steps.set-tag.outputs.tag }}
             go get github.com/xraph/cortex/optimization@${{ steps.set-tag.outputs.tag }}
+            go get github.com/xraph/cortex/mcpbridge@${{ steps.set-tag.outputs.tag }}
             ```
           generate_release_notes: true

AGENTS.md

@@ -1,6 +1,6 @@
 # AGENTS.md
 
-Cortex is an agent orchestration engine for Go. You compose agents from skills, traits, behaviors and personas, run them through a ReAct loop with tools, memory, checkpoints and suspensions, and coordinate several of them with orchestration strategies and agent-to-agent messaging. The root module is the engine on its own. Anything that drags in a heavy dependency (forge, fabriq, gRPC) lives in a module of its own.
+Cortex is an agent orchestration engine for Go. You compose agents from skills, traits, behaviors and personas, run them through a ReAct loop with tools, memory, checkpoints and suspensions, and coordinate several of them with orchestration strategies and agent-to-agent messaging. The root module is the engine on its own. Anything that drags in a heavy dependency (forge, fabriq, gRPC, the MCP SDK) lives in a module of its own.
 
 | Module | Directory | Tagged by release |
 |---|---|---|
@@ -10,6 +10,8 @@ Cortex is an agent orchestration engine for Go. You compose agents from skills,
 | `github.com/xraph/cortex/extension` | `extension` | `extension/vX.Y.Z` |
 | `github.com/xraph/cortex/integrations/fabriq` | `integrations/fabriq` | `integrations/fabriq/vX.Y.Z` |
 | `github.com/xraph/cortex/integrations/sandbox` | `integrations/sandbox` | `integrations/sandbox/vX.Y.Z` (first release pending) |
+| `github.com/xraph/cortex/optimization` | `optimization` | `optimization/vX.Y.Z` |
+| `github.com/xraph/cortex/mcpbridge` | `mcpbridge` | `mcpbridge/vX.Y.Z` |
 | `github.com/xraph/cortex/a2aremote` | `a2aremote` | no |
 | `github.com/xraph/cortex/a2aremote/grpcbind` | `a2aremote/grpcbind` | no |
 | `github.com/xraph/cortex/inttest` | `inttest` | no, never published |
@@ -50,7 +52,7 @@ cd extension && GOWORK=off go test -race -count=1 ./...
 |---|---|---|
 | `ci` (shared `xraph/workflows` `go-ci.yml@v1`, Go 1.26) | root | `make test`, `make test-coverage` (race + atomic coverage, 30-minute package timeout), golangci-lint at the version pinned there, `gofmt -l .`, `make vet`, a `go mod tidy` diff, gosec and govulncheck (both gating), and a check that nested modules require cortex at versions the proxy can serve |
 | `format` | whole tree | `goimports -l -local github.com/xraph/cortex .` must print nothing |
-| `submodules` | `api`, `sentinel`, `extension`, `integrations/fabriq`, `integrations/sandbox`, `optimization`, `a2aremote`, `a2aremote/grpcbind` | `go mod tidy` diff, `go build ./...`, `go test -race -count=1 ./...` |
+| `submodules` | `api`, `sentinel`, `extension`, `integrations/fabriq`, `integrations/sandbox`, `optimization`, `mcpbridge`, `a2aremote`, `a2aremote/grpcbind` | `go mod tidy` diff, `go build ./...`, `go test -race -count=1 ./...` |
 | `docs` | `docs/` | `pnpm install --frozen-lockfile`, `pnpm types:check`, `pnpm lint`, `pnpm build` |
 
 `a2aremote` and `a2aremote/grpcbind` now run in the sub-module CI matrix. `inttest` is still outside that matrix. Run `make test-integration` when you change it, and run `make lint` locally for every module.
@@ -119,6 +121,15 @@ Sentinels live in `errors.go` with a `cortex: ` prefix. Wrap with `%w` and the s
 
 `store.Store` embeds every subsystem's store interface plus `Migrate(ctx, ...cortex.MigrateOption)`, `Ping` and `Close`. Backends build on a `*grove.DB`. Migrations are registered in Go in each backend's `migrations.go`. The `.sql` files under `store/postgres/migrations/` aren't loaded by anything.
 
+Four capabilities sit outside `store.Store`. The engine type-asserts each one in `New` and leaves the feature off when a backend lacks it, so a custom store keeps compiling:
+
+- `run.ContinuationStore` (`run/continuation.go`): ready and reclaimable runs, leases and the stored principal. `DecideCheckpoint`, `ProvideResults` and `ContinueRun` need it, and without it they return `cortex.ErrContinuationUnavailable`.
+- `suspension.ProgressStore` (`suspension/progress.go`): the progress snapshot a run saves at each step boundary, fenced by the lease owner. Leasing needs it alongside `run.ContinuationStore`.
+- `suspension.LeasedClaimStore` (`suspension/leased.go`): claims that take the lease and write the first snapshot in the same write that moves the run to `running`.
+- `checkpoint.SweepStore` (`checkpoint/store.go`): lets the sweeper find pending checkpoints no decision can reach any more and close them.
+
+The SQLite, PostgreSQL and MongoDB stores implement all four, through `store/internal/assurance` where the logic is shared. When you change one, change the other two and run `store/storetest`'s conformance suite (`RunContinuation`, `RunProgress`, `LeasedClaims`, `CheckpointSweep`) on all three. Postgres and Mongo need Docker, and they skip without it, so check the output for SKIP.
+
 ### Engine, plugins and the Forge extension
 
 `engine.New(opts ...engine.Option)` takes `func(*Engine) error` options such as `WithStore`, `WithLLM`, `WithSafety`, `WithKnowledge`, `WithTool`, `WithToolAuthorizer` and `WithLogger`. The engine logs through `github.com/xraph/go-utils/log` and defaults to `log.NewNoopLogger()`; the Forge extension passes the app's logger. Log a failure with `e.logError` or `e.logWarn` (engine/logging.go): they log the error's type, never its text, and throttle each message to a line a minute. Fields are IDs and counts, never run content or scope values.
@@ -170,10 +181,10 @@ You release by dispatching the workflow. Never push a tag by hand.
 `.github/workflows/release.yml` runs these steps:
 
 1. It tags `main` with the input as `github-actions[bot]` and pushes the tag. Nothing checks the input before that push (`pin-submodules.sh` validates it two steps later), so type the leading `v`.
-2. It verifies the root with `go mod download`, a `go mod tidy` diff, `go build ./...` and `go test -race -count=1 ./...`, then does the same in `api`, `sentinel`, `extension`, `integrations/fabriq` and `integrations/sandbox`.
-3. It runs `scripts/pin-submodules.sh vX.Y.Z`. The script validates the version, then uses `go mod edit -require` to point every `github.com/xraph/cortex...` require in those five `go.mod` files at the new tag, and it fails the run if it finds a cortex require still at `v0.0.0` (a `v0.0.0-...` pseudo-version counts).
-4. It commits the five `go.mod` files as `chore: pin sub-modules to vX.Y.Z`, tags that commit `api/vX.Y.Z`, `sentinel/vX.Y.Z`, `extension/vX.Y.Z`, `integrations/fabriq/vX.Y.Z` and `integrations/sandbox/vX.Y.Z` (skipping any that already exist), pushes the tags and runs `git reset --hard HEAD~1`.
-5. In a throwaway module it runs `go get github.com/xraph/cortex/<mod>@vX.Y.Z` against `proxy.golang.org` for each of the five, so a tag that can't resolve fails the run.
+2. It verifies the root with `go mod download`, a `go mod tidy` diff, `go build ./...` and `go test -race -count=1 ./...`, then does the same in `api`, `sentinel`, `extension`, `integrations/fabriq`, `integrations/sandbox`, `optimization` and `mcpbridge`.
+3. It runs `scripts/pin-submodules.sh vX.Y.Z`. The script validates the version, then uses `go mod edit -require` to point every `github.com/xraph/cortex...` require in those seven `go.mod` files at the new tag, and it fails the run if it finds a cortex require still at `v0.0.0` (a `v0.0.0-...` pseudo-version counts).
+4. It commits the seven `go.mod` files as `chore: pin sub-modules to vX.Y.Z`, tags that commit `api/vX.Y.Z`, `sentinel/vX.Y.Z`, `extension/vX.Y.Z`, `integrations/fabriq/vX.Y.Z`, `integrations/sandbox/vX.Y.Z`, `optimization/vX.Y.Z` and `mcpbridge/vX.Y.Z` (skipping any that already exist), pushes the tags and runs `git reset --hard HEAD~1`.
+5. In a throwaway module it runs `go get github.com/xraph/cortex/<mod>@vX.Y.Z` against `proxy.golang.org` for each of the seven, so a tag that can't resolve fails the run.
 6. It publishes a GitHub release with notes from `git log` since the previous tag.
 
 The pin commit is never pushed to a branch, and only the sub-module tags reach it. So `main` keeps each sub-module's requires at an earlier release with `replace => ../` for local work, while a consumer who fetches `cortex/extension@vX.Y.Z` resolves the pinned requires from the tag, which means a sub-module tag and the root tag of the same version point at different commits: the root tag at the `main` commit, the sub-module tags at the pin commit on top of it.

CHANGELOG.md

@@ -4,8 +4,67 @@ All notable changes to this project are documented in this file.
 
 The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
+## [1.10.0] - 2026-10-29
+
+v1.9.0 went out on 2026-10-27 and carried the work that was on `main` before this, assured execution included. That part is experimental and stays off unless you turn it on. If you run v1.9.0 today, read "Upgrading from v1.9.0" in `MIGRATION.md` before you roll this out. Coming from v1.8.0, read "Upgrading from v1.8.0" too.
+
+This release lets your host drive a run from outside the request that started it, and, once you turn leasing on, makes a run survive the worker that was running it. It's built on forge v1.13.2, grove v1.10.1, nexus v1.9.1, shield v1.9.1 and weave v1.9.2.
+
+### Added
+
+- Your host can drive a run now. Runs have two new states, `ready` and `reclaimable`. `eng.DecideCheckpoint` records an approval or a rejection, `eng.ProvideResults` records an external tool's results, and both move the run to `ready` and return without calling the model. `eng.ContinueRun` picks the run up later, from a job on any replica. The `plugin.RunReady` and `plugin.RunReclaimable` hooks are where you queue that job, and `WithReadyRenotify` tells you again about a run nobody picked up.
+- Run leases are opt-in. Turn them on with `engine.WithRunLease(ttl)` once your host continues `RunReady` and `RunReclaimable` runs with `ContinueRun`, because nothing in cortex does that for you. On a store that implements `run.ContinuationStore` and `suspension.ProgressStore` (the SQLite, PostgreSQL and MongoDB stores all do), every run then executes under its worker's lease and saves its progress at each step boundary. When a worker dies, the sweeper marks its runs `reclaimable` and `ContinueRun` carries on from the last saved step. A call that was in flight runs again only if the tool is retry-safe. `WithReclaimLimit` and `WithWorkerID` tune it. `MIGRATION.md` and the Runs docs show the plugin that queues the job.
+- `WithRunConcurrency` caps how many runs execute at once in a process. `Stop` refuses new runs with `cortex.ErrEngineStopping` and waits for the ones in flight. With leasing on, it hands leased runs to another replica.
+- Each pause gets its own deadline. `WithSuspensionTTL` is now a default. An approval can carry its own `TTL` and an external tool its own `engine.ExternalTTL`, and a pause expires at the earliest deadline among its calls. `plugin.RunSuspended` fires for every pause with the checkpoint ID and the deadline, so you can tell someone a decision is waiting.
+- `engine.WithPrincipalCodec` stores the principal that started a run, and a resumed, continued or reclaimed run makes its tool calls as that principal. `cortex.JSONPrincipalCodec` is enough for most hosts.
+- The new `tool` package builds a tool from a Go function over a struct: it generates the JSON Schema and validates every call before your function runs. Mark the tools that are safe to run twice with `tool.RetrySafe()` or `engine.WithToolRetrySafe`.
+- The new `mcpbridge` module registers an MCP server's tools as cortex tools, and the ones the server marks read-only are retry-safe. It's a module of its own, so the root never pulls in the MCP SDK.
+- `Migrate` on PostgreSQL takes a lock and waits while another replica migrates, the rescope and session backfill included. Migrations `20261101000001` to `20261101000004` add the stored principal, the lease and ready columns, the progress table and the reply route column.
+- The Forge extension now applies `default_model`, `default_max_steps`, `default_max_tokens`, `default_temperature`, `default_reasoning_loop`, `run_concurrency` and `shutdown_timeout`. `WithNexusOptions`, `WithWeaveOptions` and `WithShieldOptions` configure the adapters it finds, and `WithoutAutoAdapters` skips them.
+- Pass `api.WithAuthorizer` and every REST route asks your `api.Authorizer` before it runs; its `DecidedBy` names the person behind a checkpoint decision. `GET /v1/tools` lists the engine's tool catalog with each tool's retry safety, and `GET /v1/tools/:name/schema` returns one tool's schema.
+- The Nexus adapter sends every call under a Nexus tenant, records your scope, run and agent with its usage, keeps a thinking model's native content between turns and passes tool choice through. The Weave adapter searches only the run's tenant plus the read-only shared collections you list. The Shield adapter evaluates the agent's `shield_profile`, and `shieldsafety.NewToolAuthorizer` scans tool calls under your own authorizer, which always answers first.
+- Set `safety.ModeCheckedSegments` with an incremental gate such as the Shield adapter, and model output streams out in pieces Shield has already checked, so you don't wait for the whole answer. `shieldsafety.WithSegmentLimit` caps how much text with no line break is held before it's cut.
+
+### Opt-in
+
+These stay off until you turn them on: run leasing (`WithRunLease`), checked segments, and every experimental assured-execution service v1.9.0 brought (the verification worker and its recovery, required verification, the context budget, result content, public event replay, prepared-team messaging, task scheduling, agent-reply and orchestration recovery, effect continuation and recovery, reflection and operational awareness). An engine built with just a store and a model starts the suspension sweeper and nothing else. `TestDefaultEngineEnablesNoExperimentalServices` and `TestRunLeaseIsOffByDefault` in `engine` fail if that ever changes.
+
+### Breaking changes
+
+Each of these has a paragraph in `MIGRATION.md`, "Upgrading from v1.9.0", saying what to do.
+
+- `engine.New` fails with `cortex.ErrPrincipalCodecRequired` when you install a `ToolAuthorizer` without `engine.WithPrincipalCodec`.
+- A resumed run makes its tool calls as the principal that started it, not the one that resumed it.
+- `ResolveCheckpoint` records the decision before the run continues. When the continuation fails before it claims the run, the run stays `ready` and the error says `ContinueRun` finishes it. An approval that arrives while the engine drains, or that can't get a run slot, is still refused before anything is recorded, as in v1.9.0.
+- `WithSuspensionTTL(0)` no longer stops the sweeper. It only drops the default deadline.
+- The Forge extension applies the config fields it used to read and ignore.
+- The Nexus adapter refuses a call it can't name a tenant for, with an error matching `cortex.ErrNoScope`.
+- Shield screens publication even when you pass `engine.WithSafety`. It evaluates real profiles and fails closed when there's none for the tenant and app, and it takes the tenant from the scope's first level, not the whole canonical scope.
+- The Weave adapter reads under the scope's first level as the Weave tenant and overrides a tenant on the context. Under the wrong tenant it finds nothing, and the agent answers without its knowledge.
+- `tool.New` panics when you pass it `tool.RetrySafe()`, which it used to drop without a word. Use `tool.Register` or `engine.WithToolRetrySafe`.
+- Pauses v1.8.0 wrote at continuation schema 1 can't be resumed. That one is in "Upgrading from v1.8.0".
+
+### Fixed
+
+- REST handlers no longer write JSON bodies twice. On most routes v1.9.0 sent two concatenated objects in one response, which `json.Unmarshal` and browsers reject.
+- A run the messaging bus starts for a directive keeps its reply route, the asker's reply token included, in a column only the engine reads. It's no longer in the run's metadata, so run JSON, events and REST responses don't carry it.
+- Run JSON and REST responses no longer carry the lease owner, which names the worker's host, and a reclaimed run that stops on an uncertain call says "another worker" in its error. The owner stays in the store.
+- The REST module answers `cortex.ErrEngineStopping` with 503, saying nothing was recorded so your client can retry on another replica, and `cortex.ErrRunInterrupted` with 409. Both were 500.
+
+### Known limits
+
+- Task-run principals: with an authorizer installed, a coordination child that resumes gets a nil principal. Authorize those on `Subject.RunID` and the claim's worker principal ref.
+- A turn whose text your publication gate redacted loses its thinking blocks when it's replayed to the model. The run carries on without that turn's reasoning.
+- `Native` holds the model's unredacted content, and it sits that way at rest in progress snapshots and continuations. No REST route returns it.
+- The Shield segment checker's CPU grows with a long held run of text that has no line break. `MaxOutputBytes` bounds it.
+- Runs under required verification or a task claim hold no lease and are never reclaimed. Their recovery stays with the `coordination` package, and so does an experimental task-claim run that reaches `ready`, which coordination treats as a usage conflict.
+- Lease expiry compares the clocks of two machines, so keep your replicas' clocks in sync if you turn leasing on.
+- `mcpbridge` takes a server's `readOnlyHint` as retry safety. Use `Include` to leave out tools whose hint you don't trust.
+
 ## [1.13.0] - Unreleased
 
+This section shipped as v1.7.0.
+
 Agents can address each other now. Not through a blackboard inside an
 orchestration somebody started, which is what v1.11.0 gave you, but
 directly: an agent names a peer, says what kind of thing it is saying,