Waterflane/contextforge api integration fixes

CHANGELOG.md

@@ -8,6 +8,41 @@ and Python distribution versions follow PEP 440.
 
 ## [Unreleased]
 
+### Added
+
+- Added independent `--fail-fast` and `--max-failures N` index policies, a
+  bounded semantic scheduler, and a job-scoped provider circuit breaker.
+- Added `--progress jsonl` for clean, flushed `ProgressEvent` schema 3 streams
+  from `index build` and `index update`.
+- Added opt-in Bridge 2.0 tracked `build`/`update` jobs, correlated
+  `$/progress` notifications, cooperative cancellation, schema capabilities,
+  and a normative Bridge 2 JSON Schema. Bridge 1.0/1.1 remain supported.
+
+### Changed
+
+- Classified authentication, authorization, missing credential, quota, rate
+  limit, model, configuration, timeout, and service failures so terminal
+  provider-wide failures are not retried per file.
+- Made semantic and repository-map analyzer identity depend on provider/model
+  and analysis contracts rather than an OpenAI-compatible endpoint. Legacy
+  `+base.<sha256>` records migrate on update without model calls.
+
+### Fixed
+
+- Enforce the caller's expected snapshot inside the index workflow, revalidate
+  it immediately before atomic publication, and check cancellation at the
+  manifest activation boundary.
+- Return Bridge index timeouts at the caller's deadline while retaining
+  cooperative cleanup and its writer lock as tracked background work.
+- Bound Bridge progress delivery with one writer task, coalesce synchronous
+  cumulative progress bursts, detect sustained client backpressure, drain every
+  completed semantic task, and preserve typed provider causes through semantic
+  and map aggregation.
+- Preserve the prior active generation on cancellation, failure limits,
+  provider circuit opening, source drift, or progress backpressure.
+- Return safe typed Bridge index errors without provider bodies, credentialed
+  URLs, absolute paths, tracebacks, or exception representations.
+
 ## [0.5.1] - 2026-09-05
 
 ### Added

README.md

@@ -28,9 +28,9 @@ commands.
 </p>
 
 > [!IMPORTANT]
-> ContextForge is pre-alpha software. Version `0.5.1` adds verified polyglot
-> declarations, resumable semantic coverage, bridge protocol 1.1, and hardened
-> exact-symbol discovery. Discovery benchmarking is experimental and
+> ContextForge is pre-alpha software. The unreleased API adds bounded failure
+> policies, JSONL progress, and opt-in Bridge 2 tracked index jobs while the
+> package version remains `0.5.1`. Discovery benchmarking is experimental and
 > its results should be reviewed alongside the recorded provider, model,
 > configuration, and source snapshot.
 
@@ -49,7 +49,8 @@ commands.
 - **Explicit trust boundaries.** ContextForge does not edit repository source,
   execute repository code, expose shell tools, or mutate Git state.
 - **Automation-friendly output.** Structured results stay on stdout while
-  progress and diagnostics stay on stderr.
+  normal progress and diagnostics stay on stderr; index jobs can opt into a
+  pure schema-3 JSONL progress stream on stdout.
 
 ## Representative workflow
 
@@ -185,17 +186,28 @@ mutating operations.
 | `contextforge diagnostics config [PATH]` | Explain effective configuration | Read-only |
 | `contextforge diagnostics provider [PATH]` | Show provider policy without probing it | Read-only |
 | `contextforge mcp serve [PATH]` | Run the local read-only stdio MCP server | Read-only session |
-| `contextforge bridge --stdio --workspace PATH` | Run persistent JSON-RPC bridge v1 | Verified read-only workspace session |
+| `contextforge bridge --stdio --workspace PATH` | Run negotiated JSON-RPC Bridge 1 or 2 | V1 read-only; V2 may atomically mutate only the index |
 | `contextforge benchmark discovery PATH` | Run manifest-driven discovery benchmarks | Repository/index read-only; experimental |
 
 Global diagnostic options are `--log-level`, `--log-format`, `--log-file`,
 repeatable `--log-component`, `--no-log-file`, `--no-color`, and `-v`/`-vv`.
 Detailed syntax, defaults, streams, side effects, mistakes, and examples are in
 the [Wiki CLI reference](https://github.com/waterflane/ContextForge/wiki/CLI-Overview).
 The local integration contract is documented in the
-[bridge v1 guide](docs/guides/bridge.md), with a runnable
+[bridge integration guide](docs/guides/bridge.md), with a runnable
 [generic client](examples/generic_bridge_client.py).
 
+Long model-backed index jobs can stop issuing new work with `--fail-fast` or
+`--max-failures N`. Existing `--fail-on-error` semantics are unchanged: without
+one of those limits ContextForge finishes the workload and declines publication
+if any semantic unit failed. Hosts that launch the CLI can consume full
+`ProgressEvent` schema 3 objects with `--progress jsonl`:
+
+```bash
+contextforge index update . --provider openai-compatible \
+  --model exact/model-id --progress jsonl --max-failures 3
+```
+
 ## Configuration
 
 Project configuration is closed, versioned TOML. Resolution order is:

ROADMAP.md

@@ -81,6 +81,22 @@ This roadmap describes broad milestones only. It does not promise dates.
 - [x] Adapt structured model requests to truncation and reported runtime context
   limits without weakening strict validation.
 
+### Unreleased host-integration hardening (complete)
+
+- [x] Add independent fail-fast and bounded-failure index policies while
+  preserving the existing `--fail-on-error` contract.
+- [x] Classify provider failures, stop retrying terminal authentication, quota,
+  model, and configuration errors, and add a job-scoped circuit breaker.
+- [x] Expose clean JSONL index progress and bounded Bridge progress
+  notifications that coalesce synchronous bursts while detecting sustained
+  client backpressure.
+- [x] Add opt-in Bridge 2 tracked build/update jobs with cooperative
+  cancellation, caller-deadline responses, background lock-safe cleanup,
+  snapshot preconditions, and atomic publication guards.
+- [x] Publish readable/current index-family schema capabilities, stabilize
+  analyzer identity across endpoint changes, and return safe typed Bridge
+  integration errors.
+
 ## Later
 
 - Full multi-root workspaces.

docs/architecture/model-providers.md

@@ -271,16 +271,29 @@ identity. Arbitrary parsed JSON is never accepted. No model name is supplied by
 default.
 
 The base URL is configurable with `[models].base_url` or CLI `--base-url`.
-Changing it changes the credential-free SHA-256 suffix on semantic and
-repository-map analyzer identity versions, invalidating model-dependent records
-without changing the persisted provider/model schema.
+It is transport configuration, not analyzer identity: changing a temporary
+loopback port does not invalidate model-dependent records. Analyzer identity is
+derived from analyzer, prompt and response-schema versions plus provider/model.
+Legacy versions ending in `+base.<sha256>` compare as their neutral identity
+and are republished without that suffix during the next update, without a model
+call.
 An optional bearer token is loaded only through the configured
 `credential_env` name. Authentication failures, safe structured error bodies,
 missing model IDs, malformed envelopes, structured-output rejection,
 unavailability, timeout, and cancellation are translated to the shared typed
 provider errors. The adapter uses the same bounded retry runtime as Ollama and
 accepts an injectable async HTTP transport for offline tests.
 
+The shared runtime distinguishes terminal provider-wide failures from transient
+ones. Authentication, authorization, missing credentials, quota/billing
+exhaustion, missing models, and invalid configuration open the job-scoped
+circuit after the first final result and are not retried for each file. Rate
+limits, timeouts, and service unavailability retain bounded request retries;
+three consecutive exhausted failures with the same safe code and
+provider/model identity open the circuit. A success resets that sequence.
+OpenAI-compatible HTTP 429 responses use bounded structured `error.code` and
+message fields to distinguish quota exhaustion from transient rate limiting.
+
 ## Troubleshooting local structured providers
 
 `request exceeds the available context size` means the configured window is

docs/architecture/progress-reporting.md

@@ -159,6 +159,14 @@ stderr receives coalesced, non-ANSI records only for meaningful phase,
 percentage, item, counter, or terminal changes. `never` suppresses rendering;
 `always` never forces terminal controls onto an unsafe redirected stream.
 
+For `contextforge index build|update`, `--progress jsonl` changes stdout into a
+pure UTF-8 JSONL stream. Every line is the complete `ProgressEvent` schema 3
+object and is flushed immediately; Rich rendering and the human build summary
+are suppressed. Diagnostics remain on stderr. The final event carries the
+generation ID, source snapshot digest, index schema, and partial flag. When a
+failure limit or cancellation stops semantic scheduling, terminal event
+metadata also reports cancelled and not-yet-started units.
+
 Direct stderr and existing stderr logging handlers are routed through the same
 live console while it is active, then restored on the single stop path. This
 prints diagnostics above the panel instead of leaving a duplicate frame. All

docs/decisions/003-host-managed-index-jobs.md

@@ -0,0 +1,80 @@
+# ADR-003: Host-managed index jobs and stable provider identity
+
+## Status
+
+Accepted for the unreleased ContextForge API following 0.5.1.
+
+## Context
+
+Local hosts need to build and update ContextForge indexes without duplicating
+writer-lock, staging, publication, progress, timeout, and cancellation logic.
+The CLI previously exposed only human progress and `--fail-on-error`, which
+finishes all semantic work before declining publication. Repeated provider-wide
+failures could therefore spend one retry sequence per file. Bridge 1 is
+intentionally model-free and read-only, so it cannot own this lifecycle.
+
+OpenAI-compatible endpoint URLs were also encoded into analyzer versions. A
+temporary loopback port could invalidate every model-backed record even when
+the provider, model, prompts, and response contracts were unchanged.
+
+## Decision
+
+The application index workflow accepts cooperative cancellation and independent
+failure limits. `fail_fast` means one failed semantic unit; `max_failures`
+defines another positive threshold. They are mutually exclusive. Reaching a
+threshold stops issuing work, cancels in-flight provider waits, aborts the
+publication transaction, and preserves the prior active generation.
+`fail_on_error` without a limit retains its original finish-then-refuse behavior.
+
+A job-scoped circuit breaker sits in the shared provider runtime. Typed
+authentication, authorization, missing credential, quota, model, and
+configuration failures open it immediately. Three consecutive exhausted
+transient failures with the same safe error code and provider/model identity
+also open it; success resets the sequence. The key never includes response
+text, endpoints, or secrets.
+
+CLI index commands expose `--progress jsonl` as a pure flushed stream of full
+`ProgressEvent` schema 3 objects. Human summary output is suppressed and
+diagnostics remain on stderr.
+
+Bridge protocol 2.0 adds a closed `index` method for `build` and `update`. It
+requires `expected_snapshot_digest`, owns the complete application workflow,
+and sends correlated `$/progress` notifications. Cancellation, caller timeout,
+EOF, and shutdown feed the same cooperative cancellation event. Bridge 1.0 and
+1.1 remain read-only and model-free. Bridge capabilities explicitly publish
+current and readable index, manifest, record, progress, and context-package
+schema versions.
+
+The application validates the expected digest against its own build snapshot
+and checks cancellation inside the publication transaction immediately before
+manifest activation. Bridge timeout sets cooperative cancellation before it
+returns the deadline error, and the bridge tracks background cleanup so a
+structural worker retains its writer lock until it has stopped. Progress
+notifications use one writer task and a bounded queue. Cumulative snapshots from
+synchronous producer bursts are coalesced; sustained backpressure cancels rather
+than accumulating unbounded tasks.
+
+Analyzer identity includes analyzer, prompt, response-schema, provider, and
+model identity, but excludes transport endpoint. Legacy analyzer versions with
+a terminal `+base.<sha256>` suffix compare as the neutral identity. The next
+update republishes semantic and repository-map records with the neutral
+identity without new model calls.
+
+Known Bridge index failures use safe typed JSON-RPC categories and bounded
+structured data. Raw provider bodies, credentialed URLs, secrets, absolute
+paths, tracebacks, and exception representations are excluded. `-32603` is
+reserved for unexpected defects.
+
+## Consequences
+
+- Hosts can track and cancel one atomic index job without managing internal
+  storage state.
+- A failed provider cannot trigger unbounded repository-wide repeated calls.
+- CLI subprocess integrations receive stable machine progress without parsing
+  Rich output.
+- Endpoint changes no longer create false staleness, while provider/model or
+  analysis-contract changes still invalidate records.
+- Bridge 2 has narrowly scoped index and provider authority; it still cannot
+  write source, mutate Git, or execute arbitrary commands.
+- Persisted index, manifest, and record schemas remain version 2; progress
+  remains version 3 and the package version remains 0.5.1 until release work.