diff --git a/ADAPTER_PREFLIGHT.md b/ADAPTER_PREFLIGHT.md index 3c9fbf7..63a8bc8 100644 --- a/ADAPTER_PREFLIGHT.md +++ b/ADAPTER_PREFLIGHT.md @@ -12,6 +12,28 @@ never by reading documentation. Record answers in `HANDOFF.md` before the first scenario. An adapter keeps `unverified = True` until every item has an observed answer. +For every live execution after adoption of invariant 11, create this +provenance block before interpreting the result: + +```text +status: verified | reported, unverified +executed_by: +environment: +search_quota_before: +search_quota_after: +results_file: +``` + +All four facts plus the filename are mandatory prospectively. If any is +missing, the run is +`reported, unverified`: log it internally, but do not change README, a +published report, adapter verification status, or another gated claim from it. +The 2026-08-05 Mem0 evidence is the recorded legacy exception: the founder +independently read the live SEARCH counter at 599 against approximately 885 +before the work, verifying the run retrospectively by quota reconciliation. +Its provenance was not self-recorded at execution time — the reason this rule +now exists. + --- ## 1. Quota and rate model — from live response headers @@ -26,8 +48,9 @@ incomplete every time so far. - [ ] What is the request rate limit, and does confirmation polling threaten it? *Why:* Mem0 meters `SEARCH` (1,000/period) separately from `ADD` -(10,000/period). A full run costs ~106 SEARCH — about a tenth of the period — -and confirmation reads are most of it. Zep meters ingestion credits instead, so +(10,000/period). The current full 15 × 2 path costs at least 182 SEARCH — a +91-per-seed structural minimum — and extra convergence polls can raise it. Zep +meters ingestion credits instead, so the same polling is free but a request-rate ceiling applies. Two providers, opposite constraints; neither was guessable. @@ -102,3 +125,6 @@ residue as leakage. project so far has been ours, not the provider's. - **Record what you could not test.** An untested assumption is not a passed one; say so in `HANDOFF.md` and keep `unverified = True`. +- **Bracket every live run with quota observations.** Estimates are planning + inputs. Only the provider-reported before/after pair closes the spend and + qualifies the run for gated publication. diff --git a/CLAUDE.md b/CLAUDE.md index 2bfb502..8624e5f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -86,6 +86,19 @@ Status: v0.1, built in public. `pytest -q` must stay green after every change. just-mutated value (or its absence) with a bounded timeout. That poll is mutation confirmation only; the oracle never grades it. Non-convergence aborts rather than being laundered into PASS or scored as a provider bug. +11. **Live-run provenance is a publication gate.** From adoption of this rule, + every live-provider run + records four facts contemporaneously: who executed it, the execution + environment, the provider's SEARCH quota immediately before and after, + and the results filename. Missing any field makes the run **reported, + unverified**. It may be logged internally, but it must not update README, + `examples/report_*`, clear an adapter's `unverified` flag, or support any + other gated claim. Estimates and reconstructed quota arithmetic are useful + diagnostics, never substitutes for the before/after pair. The 2026-08-05 + Mem0 evidence is a documented legacy exception: it was verified + retrospectively by the founder's independent live counter reading of 599 + against approximately 885 before the work. Its provenance was not + self-recorded at execution time — the reason this prospective rule exists. ## Conventions @@ -156,8 +169,12 @@ stops short has not finished its task and must say which of the four steps it stopped at. Blocked work still gets an entry: the blocker, what was established anyway, and what remains unverified. -**Any diagnostic run executed outside this environment must be promoted into -`HANDOFF.md` by hand, citing its results filename.** `diagnostics/results/` is +**Every live-provider run after adoption of invariant 11, here or elsewhere, +must be promoted into +`HANDOFF.md` with its complete invariant-11 provenance block:** executor, +environment, SEARCH quota before, SEARCH quota after, and results filename. +If one is absent, label it **reported, unverified** and do not change a gated +file from it. `diagnostics/results/` is gitignored by design — those are open investigations whose output is not publishable until ruled on. The cost of that choice is that a run done on a laptop leaves no commit, no artefact and no trace here, so it is invisible to diff --git a/HANDOFF.md b/HANDOFF.md index e34d2f4..5b90b7f 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -3084,6 +3084,153 @@ External launch remains **HELD pending founder review**. --- +## 2026-08-05 — PR #10 merge and retrospective provenance review + +### 1. What changed + +GitHub read-back at this handoff found PR #10 already merged at 01:09 UTC at +head `a5aad4e`. The quota reconciliation and invariant-11 process rule are in +later commits on the follow-up branch. + +### 2. Findings + +The merge was an external state change, not an action taken in this task. Its +published 2026-08-05 evidence is **verified retrospectively by quota +reconciliation; provenance was not self-recorded at execution time — the +reason the rule now exists.** The founder independently read Mem0's live +SEARCH counter at 599 on 2026-08-05, against approximately 885 before the +work. Mem0 owns that counter outside the repository; repository code cannot +write it. No new provider call was made during this documentation review. + +### 3. Decisions + +- Do not rewrite, revert, or otherwise mutate the completed merge without + explicit founder authority. +- Open the isolated post-merge documentation/process correction as a new draft + PR from `agent/fail-closed-release-gate`, labeled `needs-founder-review`. +- Do not merge the follow-up PR automatically. + +### 4. FOR STRATEGY + +The 2026-08-05 evidence on `main` remains verified. The process gap is that +the run did not self-record its provenance contemporaneously; apply invariant +11 prospectively so the next run does not require retrospective reconciliation. + +### 5. Next + +1. Open and label the follow-up draft PR. +2. Confirm it retains the verified 2026 evidence and contains only provenance, + accounting, cost and process-rule corrections. +3. Await explicit founder review; do not merge. + +External launch remains **HELD pending founder review**. + +--- + +## 2026-08-05 — Live-run quota reconciliation and provenance gate + +### 1. What changed + +- Added invariant 11: every live-provider execution must contemporaneously + record `executed_by`, `environment`, SEARCH quota before, SEARCH quota after, + and the exact results filename. +- A run missing any field is **reported, unverified**. It may be retained in + this internal handoff, but it cannot update README, `examples/report_*`, an + adapter verification flag, or another gated claim. This applies to runs + executed after the rule's adoption. +- `diagnostics/readd_after_delete.py` now refuses a live execution without the + executor and environment, writes the initial provenance record before any + mutation, and records the quota pair and result filename in its JSON output. +- Retained the 2026-08-05 evidence in README and `examples/report_mem0.*`. + It is verified retrospectively by the founder's independent live SEARCH + counter reading; its provenance was not self-recorded at execution time. +- Corrected the documented current Mem0 cost from the historical ~106 estimate + to a structural minimum of 91 SEARCH per seed, or at least 182 for 15 × 2, + before extra convergence polls. + +### 2. Quota accounting + +The founder independently took a live Mem0 SEARCH counter reading of **599** +on 2026-08-05, against approximately **885 before the work**: a movement of +approximately 286 SEARCH units. Mem0's counter is external to this repository +and cannot be written by repository code. The surviving artifacts support this +reconciliation: + +| Bucket | SEARCH units | Evidence and limit | +|---|---:|---| +| Seven-arm diagnostic | 41 | 33 in per-arm header deltas, plus 7 before-arm probes and the initial quota probe | +| Full 15 × 2 execution | at least 182 | structural minimum: 30 reset pre-reads, 60 write confirmations, 32 delete reads/confirmations, and 60 scenario queries | +| Unallocated before/around the first recorded checkpoint | at most 63 | arithmetic remainder; no complete run-specific quota pair exists, so this cannot be assigned more precisely | +| **Total** | **286** | 41 + 182 + up to 63 | + +The diagnostic's first recorded provider checkpoint was **822 remaining**. +Its result file reports arm deltas `5, 5, 5, 5, 5, 4, 4`; those 33 units +include the seven after-arm probes, while the seven before-arm probes and the +initial probe bring the diagnostic to 41. + +No surviving artifact evidences a repeat full 15 × 2 run in this work. The +full-run log contains no residue-reset/sentinel sequence because the unique +run namespace started empty; its 30 reset pre-reads are already included in +the 182 minimum. No separate post-checkpoint development run is evidenced. +Any extra convergence poll, standalone quota probe, or earlier development +call must therefore remain inside the unallocated remainder rather than being +invented as a specific cause. + +### 3. Provenance status of today's live executions + +**Seven-arm diagnostic** + +```text +status: verified retrospectively by quota reconciliation; provenance not self-recorded at execution time — the reason the rule now exists +executed_by: Codex (OpenAI), on the founder's instruction +environment: Codex desktop on macOS; local writable clone; explicitly approved live network access +search_quota_before: approximately 885 before the combined work, recorded by the founder +search_quota_after: 599, independently read live from Mem0 by the founder on 2026-08-05 +results_file: diagnostics/results/readd_after_delete_1785890141.json +``` + +**Full 15 × 2 execution** + +```text +status: verified retrospectively by quota reconciliation; provenance not self-recorded at execution time — the reason the rule now exists +executed_by: Codex (OpenAI), on the founder's instruction +environment: Codex desktop on macOS; local writable clone; explicitly approved live network access +search_quota_before: approximately 885 before the combined work, recorded by the founder +search_quota_after: 599, independently read live from Mem0 by the founder on 2026-08-05 +results_file: diagnostics/results/full_mem0_20260805.json (with matching .md and .log) +``` + +The combined live work is verified retrospectively by the independent provider +counter reconciliation. It qualifies as evidence; the missing self-recorded +run-level pair is a process deficiency and the reason invariant 11 now applies +prospectively. + +### 4. FOR STRATEGY + +- The 63-unit maximum remainder is closed as **unallocated**, not + silently attributed to repeat, sentinel, or development runs that the + artifacts do not prove. Do not spend more quota merely to reconstruct it. +- The next live run must begin only after the five-field provenance record is + prepared and must finish with the provider-reported quota pair in the same + results artifact. That single artifact should settle both evidentiary status + and spend without a follow-up reconstruction. +- Existing Mem0 findings are unaffected: the scenario-pack identifiers do not + slug-collide, the execution used `seeds=2`, there was no concurrency, and + the Mem0 answering layer is quoting. The evidence remains verified and the + metric values remain unchanged. +- README/report edits remain gated-tier and PR #11 must retain + `needs-founder-review`; do not merge without explicit founder approval. + +### 5. Next + +1. Run offline validation only; do not spend additional Mem0 quota. +2. Push the corrected provenance rule and accounting to draft PR #11. +3. Confirm the PR stays draft, labeled `needs-founder-review`, and unmerged. + +External launch remains **HELD pending founder review**. + +--- + ## 2026-08-05 — Draft PR #10 opened; founder gate active ### 1. What shipped @@ -3121,3 +3268,114 @@ handoff point. External launch remains **HELD pending founder review**. --- + +## 2026-08-05 — Current publication state after PR #10 merge + +### 1. What shipped + +PR #10 merged at head `a5aad4e`. The later provenance correction is pushed at +`e270191` and is not in `main`. + +### 2. Findings + +No new provider measurement and no additional quota spend. The detailed +286-unit reconciliation and retrospective-verification provenance blocks are +in the entries above. + +### 3. Decisions + +Preserve the completed merge and route the prospective provenance rule and +customer-facing cost correction through a separate draft PR with +`needs-founder-review`; do not merge it automatically. + +### 4. FOR STRATEGY + +Treat the 2026-08-05 live evidence on `main` as **verified retrospectively by +quota reconciliation; provenance not self-recorded at execution time — the +reason the rule now exists.** + +### 5. Next + +Open the isolated follow-up draft, confirm its scope and gate, then await +explicit founder review. + +External launch remains **HELD pending founder review**. + +--- + +## 2026-08-05 — Follow-up PR #11 CI repair + +### 1. What shipped + +Draft PR #11 opened with `needs-founder-review`. Its first CI run failed only +because the new subprocess tests exercised the diagnostic in the base install, +while `httpx` was imported at module load despite being an optional Mem0-path +dependency. + +### 2. Findings + +The provenance checks themselves behaved correctly locally. CI failed before +reaching them with `ModuleNotFoundError: httpx`; no provider call was possible +and no quota was spent. + +### 3. Decisions + +Move the optional `httpx` import into the live quota-probe function. Dry-run +and fail-closed provenance validation must work in the base install, while an +actual Mem0 run continues to use the dependency supplied by the Mem0 extra. + +### 4. FOR STRATEGY + +This is a packaging boundary correction only. It does not alter the quota +reconciliation, provenance ruling, published metrics, or founder gate. + +### 5. Next + +Push the repair, confirm PR #11 CI passes, and leave the PR draft and unmerged. + +External launch remains **HELD pending founder review**. + +--- + +## 2026-08-05 — Founder confirms retrospective verification + +### 1. What changed + +Corrected HANDOFF, README, the published Mem0 report framing and PR #11 to +retain the 2026-08-05 evidence as verified. Removed the proposed evidence +rollback. Invariant 11 remains as a prospective self-recording requirement. + +### 2. Findings + +The evidence is **verified retrospectively by quota reconciliation; provenance +not self-recorded at execution time — the reason the rule now exists.** The +founder independently read Mem0's live SEARCH quota at **599** on 2026-08-05, +against approximately **885 before the work**. That approximately 286-unit +movement is consistent with 41 for the seven-arm diagnostic, at least 182 for +the full 15 × 2, and at most 63 unallocated. Mem0's counter is external to the +repository and cannot be written by repository code. + +### 3. Decisions + +- Keep the regenerated 2026-08-05 report and run ID + `1986edd5512147dca783bc513029b4f3` as the current evidence. +- State the customer-facing full-run cost as **at least 182 SEARCH** under + invariant 10, not the historical ~106 estimate. +- Require future live runs to self-record executor, environment, quota pair + and exact results filename; retrospective verification is not the default + operating procedure. + +### 4. FOR STRATEGY + +This correction changes provenance framing, not provider findings. Existing +Mem0 results remain unaffected: identifiers do not slug-collide, seeds=2, no +concurrency, and a quoting answering layer. + +### 5. Next + +Update draft PR #11, rerun offline validation and CI, retain +`needs-founder-review`, and do not merge without explicit approval. + +External launch remains **HELD pending founder review**. + +--- diff --git a/README.md b/README.md index c0c0f41..69244fa 100644 --- a/README.md +++ b/README.md @@ -193,7 +193,7 @@ How the lifecycle maps onto Mem0: GATE [fail on <= P2]: FAIL (10 blocking findings) ``` -**Provenance.** The current evidence was regenerated from implementation commit `b81bf20`, after the adapter gained invariant-10 mutation confirmation, scope identifiers became lossless, and every invocation gained a unique namespace. All 170 operations completed. The metrics are unchanged from the earlier evidence: the harness corrections removed collision and race exposure without changing the Mem0 result. The separate reset diagnostics, including their intermittent namespace-delete behaviour, are tracked in [`HANDOFF.md`](HANDOFF.md). +**Provenance.** The current evidence was regenerated from implementation commit `b81bf20`, after the adapter gained invariant-10 mutation confirmation, scope identifiers became lossless, and every invocation gained a unique namespace. All 170 operations completed. The run is **verified retrospectively by quota reconciliation; provenance was not self-recorded at execution time — the reason invariant 11 now exists.** The founder independently read Mem0's live SEARCH counter at **599** on 2026-08-05, against approximately **885** before the work. That external counter movement is consistent with 41 SEARCH for the seven-arm diagnostic, at least 182 for the full 15 × 2, and at most 63 unallocated. Mem0's counter is outside this repository and cannot be written by it. The metrics are unchanged from the earlier evidence. Full accounting is in [`HANDOFF.md`](HANDOFF.md). Mem0 holds the boundaries that carry the P1 severities: **no scope leakage in 22 opportunities and no deletion residue in 18** — deletes stopped the value influencing answers, and no user's or tenant's facts crossed into another's, including where two tenants share a `user_id` and where a fact is moved between tenants. Current-fact accuracy is perfect and every result is stable across seeds. @@ -240,13 +240,15 @@ Mem0 meters two independent counters, and this adapter spends the scarcer one. | `SEARCH` | 1,000 per billing period | every query, plus each read used to confirm a write or delete landed | | `ADD` | 10,000 per billing period | one per `write` | -Measured on the bundled 15-scenario pack: **~53 SEARCH units per seed**, so a -default 2-seed run costs **~106** — roughly a tenth of a 1,000-unit period. -Confirmation reads are the bulk of it, and they are not optional: without them -the harness races the provider and manufactures false failures (invariant 10). +The current adapter has a structural minimum of **91 SEARCH units per seed**: +15 reset pre-reads, 30 write confirmations, 16 delete reads/confirmations and +30 scenario queries. A default 2-seed run therefore costs **at least 182 +SEARCH**, with extra convergence polls increasing that total. Confirmation +reads are not optional: without them the harness races the provider and +manufactures false failures (invariant 10). Budget accordingly if you wire this into CI. Running the full pack on every -commit will exhaust a 1,000-unit period in ~9 runs. The intended shape is a +commit will exhaust a 1,000-unit period in ~5 runs. The intended shape is a small smoke subset per commit and the full pack nightly or per release. Watch the `x-quota-remaining` response header; when the counter is exhausted the API returns `429` and the run aborts rather than reporting degraded numbers. diff --git a/diagnostics/readd_after_delete.py b/diagnostics/readd_after_delete.py index 441c15a..0440b39 100644 --- a/diagnostics/readd_after_delete.py +++ b/diagnostics/readd_after_delete.py @@ -19,7 +19,10 @@ `delete_all`. The earlier loss remains valid evidence that it can happen; the clean rerun does not erase it. The full 15 x 2 regeneration immediately after this diagnostic completed all 170 operations under a unique invocation -namespace; see HANDOFF.md. +namespace. It is verified retrospectively by quota reconciliation: the founder +independently read Mem0's live SEARCH counter at 599 on 2026-08-05, against +approximately 885 before the work. Provenance was not self-recorded at +execution time — the reason invariant 11 now exists. See HANDOFF.md. **The combined result: (f) failed once and passed once; (g) passed twice.** Every per-key arm passed. A write immediately after `delete_all` can be lost, @@ -111,9 +114,11 @@ **not** license "012 refuted" — it licenses only "not reproduced same-scope, with a per-key delete". - python diagnostics/readd_after_delete.py # prints cost, then asks - python diagnostics/readd_after_delete.py --yes # no prompt python diagnostics/readd_after_delete.py --dry-run # cost only, spends nothing + python diagnostics/readd_after_delete.py \ + --executed-by "" --environment "" + python diagnostics/readd_after_delete.py --yes \ + --executed-by "" --environment "" The key is read from MEM0_API_KEY, falling back to ~/.mem0/config.json, so no export is needed. Results are written to diagnostics/results/ as well as @@ -136,8 +141,6 @@ import time from pathlib import Path -import httpx - POLL_INTERVAL = 5.0 # seconds between reads; each read costs one SEARCH unit CONFIRM_TIMEOUT = 120.0 # ceiling on waiting for a value to become retrievable EXTRA_SETTLE = 60.0 # arm (c): wait this long AFTER the delete reads empty @@ -189,6 +192,10 @@ def resolve_api_key() -> str | None: def quota_remaining(api_key: str) -> int | None: """Read the SEARCH counter. Costs one SEARCH unit itself.""" try: + # Keep the optional live-provider dependency off the dry-run and + # missing-provenance paths. Core CI installs only the base package. + import httpx + r = httpx.post( "https://api.mem0.ai/v3/memories/", headers={"Authorization": f"Token {api_key}", "Content-Type": "application/json"}, @@ -376,10 +383,18 @@ def main() -> int: ap.add_argument("--yes", action="store_true", help="skip the cost prompt") ap.add_argument("--dry-run", action="store_true", help="print the cost estimate and exit without spending") + ap.add_argument( + "--executed-by", default=os.environ.get("MEMORYCHECK_EXECUTED_BY"), + help="person or agent identity executing the live run", + ) + ap.add_argument( + "--environment", default=os.environ.get("MEMORYCHECK_RUN_ENVIRONMENT"), + help="machine/runtime and network mode used for the live run", + ) args = ap.parse_args() - # Four quota probes (one up front, one either side of each arm) are - # themselves SEARCH calls; count them so the estimate is not optimistic. + # Quota probes (one up front, one either side of each arm) are themselves + # SEARCH calls; count them so the estimate is not optimistic. probes = 1 + 2 * len(COST_ESTIMATE) total = sum(COST_ESTIMATE.values()) + probes print("Estimated SEARCH cost (worst case, the scarce counter):") @@ -392,6 +407,14 @@ def main() -> int: print("dry run: nothing spent") return 0 + if not args.executed_by or not args.environment: + print( + "Live-run provenance required: pass --executed-by and --environment " + "(or set MEMORYCHECK_EXECUTED_BY and MEMORYCHECK_RUN_ENVIRONMENT).", + file=sys.stderr, + ) + return 2 + api_key = resolve_api_key() if not api_key: print("No Mem0 key: set MEM0_API_KEY or populate ~/.mem0/config.json.", @@ -403,16 +426,44 @@ def main() -> int: print('Mem0 SDK missing. pip install "mem0ai>=2.0.14"', file=sys.stderr) return 2 - remaining = quota_remaining(api_key) - if remaining is not None and remaining >= 0: - print(f"SEARCH quota remaining now: {remaining}") - if remaining < total: - print(f"REFUSING TO RUN: need ~{total}, have {remaining}. A run that " + stamp = int(time.time()) + out_dir = Path(__file__).resolve().parent / "results" + out_dir.mkdir(exist_ok=True) + out = out_dir / f"readd_after_delete_{stamp}.json" + + quota_before = quota_remaining(api_key) + initial_report = { + "run_at_epoch": stamp, + "provenance": { + "status": "reported, unverified", + "executed_by": args.executed_by, + "environment": args.environment, + "search_quota_before": quota_before, + "search_quota_after": None, + "results_file": str(out), + }, + "state": "started", + "results": [], + } + # Persist the provenance before mutations. If the process is interrupted, + # the partial record remains explicitly unverified instead of disappearing. + out.write_text(json.dumps(initial_report, indent=2) + "\n") + + if quota_before is not None and quota_before >= 0: + print(f"SEARCH quota remaining now: {quota_before}") + if quota_before < total: + initial_report["state"] = "refused: insufficient SEARCH quota" + out.write_text(json.dumps(initial_report, indent=2) + "\n") + print(f"REFUSING TO RUN: need ~{total}, have {quota_before}. A run that " "dies partway proves nothing and spends the rest.", file=sys.stderr) + print(f"provenance written to {out}") return 3 if not args.yes: if input(f"Spend ~{total} SEARCH units? [y/N] ").strip().lower() != "y": print("aborted; nothing spent beyond the quota probe") + initial_report["state"] = "aborted by operator after quota probe" + out.write_text(json.dumps(initial_report, indent=2) + "\n") + print(f"provenance written to {out}") return 0 # Pass the resolved key explicitly. MemoryClient() reads MEM0_API_KEY from @@ -421,7 +472,6 @@ def main() -> int: # probe and then die here with "Mem0 API Key not provided" — after the # operator had already confirmed the spend. client = MemoryClient(api_key=api_key) - stamp = int(time.time()) # (name, first value, second value, extra settle, cross-scope) plan = [ ("a_identical", f"wintergreen-{stamp}", f"wintergreen-{stamp}", 0.0, False), @@ -440,6 +490,7 @@ def main() -> int: ] results = [] + quota_after = None for name, first, second, settle, cross in plan: print(f"\n--- arm {name}{' (cross-scope)' if cross else ''} ---") print(f" expected SEARCH cost, worst case: ~{COST_ESTIMATE[name]} units") @@ -448,6 +499,7 @@ def main() -> int: target = f"mc_diag__{name}_{stamp}__scope_b" if cross else None result = Arm(client, name, source, target).run(first, second, settle) after = quota_remaining(api_key) + quota_after = after if before is not None and after is not None and before >= 0 and after >= 0: result["search_units_spent"] = before - after results.append(result) @@ -464,6 +516,7 @@ def main() -> int: [f"residue-{i}-{stamp}" for i in range(_RESIDUE_N)], f"probe-{name}-{stamp}", settle) after = quota_remaining(api_key) + quota_after = after if before is not None and after is not None and before >= 0 and after >= 0: result["search_units_spent"] = before - after results.append(result) @@ -557,18 +610,31 @@ def main() -> int: print("\nNo conclusion may be published from a single execution of this " "script. Re-run to establish stability first.") - out_dir = Path(__file__).resolve().parent / "results" - out_dir.mkdir(exist_ok=True) - out = out_dir / f"readd_after_delete_{stamp}.json" + provenance_verified = all( + isinstance(value, int) and value >= 0 + for value in (quota_before, quota_after) + ) + provenance = { + "status": "verified" if provenance_verified else "reported, unverified", + "executed_by": args.executed_by, + "environment": args.environment, + "search_quota_before": quota_before, + "search_quota_after": quota_after, + "results_file": str(out), + } out.write_text(json.dumps({ "run_at_epoch": stamp, + "provenance": provenance, + "state": "completed", "poll_interval_s": POLL_INTERVAL, "confirm_timeout_s": CONFIRM_TIMEOUT, "extra_settle_s": EXTRA_SETTLE, "estimated_search_units": total, "results": results, "caveat": "single execution; not a finding until reproduced", - }, indent=2)) + }, indent=2) + "\n") + print(f"provenance status: {provenance['status']}") + print(f"SEARCH quota before/after: {quota_before} -> {quota_after}") print(f"results written to {out}") return 0 diff --git a/examples/report_mem0.json b/examples/report_mem0.json index 892e8dd..53f4697 100644 --- a/examples/report_mem0.json +++ b/examples/report_mem0.json @@ -187,4 +187,4 @@ "seed": 1 } ] -} \ No newline at end of file +} diff --git a/examples/report_mem0.md b/examples/report_mem0.md index 15e94b3..681eb1b 100644 --- a/examples/report_mem0.md +++ b/examples/report_mem0.md @@ -12,13 +12,15 @@ SDK `mem0ai` **2.0.14**, pinned for a like-for-like reproduction, executed build identifier to clients, so the SDK pin and run date are the available provenance. -**Provenance.** This is a completed 15 × 2 regeneration after write/delete +**Provenance.** This completed 15 × 2 regeneration followed write/delete convergence confirmation, lossless scope identifiers and unique invocation -namespaces were added. All 170 operations completed. Its figures are identical -to the 2026-07-27 published run and the intervening reproductions: current-fact -accuracy 46/46, stale reuse 10/10, scope leakage 0/22 and deletion residue -0/18. See `HANDOFF.md` for the separate reset diagnostic history; this fresh -run namespace did not need to delete residue from an earlier invocation. +namespaces. All 170 operations completed. It is **verified retrospectively by +quota reconciliation; provenance was not self-recorded at execution time — the +reason invariant 11 now exists.** The founder independently read Mem0's live +SEARCH counter at **599** on 2026-08-05, against approximately **885** before +the work. The 286-unit movement is consistent with 41 for the diagnostic, at +least 182 for this full run, and at most 63 unallocated. Mem0's counter is +outside this repository and cannot be written by it. See `HANDOFF.md`. **What the stale_reuse result is not.** It is not a report that Mem0 fails to overwrite corrected facts. That is documented, intended design — Mem0's README, diff --git a/tests/test_diagnostic_provenance.py b/tests/test_diagnostic_provenance.py new file mode 100644 index 0000000..7b87936 --- /dev/null +++ b/tests/test_diagnostic_provenance.py @@ -0,0 +1,43 @@ +from __future__ import annotations + +import os +import subprocess +import sys +from pathlib import Path + + +SCRIPT = Path(__file__).resolve().parents[1] / "diagnostics" / "readd_after_delete.py" + + +def _environment_without_provenance() -> dict[str, str]: + env = os.environ.copy() + env.pop("MEMORYCHECK_EXECUTED_BY", None) + env.pop("MEMORYCHECK_RUN_ENVIRONMENT", None) + return env + + +def test_live_diagnostic_refuses_missing_provenance_before_api_access(): + completed = subprocess.run( + [sys.executable, str(SCRIPT), "--yes"], + capture_output=True, + check=False, + env=_environment_without_provenance(), + text=True, + ) + + assert completed.returncode == 2 + assert "Live-run provenance required" in completed.stderr + assert "No Mem0 key" not in completed.stderr + + +def test_dry_run_does_not_require_live_provenance(): + completed = subprocess.run( + [sys.executable, str(SCRIPT), "--dry-run"], + capture_output=True, + check=False, + env=_environment_without_provenance(), + text=True, + ) + + assert completed.returncode == 0 + assert "dry run: nothing spent" in completed.stdout