fix(mem0): replace falsified abort trigger; add namespace arms (f)/(g)

HANDOFF.md

@@ -2085,9 +2085,10 @@ whatever comes back.
 
 ### 2. Findings
 
-**Both runs are blocked. Egress is still refused.** A Mem0 credential is now
-present in the session (`~/.mem0/config.json`), but `api.mem0.ai` remains
-blocked:
+**Both runs are blocked. Egress is still refused.** ~~A Mem0 credential is now
+present in the session (`~/.mem0/config.json`)~~ — **CORRECTED 2026-08-03: this
+was wrong, there is no credential here; see the correction entry below** — and
+`api.mem0.ai` remains blocked:
 
 ```
 curl https://api.mem0.ai/v1/ping/  ->  CONNECT tunnel failed, response 403
@@ -2166,7 +2167,9 @@ uninterpretable against the (a) baseline. Say if you want it the other way.
 ### 4. FOR STRATEGY
 
 - **An environment with egress is now the single blocker for all Mem0 work.**
-  The credential is in place; the network is not. Everything else is staged
+  ~~The credential is in place; the network is not.~~ **CORRECTED
+  2026-08-03: neither is in place — see the correction entry below.**
+  Everything else is staged
   and push-button.
 - Unchanged: whether to pin `mem0ai` 2.0.14 for the regen, whether the extras
   should be exact pins, whether an open and contested #6017 changes the
@@ -2430,3 +2433,133 @@ establish.
 External launch remains **HELD**.
 
 ---
+## 2026-08-03 — Falsified abort message replaced; arms (f)/(g); credential claim corrected
+
+### 1. What shipped
+
+Three things, all internal tier.
+
+| Change | File |
+|---|---|
+| Abort message: falsified trigger replaced with the evidenced one | `adapters/mem0.py` |
+| Arms (f)/(g): the `delete_all` namespace condition | `diagnostics/readd_after_delete.py` |
+| Credential claim corrected in two prior entries | `HANDOFF.md` |
+
+### 2. Findings
+
+**The abort message was asserting a falsified mechanism.** `write()` said:
+
+> Known trigger: re-adding text identical to a value deleted moments earlier
+> (see HANDOFF, scenario 012)
+
+Three things now contradict that. The **2026-08-03 abort fired at
+`003-scope-boundaries`, op #26** — reported, not observed here — with no
+re-add of deleted text anywhere near it. `003` does not delete at all. And
+arms (a)-(e) were built to test exactly that mechanism. Replaced with the
+evidenced trigger: **a write issued after a `delete_all`, even when the
+namespace was polled until it read empty**, citing `reset()`'s own measurement
+of **6/14 writes lost following a delete_all versus 0/10 with no preceding
+delete**. That measurement was in the codebase the whole time, one method
+above the message that ignored it.
+
+This mattered beyond tidiness: the message is what an operator reads at the
+moment a run dies, and it was pointing them at a mechanism five arms had
+already excluded.
+
+**Arms (f)/(g) — and why (a)-(e) were the wrong instrument.** Every arm so far
+uses a **per-key** delete (fetch scope, filter on metadata key, delete by id).
+`reset()` does something categorically different: one
+`delete_all(app_id=...)` wiping the whole namespace across every scope, then a
+poll until empty, then return — after which the runner immediately writes.
+
+That per-key substitution was **my call**, made when (d)/(e) were added: the
+brief said `delete_all` and I used per-key to keep the delete mechanism fixed
+across arms for comparability. Comparability was preserved and the mechanism
+under investigation was not tested. It is a plausible reason all five passed.
+
+- **(f)** seeds `_RESIDUE_N`=5 values so the namespace holds real residue,
+  calls the same `delete_all(app_id=...)`, polls until the namespace reads
+  empty, then writes and polls for retrievability.
+- **(g)** identical, plus a 60s settle after the namespace reads empty.
+
+Together they separate *"reads empty means propagated"* from *"reads empty but
+still reaping"*.
+
+**A design flaw in (f)/(g), caught by simulation before any spend.** The first
+draft opened with a cleanup `delete_all` before seeding. Under the very
+hypothesis being tested, that opening call would swallow the seed writes and
+the arm would die in setup having measured nothing — the simulation returned
+`SETUP_FAILED` for both reaping behaviours. Fixed by giving each namespace arm
+a **fresh `app_id`**, so the namespace is empty by construction and the arm
+contains exactly one `delete_all`: the one under test.
+
+Validated offline against three known `delete_all` behaviours, spending
+nothing:
+
+| delete_all behaviour | (f) no settle | (g) 60s settle |
+|---|---|---|
+| clean | WRITE_VISIBLE | WRITE_VISIBLE |
+| reaps, transient | **WRITE_LOST** | WRITE_VISIBLE |
+| reaps, persistent | **WRITE_LOST** | **WRITE_LOST** |
+
+In every simulated case the namespace **read empty before the write**, which
+is the whole point: emptiness is what `reset()` currently trusts.
+
+**Reading stated in advance.** If (f) fails and (g) passes, polling a
+namespace until empty is not sufficient — and since that is precisely what
+`reset()` does before returning, **every run is exposed**, with the first
+writes after any reset liable to be reaped. That would be a harness-side
+defect as much as a provider behaviour, and the fix is a settle sized from the
+measured empty-to-safe gap, not a fixed sleep.
+
+**Cost.** All seven arms now ~228 units worst case; (f)+(g) are ~66 of that,
+~10-12 if they pass. Worst case remains the failing case.
+
+**Credential claim corrected.** Two prior entries state a Mem0 credential is
+present in this environment. **It is not.** `~/.mem0/config.json` is 57 bytes
+containing a single `user_id` field — the Mem0 CLI's local identity file, not
+an API key. There is no `platform.api_key`, so the script's `resolve_api_key()`
+returns `None` from it, and `MEM0_API_KEY` is unset. I inferred the credential
+from the file existing and never opened it. Both entries are struck through in
+place and point here.
+
+So **both** blockers are live, not one: no key, and no egress
+(`connect_rejected api.mem0.ai:443`, re-verified). Either alone is sufficient
+to stop every Mem0 run from this environment.
+
+### 3. Decisions
+
+- **Per-key delete is not substituted in (f)/(g).** Explicit instruction, and
+  correct: the substitution is what made the earlier arms miss.
+- **Fresh `app_id` per namespace arm**, so the arm holds exactly one
+  `delete_all`. Adopted after the simulation showed the alternative aborts in
+  setup under the hypothesis being tested.
+- **The struck-through claims were left visible** rather than deleted. The
+  entries are merged and public; silently rewriting them would hide that the
+  log was wrong for several hours.
+
+### 4. FOR STRATEGY
+
+- **The `003` abort is a stronger data point than anything the arms have
+  produced, and it is second-hand here.** Worth promoting properly: the
+  results filename, the full latency curve from the new instrumentation, and
+  whether it was the regen or a shorter run. Per the promotion rule, an
+  unlogged run is invisible to the next session — right now this one is a
+  single sentence.
+- If (f)/(g) implicate `reset()`, the fix lands in `adapters/mem0.py` and
+  **could flip provider findings**, so it is gated tier and needs a ruling
+  before merge.
+- Unchanged: pinning `mem0ai` 2.0.14 for the regen, exact pins for extras,
+  whether #6017 changes the publication plan, and the `get_all` pagination
+  limitation.
+
+### 5. Next
+
+1. Run (f)/(g) on the Mac — the leading hypothesis, ~66 units worst case.
+2. Promote the `003` abort properly, with its curve and results filename.
+3. Full 15 × 2 regen, capturing stderr for the latency curve.
+4. Founder ruling before any of it touches published text.
+
+External launch remains **HELD**.
+
+---

diagnostics/readd_after_delete.py

@@ -61,10 +61,34 @@
   (e) as (d), but WAIT an additional 60s before writing to B
       Separates propagation timing from the cross-scope factor itself.
 
-Read the arms together and across the pair boundary. (a) alone says nothing;
-the informative comparisons are (a) vs (b), (a) vs (c), (a) vs (d), and
-(d) vs (e). In particular a clean sweep of (a)-(c) does **not** license
-"012 refuted" — it licenses only "not reproduced same-scope".
+NAMESPACE ARMS (f)-(g). **The leading hypothesis, and the only arms that use
+the delete the harness actually performs.**
+
+Arms (a)-(e) all use a PER-KEY delete: fetch the scope, filter on the metadata
+key, delete by id. `reset()` does something different — a single
+`delete_all(app_id=...)` that wipes the whole namespace across every scope.
+The 2026-08-03 abort at `003-scope-boundaries` (op #26) followed a `reset()`,
+with no re-add of deleted text anywhere near it, which per-key arms cannot
+model. Substituting per-key for delete_all is very likely why (a)-(e) passed.
+
+  (f) write N values so the namespace holds real residue, call the SAME
+      `delete_all(app_id=...)` that `reset()` calls, poll until the namespace
+      reads empty, then write and poll for retrievability
+      The evidenced condition. `reset()`'s own docstring records 6/14 writes
+      lost after a delete_all versus 0/10 without one.
+
+  (g) as (f), but WAIT an additional 60s after the namespace reads empty
+      Separates "reads empty means propagated" from "reads empty but the
+      delete is still reaping". If (f) fails and (g) passes, then polling a
+      namespace until empty — which is exactly what `reset()` does before
+      returning — is NOT sufficient, and every run is exposed.
+
+Read the arms together and across the pair boundaries. (a) alone says nothing;
+the informative comparisons are (a) vs (b), (a) vs (c), (a) vs (d), (d) vs (e),
+and — now the important one — (a)-(e) vs (f)/(g), which is per-key delete
+versus namespace delete_all. In particular a clean sweep of (a)-(c) does
+**not** license "012 refuted" — it licenses only "not reproduced same-scope,
+with a per-key delete".
 
     python diagnostics/readd_after_delete.py             # prints cost, then asks
     python diagnostics/readd_after_delete.py --yes       # no prompt
@@ -100,6 +124,11 @@
 APP_ID = "mc_diag_readd"
 KEY = "handover-note"
 
+# How many values (f)/(g) seed before wiping, so the namespace holds real
+# residue rather than a single row. A delete_all over one memory is not the
+# condition a run hits — `reset()` fires when a namespace has accumulated.
+_RESIDUE_N = 5
+
 # Worst-case SEARCH reads per arm: 1 pre-read + write-confirm polls
 # + 1 delete lookup + delete-confirm polls + re-add-confirm polls.
 #
@@ -115,6 +144,9 @@
     "c_settle_then_identical": 2 + 2 + 1 + _MAX_POLLS,
     "d_cross_scope_identical": 2 + 2 + 1 + _MAX_POLLS,
     "e_cross_scope_settle": 2 + 2 + 1 + _MAX_POLLS,
+    # (f)/(g) seed N values, so add N confirm reads to the usual shape.
+    "f_namespace_delete_all": 2 + _RESIDUE_N + 2 + _MAX_POLLS,
+    "g_namespace_settle": 2 + _RESIDUE_N + 2 + _MAX_POLLS,
 }
 
 
@@ -149,8 +181,14 @@ def quota_remaining(api_key: str) -> int | None:
 
 class Arm:
     def __init__(self, client, name: str, user_id: str,
-                 target_user_id: str | None = None):
+                 target_user_id: str | None = None, app_id: str = APP_ID):
         self.c, self.name, self.uid = client, name, user_id
+        # Per-arm namespace for (f)/(g): a fresh `app_id` starts empty by
+        # construction, so the arm contains exactly ONE delete_all — the one
+        # under test. An opening cleanup delete_all would, under the very
+        # hypothesis being tested, swallow the seed writes and abort the arm
+        # in setup before it measured anything.
+        self.app_id = app_id
         # Where the re-add goes. Same scope for (a)-(c); a different `user_id`
         # under the same `app_id` for the cross-scope arms (d)-(e), which is
         # what the 012 rescope actually does.
@@ -181,7 +219,7 @@ def _wait_until(self, predicate, timeout: float) -> tuple[bool, float]:
             time.sleep(POLL_INTERVAL)
 
     def _add(self, value: str, uid: str | None = None) -> None:
-        self.c.add(f"{KEY}: {value}", user_id=uid or self.uid, app_id=APP_ID,
+        self.c.add(f"{KEY}: {value}", user_id=uid or self.uid, app_id=self.app_id,
                    metadata={"key": KEY}, infer=False)
 
     def _delete_key(self) -> int:
@@ -243,12 +281,74 @@ def run(self, first: str, second: str, extra_settle: float) -> dict:
                        if landed else
                        f"re-added value NOT retrievable {where} within "
                        f"{CONFIRM_TIMEOUT:.0f}s").replace("  ", " "),
+            "delete_mechanism": "per-key delete by id",
             "cross_scope": self.cross_scope,
             "delete_observed_empty_after_s": round(empty_after),
             "extra_settle_s": extra_settle,
             "reads": self.reads,
         }
 
+    def run_namespace(self, values: list[str], probe: str,
+                      extra_settle: float) -> dict:
+        """Arms (f)/(g): the `reset()` condition, reproduced exactly.
+
+        Deliberately NOT the per-key delete used by (a)-(e). `reset()` issues
+        `delete_all(app_id=...)` — one call wiping the namespace across every
+        scope — then polls the namespace until it reads empty and returns,
+        after which the runner immediately writes. That sequence is what this
+        reproduces, including the emptiness poll, because the open question is
+        whether reading empty actually means the delete has propagated.
+        """
+        # No opening cleanup: this arm uses a fresh `app_id`, so the namespace
+        # is empty already and the delete_all below is the only one in the arm.
+        # Seed real residue across a couple of scopes, as a live namespace has.
+        for i, value in enumerate(values):
+            self._add(value, f"{self.uid}__s{i % 2}")
+        seeded, _ = self._wait_until(
+            lambda: len(self._namespace_rows()) >= len(values), CONFIRM_TIMEOUT)
+        if not seeded:
+            return {"arm": self.name, "outcome": "SETUP_FAILED",
+                    "detail": f"only {len(self._namespace_rows())} of "
+                              f"{len(values)} seed writes became retrievable",
+                    "reads": self.reads}
+
+        # The call reset() makes, verbatim.
+        self.c.delete_all(app_id=self.app_id)
+        emptied, empty_after = self._wait_until(
+            lambda: not self._namespace_rows(), CONFIRM_TIMEOUT)
+        if not emptied:
+            return {"arm": self.name, "outcome": "NAMESPACE_NEVER_EMPTIED",
+                    "detail": f"namespace still non-empty {CONFIRM_TIMEOUT}s "
+                              "after delete_all", "reads": self.reads}
+
+        if extra_settle:
+            time.sleep(extra_settle)
+
+        target = f"{self.uid}__s0"
+        self._add(probe, target)
+        landed, took = self._wait_until(
+            lambda: self._visible(probe, target), CONFIRM_TIMEOUT)
+        self.c.delete_all(app_id=self.app_id)
+
+        return {
+            "arm": self.name,
+            "outcome": "WRITE_VISIBLE" if landed else "WRITE_LOST",
+            "detail": (f"post-reset write retrievable after {took:.0f}s"
+                       if landed else
+                       f"post-reset write NOT retrievable within "
+                       f"{CONFIRM_TIMEOUT:.0f}s"),
+            "delete_mechanism": "delete_all(app_id) — as reset()",
+            "residue_seeded": len(values),
+            "namespace_read_empty_after_s": round(empty_after),
+            "extra_settle_s": extra_settle,
+            "reads": self.reads,
+        }
+
+    def _namespace_rows(self) -> list[dict]:
+        self.reads += 1
+        resp = self.c.get_all(filters={"app_id": self.app_id})
+        return resp.get("results", []) if isinstance(resp, dict) else (resp or [])
+
 
 def main() -> int:
     ap = argparse.ArgumentParser(description=__doc__)
@@ -312,6 +412,11 @@ def main() -> int:
         ("e_cross_scope_settle", f"saltmarsh-{stamp}", f"saltmarsh-{stamp}",
          EXTRA_SETTLE, True),
     ]
+    # (name, extra settle) — namespace arms, run via run_namespace().
+    namespace_plan = [
+        ("f_namespace_delete_all", 0.0),
+        ("g_namespace_settle", EXTRA_SETTLE),
+    ]
 
     results = []
     for name, first, second, settle, cross in plan:
@@ -328,6 +433,22 @@ def main() -> int:
         for k, v in result.items():
             print(f"  {k}: {v}")
 
+    for name, settle in namespace_plan:
+        print(f"\n--- arm {name} (namespace delete_all — the reset() condition) ---")
+        print(f"    expected SEARCH cost, worst case: ~{COST_ESTIMATE[name]} units")
+        before = quota_remaining(api_key)
+        arm = Arm(client, name, f"mc_diag__{name}_{stamp}",
+                  app_id=f"{APP_ID}_{name}_{stamp}")
+        result = arm.run_namespace(
+            [f"residue-{i}-{stamp}" for i in range(_RESIDUE_N)],
+            f"probe-{name}-{stamp}", settle)
+        after = quota_remaining(api_key)
+        if before is not None and after is not None and before >= 0 and after >= 0:
+            result["search_units_spent"] = before - after
+        results.append(result)
+        for k, v in result.items():
+            print(f"  {k}: {v}")
+
     print("\n" + "=" * 68)
     for r in results:
         print(f"  {r['arm']:26s} {r['outcome']}")
@@ -383,6 +504,35 @@ def main() -> int:
     else:
         print("  Pattern not anticipated. Record the outcomes and reason from them.")
 
+    f, g = by.get("f_namespace_delete_all"), by.get("g_namespace_settle")
+    W_LOST, W_OK = "WRITE_LOST", "WRITE_VISIBLE"
+    print("\n  NAMESPACE (f)-(g) — the reset() condition, and the leading "
+          "hypothesis:")
+    if f == W_LOST and g == W_OK:
+        print("  (f) failed, (g) succeeded -> POLLING A NAMESPACE UNTIL EMPTY IS\n"
+              "  NOT SUFFICIENT. reset() does exactly that and then returns, so\n"
+              "  every run is exposed: the first writes after any reset can be\n"
+              "  reaped by a delete_all that has stopped being visible. This is\n"
+              "  a harness-side defect as much as a provider behaviour — the fix\n"
+              "  is a settle after reset, sized from the measured empty-to-safe\n"
+              "  gap, not a fixed sleep. Founder ruling before publication.")
+    elif f == W_LOST and g == W_LOST:
+        print("  (f) and (g) both failed -> waiting does not clear it either. A\n"
+              "  delete_all suppresses subsequent writes to the namespace for\n"
+              "  longer than 60s, or until something other than time changes.\n"
+              "  The most serious form, and it would make delete_all unusable as\n"
+              "  a reset primitive.")
+    elif f == W_OK and all(x == "RE_ADD_VISIBLE" for x in (a, b, c, d, e) if x):
+        print("  (f) succeeded and every other arm passed -> no isolated arm\n"
+              "  reproduces the abort. Combined with the standing caveat, that\n"
+              "  points hard at load/backlog dependence, which only a full run\n"
+              "  can exhibit. Do NOT read it as 'nothing is wrong'.")
+    elif f == W_OK:
+        print("  (f) succeeded -> the reset() condition does not reproduce in\n"
+              "  isolation. Read it against the per-key arms before concluding.")
+    else:
+        print("  Pattern not anticipated. Record the outcomes and reason from them.")
+
     print("\nNo conclusion may be published from a single execution of this "
           "script. Re-run to establish stability first.")