Repository navigation
Conversation
zlareb1
force-pushed
the
fix/key-lock-health-recovery
branch
from
November 20, 2026 08:20
08ab7b8 to
1055d3c
Compare
8 of 9 tasks
zlareb1
force-pushed
the
fix/key-lock-health-recovery
branch
from
November 20, 2026 12:49
1055d3c to
30ee114
Compare
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This was referenced Nov 20, 2026
zlareb1
force-pushed
the
fix/key-lock-health-recovery
branch
3 times, most recently
from
November 21, 2026 00:08
58d8f0c to
84d6257
Compare
8 of 9 tasks
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com> (cherry picked from commit 772e2c3) Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com> (cherry picked from commit e3e9b10) Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com> (cherry picked from commit c74ea94) Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Shared client schema reads run with an independent context. Record only decision-tagged statements while retaining unbounded decision reads as failures. Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com> (cherry picked from commit 57327d5) Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Exercise each default-field refusal and JSON numeric distinction before consolidating repeated default comparisons. Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com> (cherry picked from commit 73ef63b) Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com> (cherry picked from commit cb594d0) Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Keep every default-field and JSON type check while avoiding a separate encoder setup for each field. Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com> (cherry picked from commit 0bfc712) Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com> (cherry picked from commit a94d2f9) Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Keep the blocked close, deadline observers and loop timers on one private clock. A controlled independent clock advance reproduces the elapsed-time dependency while the original cleanup and refusal assertions stay intact. Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com> (cherry picked from commit 4b03db4) Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com> (cherry picked from commit 0796d4d) Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
The fake leg delays and request budget share a private event loop clock. Preserve the original timing inputs and assertions, and require blocked legs to time out and cancel. Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com> (cherry picked from commit bfb9d40) Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
zlareb1
force-pushed
the
fix/key-lock-health-recovery
branch
from
November 21, 2026 00:53
84d6257 to
58b9de6
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related: #1094
Related: #1076
Related: #1025
Changelog category
Changelog entry
Health clears a previous memory key-lock failure after a later lease acquisition succeeds. Busy locks and acquisition failures continue to refuse the protected operation.
Why
The lock records backend failures in provider health but never records successful acquisition. A recovered lock therefore leaves health degraded. Record success after receiving a lease, before entering the caller's body, so an unrelated body error cannot be counted as an acquisition failure.
Scope and risk
Replicated-erasure lease provisioning and its unit regression controls are on main. This branch preserves them unchanged. No production lease or erasure rule changes.
One server health module, one Cell proof helper, one integration fixture and six unit-test modules change. The production change records success for one health entry after confirmed lease acquisition. Regression cases cover recovery, a later body error, busy acquisition, another provider's failure and lease release.
The cancellation fixture adds bounded coroutine and signal state to its existing child-stack diagnostic after the unchanged five-second failure. It excludes input, arguments, locals, task names and raw exception text. The original cancellation failure still fails. Its cause remains unknown.
The search-deadline fixture uses a private idle clock for the first-content case. Its original 400 ms budget, fake leg delays, assertions and cancellation finalizers remain. Controls exercise pre-leg elapsed-time sensitivity, genuinely blocked legs, orphan rejection and binding restoration. This fixture correction does not establish the historical scheduling cause or change production deadlines.
The composed test prerequisites scope SCIM person-query observations to their decision tag, keep Cell cleanup watchdogs on a private idle clock, and reduce repeated JSON encoder setup in Cell proof comparisons. Existing refusal controls, cleanup bounds, typed comparisons and field-mutation assertions remain intact. The local encoder-count checks do not establish end-to-end Cell performance.
Compatibility, rollout and rollback
Tests
This head includes the exact reviewed cancellation diagnostic and search-deadline fixtures. All 79 current focused cases pass across memory key-lock health, provider health, cancellation diagnostics and search deadlines. Scoped lint, formatting, the existing strict type check, whitespace and prose checks pass. Full exact-head hosted CI and a new independent whole review remain required.
Earlier feature qualification
The following feature-specific results were recorded before this composition. They are prior-source receipts, not new tests of this composed head.
The three erasure-fixture regressions pass after current-main composition.
LOOM_ENV_FILE= .venv/bin/python -m pytest tests/unit/test_memory_key_write_lock.py tests/unit/test_provider_health.py -q: 32 passed.LOOM_ENV_FILE= .venv/bin/python -m pytest tests/unit/test_native_command_retirement.py -k 'prompt_cancellation or prompt_answer' -q: nine passed, five deselected.Direct prerequisite modules cover 10 SCIM observation cases, 47 typed default-comparison cases and nine cleanup cases.
Scoped Ruff lint and format, strict mypy for the changed server module, original-test assertion comparisons and complete-diff whitespace checks pass.
A separate parser cost assertion failed in earlier hosted CI. Its cause remains unresolved; focused results do not qualify that failure or replace full exact-head hosted CI.
AI assistance in the code
AI-generated: Codex wrote the health correction and fixture diagnostics, then checked acquisition ordering, refusal paths, per-provider isolation and release after body errors.
I can explain every line of this diff and how it interacts with the rest of Loom.
Checklist