Repository navigation
state: a rehydrated run sees its data in the order the live run built it (#871) - #878
Merged
Merged
Conversation
… it (#871) The graph snapshot and both stores persisted state with sort_keys=True, so a map a program built as {web, code, data} came back {code, data, web} after a cross-process resume. Same content, different string -- so anything derived from it differed across a restart: a draft, a hash, an @exactly_once key. That last one is how it was found, an idempotent publish re-firing on a rehydrated replay of an identical draft. JSON objects preserve order and json.load round-trips it, so nothing but the sort was destroying it. Step results and state cells both round-trip now, on both backends. CANONICAL FORM IS UNCHANGED WHERE AN IDENTITY IS COMPUTED. compute_action_id and the bytecode cache key still hash sorted payloads and must: there two equal maps built in different orders have to produce one answer. The line is REPORTED vs COMPARED -- persisted state is read back and handed to a program, an identity is hashed -- and the test asserts both directions, so removing the sort where it IS load-bearing fails too. Without that half this reads as "sorting is always wrong", which is how the next person removes the wrong one. One rule for the whole record rather than a guest-data field list: `_state_json` names it once. `claim` and `wait` are mostly bookkeeping, but `wait` carries the guest's payload, and a hand-maintained split between "may reorder" and "may not" is the kind that drifts. Nothing there is content-addressed, compared as a string, or indexed on. No canonical mode added to std:json. It was the alternative fix and is not needed for this: @exactly_once already canonicalises internally, so a key derived from a step result is stable across a restart without the caller doing anything. Two test-authoring notes, both found by the tests failing for the wrong reason: - The first store assertion read the repo-root store and took list_runs()[-1], which is a stranger's record among ~400. It looks up its own run by id now, and the store-level check builds its stores in a tempdir instead. - Parametrising the harness by NODUS_WORKFLOW_STORE_BACKEND does not work: the variable is read when the default runner is BUILT and that runner is cached per process, so setting it in setUp after another test built one silently reads a store it is not writing to. Passed alone, failed in the file. The snapshot round trip is backend-independent, so one run covers it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #871.
The defect
The graph snapshot and both stores persisted state with
sort_keys=True, so amap a program built as
{web, code, data}came back{code, data, web}after across-process resume:
Same content, different string — so anything derived from it differed across a
restart: a draft, a hash, an
@exactly_oncekey. That last one is how thereporting project found it, an idempotent publish re-firing on a rehydrated
replay of an identical draft.
JSON objects preserve order and
json.loadround-trips it, so nothing but thesort was destroying it.
Where it was, and where it stays
Six
sort_keyssites, three kinds:compute_action_id, bytecode cache keynodus graphoutput, dependency graph fileThe distinction is
reportedvscompared. Persisted state is read back andhanded to a program, so it must come back as it went in. An identity is hashed,
and there two equal maps built in different orders have to produce one answer.
The regression test asserts both directions — removing the sort where it is
load-bearing has to fail too. Without that half this reads as "sorting is always
wrong", which is how the next person removes the wrong one.
Scope: one site or four?
The graph snapshot alone fixes the reported symptom — step results and state
cells both round-trip after that one line. I fixed the stores as well, because
the store keeps a second copy of
workflow_stateandnodus workflow inspectprints it, so a user was seeing sorted keys while the program sawinsertion order. Measured before and after:
_state_jsonnames the rule once rather than splitting guest data frombookkeeping field by field:
claimandwaitare mostly bookkeeping, butwaitcarries the guest's payload, and a hand-maintained "may reorder / may not" list
is the kind that drifts. Nothing there is content-addressed, compared as a
string, or indexed on, so there is nothing to trade away.
No canonical mode added to
std:jsonThat was the issue's alternative fix. With order round-tripping it is not needed
here:
@exactly_oncealready canonicalises internally, so a key derived from astep result is now stable across a restart without the caller doing anything.
Said so in the changelog rather than leaving it as an open question.
Tests
tests/test_rehydrated_key_order.py, 8 tests. Three per-decision neuters,restoring from saved byte copies:
The third is the inverse direction, and it is the one that matters for the next
reader.
Two test-authoring bugs of mine, both caught by a test failing for the wrong
reason — worth naming because both would have produced a green test that
checked nothing:
list_runs()[-1]— a stranger's record among ~400. It looks its own run up by id now, and the
store-level check builds its stores in a tempdir.
NODUS_WORKFLOW_STORE_BACKENDdoes not work:the variable is read when the default runner is built, and that runner is
cached per process, so setting it in
setUpafter another test built onesilently reads a store it is not writing to. The tests passed alone and failed
in the file. The snapshot round trip is backend-independent, so one run covers
it; the store serialization is covered directly instead.
Checks
lifecycle, migration, effects, determinism boundary): 125 passed
ruffclean ·mypy src/nodus/clean — note it does not coversrc/nodus_lang_workflow/, so CI is the only type check for that halfnodus_gate --allgreen, all ten phases (BUG-869: the local store's age filter hides waiting runs from listing, migration, rehydration and the #174 warning #869's lesson)--alldoes not run them)🤖 Generated with Claude Code