Skip to content

fix(retention): retry failed graph staging releases - #4723

Closed
ScriptedAlchemy wants to merge 1 commit into
masterfrom
fix/4616-graph-staging-retention-retry
Closed

ScriptedAlchemy wants to merge 1 commit into
masterfrom
fix/4616-graph-staging-retention-retry

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

Root cause

The maintenance owner invokes graph-staging cleanup and then independently processes a graph-replay release page. In reconcile_graph_replay_releases, a failed release_one_sealed_generation_staging_rows call already logs graph_staging_release_failed, and its cursor correctly stays unchanged. However, a successful (or empty) release page then returned Complete, Retained, or MoreWork, overwriting the failure. With no other work or backlog, the caller can report a successful full tick and park the failed staging cleanup until the normal long maintenance interval.

Fix

  • Retain the staging-release failure independently of the release-page status, with failure taking precedence after a successfully processed page.
  • Still process the independent replay release page and preserve its cursor/progress; do not block safe work or mutate any durable receipt.
  • Preserve existing typed Failed behavior for malformed release evidence, cancellation, or a failed graph replay.
  • Add a unit regression covering every release-page outcome with staging success and failure, including the empty page case.

A failing staging release now keeps the maintenance owner's existing short retry cadence. This avoids mistaking a logged failed cleanup for a completed tick; no new scheduler, backoff, journal, or destructive path is introduced.

Validation

  • Reviewed the caller in store_maintenance/mod.rs and the two independent graph maintenance paths; the caller maps ReconcileOutcome::Failed to CodeGenerationRetentionOutcomeV1::Failed while allowing independently planned generation collection to continue.
  • Inspected the isolated one-file diff against master at b47a5d3f: 49 insertions, 4 deletions, no other production paths changed.
  • Not executed: Bazel, Rust compilation, rustfmt, or the new unit regression. This environment has no checked-out repository or Bazel/Rust toolchain. Hosted CI must compile and exercise //crates/tracedecay-maintenance:unit_test and the registered retention journey.

Scope

Refs #4616 (partial). This repairs a silent cleanup retry failure. It does not implement the missing sealed-generation collector, unsafe text-artifact isolation, bounded lock starvation, Doctor reclaim guidance, or 70-generation storage-report resource acceptance. Keep #4616 open. This PR intentionally avoids overlapping #4716 (typed sealed-directory deletion errors) and #4714 (bounded metadata scanning).

The branch is a single, non-force commit against the current default branch; it does not alter any other open PR branch. The separate PR-repair automation owns CI follow-through and merges.


Devin Review

Preserve an earlier staging-release failure when the independent graph
replay release page reports Complete, Retained, or MoreWork. That keeps
the existing short retry cadence instead of parking failed staging
cleanup behind the full maintenance interval.

Refs #4616
@changeset-bot

changeset-bot Bot commented Nov 3, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 7b6454e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Consolidated into #4742 (integrate/open-pull-requests) at the captured head shown in its description, as requested. Continue review and further fixes on #4742. This closure does not assert CI readiness; the replacement remains draft. Source branch retained.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant