fix(sync): preserve completed items when another hydration fails

CHANGELOG.md

@@ -2,6 +2,8 @@
 
 ## Unreleased
 
+- Isolate shared-head workflow failures without discarding unrelated completed items, stop acquisition and post-batch lookups at the first quota-reserve failure, and retain rolled-back item failures by requested family so metadata-only retries cannot clear failed children.
+- Preserve completed issue and PR hydrations when another item fails, with atomic per-item writes, scoped failure resolution, and unchanged successful-sync coverage. Record issue and comment fetch failures in the existing ledger; resolve recovered parent failures even when child fetching still fails. `sync` and `fill-pr-details` return committed counts while exiting nonzero on incomplete acquisition, including quota stops.
 - Update SQLite's C-parser dependency to v4.29.6 and pin the automation guide's artifact-upload example to v7.0.1.
 - Add explicit cloud archive admission with immutable source, integrity, enrichment, and warning evidence while preserving strict publication defaults. Thanks @vincentkoc.
 - Compatibility: report portable publication time as `last_export_at`; `last_sync_at` now describes retained successful sync runs instead of old repository scan checkpoints. Thanks @obviyus.

docs/commands.md

@@ -43,7 +43,7 @@ These work on every command.
 | Command | Purpose | Docs |
 | --- | --- | --- |
 | `gitcrawl sync owner/repo [--state --since --numbers <refs> --limit --include-comments --include-pr-details --with pr-details --progress-file <absolute-path> --json]` | Sync issues and PRs from GitHub into local SQLite | [Sync](/sync/) |
-| `gitcrawl sync-failures owner/repo [--include-resolved --limit N --json]` | List failed PR hydration attempts and optional resolved history | [Sync](/sync/#hydration-depth) |
+| `gitcrawl sync-failures owner/repo [--include-resolved --limit N --json]` | List failed issue, comment, and PR hydration attempts and optional resolved history | [Sync](/sync/#hydration-depth) |
 | `gitcrawl coverage [owner/repo \| --repos owner/a,owner/b] [--min-missing-pr-details N --json]` | Report archive, PR-detail, and enrichment coverage/freshness | — |
 | `gitcrawl fill-pr-details owner/repo [--limit --order --batch-size --reserve-rate-limit --include-comments --json-progress --json]` | Hydrate locally missing pull request detail rows in bounded batches | — |
 | `gitcrawl capture owner/repo [--schema gitcrawl.capture.v1 --since RFC3339 --output path --json]` | Export a deterministic code-free conversation snapshot | [Conversation capture](/capture/) |
@@ -59,6 +59,8 @@ request would cross that floor. The live probe observes other processes and
 tools that share the token. This is best-effort because an unrelated consumer
 can spend quota between the probe and request; the 1500 default provides
 concurrency headroom. Pass `--reserve-rate-limit N` to choose another floor.
+An incomplete fill, including a quota stop, exits nonzero while returning the
+counts already committed. See [partial failures](/sync/#partial-failures).
 
 For an end-to-end first-run sequence that combines `status --json`, `doctor --json`, `sync --numbers`, bounded `--sync-if-stale` search, `gitcrawl runs`, and Octopool live reads, see the [maintainer archive workflow](/maintainer-archive/).
 

docs/sync.md

@@ -28,7 +28,7 @@ A sync writes:
 - `thread_revisions` — immutable revisions with content-addressed canonical evidence payloads when fully hydrated thread or review evidence changes
 - `thread_fingerprints` — one deterministic `thread-fingerprint-v2` row for each persisted revision
 - `documents` — canonical thread documents (when bodies change)
-- `run_records` — sync run statistics
+- `sync_runs` — sync run statistics and retry checkpoints
 
 Revision and fingerprint production fails closed on incomplete evidence. Issues require
 `--include-comments`; pull requests require both `--include-comments` and
@@ -90,7 +90,8 @@ observation ordering. It does not fetch, clear, or mark files, commits, checks,
 workflow runs, or review-thread resolution as fresh. Comments remain independent:
 add `--include-comments` when needed. Selecting both hydration modes uses full
 `pr-details` hydration. Metadata-only hydration does not create full PR revisions
-or fingerprints, and it resolves only earlier metadata-fetch failures.
+or fingerprints. It resolves PR metadata-fetch failures, not failures of omitted
+child collections.
 
 Full PR details also populate `pull_request_files`, `pull_request_commits`,
 `pull_request_checks`, and `github_workflow_runs` for local review and search.
@@ -108,7 +109,40 @@ the incomplete review-thread response is not saved as complete evidence.
 
 Use `gitcrawl coverage [owner/repo] --json` to inspect archive completeness after a sync. It reports issue, PR, comment, and review counts alongside hydrated PR detail rows, missing PR details, known failed hydrations, and detail-table row counts per repository. The additive `enrichment` object exposes supported, eligible, covered, fresh, missing, stale, completeness, ratios, and latest timestamps for revisions, fingerprints, key summaries, clusters, and PR details. Use `--repos owner/a,owner/b` to compare selected repositories and `--min-missing-pr-details N` to focus backfill work on repositories with gaps.
 
-`gitcrawl sync-failures owner/repo --json` lists unresolved PR hydration failures with their operation, error class and message, timestamps, and retry count. Add `--include-resolved` to inspect failures cleared by a later successful hydration. This operational ledger stays local when `portable prune` runs unless the publisher explicitly passes `--include-sync-failures`, which retains the ledger only after replacing every error message with a redaction marker.
+`gitcrawl sync-failures owner/repo --json` lists unresolved issue, comment, and PR hydration failures with their operation, error class and message, timestamps, and retry count. Add `--include-resolved` to inspect failures cleared by a later successful hydration of that same family. This operational ledger stays local when `portable prune` runs unless the publisher explicitly passes `--include-sync-failures`, which retains the ledger only after replacing every error message with a redaction marker.
+
+## Partial failures
+
+Each completed issue or PR commits atomically with its requested children,
+document, revision, fingerprint, and failure resolutions. A failed item does not
+roll back completed siblings. Shared-head workflow observations are consolidated
+before those writes, so sibling ordering cannot replace a newer snapshot.
+A consolidation failure excludes only its shared-head group, records those
+items as failed PR-detail hydrations, and preserves unrelated completed items.
+
+An item fetch failure records its actual operation: `issue`, `issue_comments`,
+`pull_reviews`, `pull_review_comments`, `pull_review_threads`,
+`pull_request_metadata`, or `pull_request_details`. A failed issue lookup does
+not create a thread stub. Failed child fetches can retain the observed parent
+metadata, but do not replace incomplete child collections or certify complete
+evidence. Cancellation stops further work; transactions already committed remain.
+A quota-reserve failure also stops new acquisition, including quota probes.
+Completed payloads can still commit. Skipped requests are not recorded as
+failures; shared-head groups that still need verification remain uncommitted.
+
+An item transaction failure records each requested operation after rollback,
+with only an existing parent reference when available. It does not recreate the
+rolled-back item. A retry resolves only the families it actually persists:
+metadata-only retries leave comments and full-detail failures unresolved.
+
+An incomplete batch exits nonzero and never records a successful sync or advances
+the closed-sweep watermark. Before partial writes, archives without a recorded
+watermark retain their previous retry lower bound as a `checkpoint` in
+`sync_runs`. A new archive uses the default 24-hour lower bound. This checkpoint
+is not successful list coverage or freshness. Retry the failed numbers with the
+same hydration flags after resolving the reported cause.
+Older binaries can read these archives but do not honor the retry checkpoint
+when writing. Do not downgrade the writer to resume a partially completed sync.
 
 `--include-code` is accepted for compatibility but is currently a no-op.
 
@@ -177,6 +211,15 @@ stale-observation counts, the requested scope when present, and the database
 write destination. Use `gitcrawl runs owner/repo --kind sync --json` for
 recorded run IDs.
 
+After partial persistence, `sync --json` still emits committed counts, but exits
+nonzero and leaves progress marked `failed`. `fill-pr-details --json` likewise
+reports committed `filled` and remaining selected items, including a partially
+completed batch. A quota stop exits nonzero with
+`stopped_reason: "rate-limit-reserve"`; other sync failures use `"sync-failed"`.
+The partial batch and final result retain the stopping request's quota snapshot
+without a subsequent credential or quota lookup.
+Automation must check the exit status, not treat a JSON result as success.
+
 ## Common workflows
 
 ### First-time setup for a repo

internal/cli/app_test.go

@@ -6895,8 +6895,9 @@ func TestFillPRDetailsDefaultRateLimitFloorUsesLiveSharedQuota(t *testing.T) {
 	run := New()
 	var stdout bytes.Buffer
 	run.Stdout = &stdout
-	if err := run.Run(ctx, []string{"--config", configPath, "fill-pr-details", "openclaw/gitcrawl", "--limit", "1", "--json"}); err != nil {
-		t.Fatalf("fill-pr-details: %v", err)
+	var reserveErr *gh.RateLimitReserveError
+	if err := run.Run(ctx, []string{"--config", configPath, "fill-pr-details", "openclaw/gitcrawl", "--limit", "1", "--json"}); !errors.As(err, &reserveErr) {
+		t.Fatalf("incomplete fill must return quota failure: %v", err)
 	}
 	var result struct {
 		Selected      int                  `json:"selected"`
@@ -7023,8 +7024,9 @@ func TestFillPRDetailsReserveRateLimitStopsBeforeCrossingDuringBatch(t *testing.
 	run := New()
 	var stdout bytes.Buffer
 	run.Stdout = &stdout
-	if err := run.Run(ctx, []string{"--config", configPath, "fill-pr-details", "openclaw/gitcrawl", "--limit", "1", "--reserve-rate-limit", "10", "--json"}); err != nil {
-		t.Fatalf("fill-pr-details: %v", err)
+	var reserveErr *gh.RateLimitReserveError
+	if err := run.Run(ctx, []string{"--config", configPath, "fill-pr-details", "openclaw/gitcrawl", "--limit", "1", "--reserve-rate-limit", "10", "--json"}); !errors.As(err, &reserveErr) {
+		t.Fatalf("incomplete fill must return quota failure: %v", err)
 	}
 	var result struct {
 		Selected      int                  `json:"selected"`

internal/cli/sync.go

@@ -76,20 +76,19 @@ func (a *App) runSync(ctx context.Context, args []string) error {
 		IncludePRDetails:  *includePRDetails || with["pr-details"],
 		Progress:          progress.report,
 	})
+	terminalState := syncProgressSucceeded
 	if err != nil {
-		if progressErr := progress.finish(syncProgressFailed); progressErr != nil {
-			return progressErr
-		}
-		return err
+		terminalState = syncProgressFailed
 	}
-	if err := progress.finish(syncProgressSucceeded); err != nil {
+	err = errors.Join(err, progress.finish(terminalState))
+	if stats.Repository == "" {
 		return err
 	}
 	result := struct {
 		syncer.Stats
 		dbTargetInfo
 	}{Stats: stats, dbTargetInfo: target}
-	return a.writeOutput("sync", result, true)
+	return errors.Join(err, a.writeOutput("sync", result, true))
 }
 
 type syncOptions struct {
@@ -208,6 +207,7 @@ func (a *App) runFillPRDetails(ctx context.Context, args []string) error {
 		ReserveRateLimit: reserve,
 		dbTargetInfo:     rt.dbTarget(),
 	}
+	var syncErr error
 	for i := 0; i < len(numbers); i += batchSize {
 		end := i + batchSize
 		if end > len(numbers) {
@@ -229,17 +229,16 @@ func (a *App) runFillPRDetails(ctx context.Context, args []string) error {
 			Quiet:            *jsonProgress,
 			RateLimitReserve: reserve,
 		})
-		if err != nil {
-			var reserveErr *gh.RateLimitReserveError
-			if errors.As(err, &reserveErr) {
-				rate := fillRateLimitResultFromSnapshot(reserveErr.RateLimit, reserve)
-				result.StoppedReason = "rate-limit-reserve"
-				result.RateLimit = &rate
-				break
-			}
-			return err
+		var reserveErr *gh.RateLimitReserveError
+		var rate fillRateLimitResult
+		var hasRate bool
+		if errors.As(err, &reserveErr) {
+			// The stopping request owns this snapshot; do not resolve credentials
+			// or inspect a different resource after the quota guard has fired.
+			rate, hasRate = fillRateLimitResultFromSnapshot(reserveErr.RateLimit, reserve), true
+		} else if err == nil {
+			rate, hasRate = a.currentFillRateLimit(ctx, reserve)
 		}
-		rate, hasRate := a.currentFillRateLimit(ctx, reserve)
 		batch := fillPRDetailsBatch{
 			Index:              len(result.Batches) + 1,
 			Numbers:            batchNumbers,
@@ -250,8 +249,18 @@ func (a *App) runFillPRDetails(ctx context.Context, args []string) error {
 			batch.RateLimit = &rate
 			result.RateLimit = &rate
 		}
-		result.Batches = append(result.Batches, batch)
+		if err == nil || stats.PRDetailsSynced > 0 {
+			result.Batches = append(result.Batches, batch)
+		}
 		result.Filled += stats.PRDetailsSynced
+		if err != nil {
+			syncErr = err
+			result.StoppedReason = "sync-failed"
+			if reserveErr != nil {
+				result.StoppedReason = "rate-limit-reserve"
+			}
+			break
+		}
 		if *jsonProgress {
 			a.writeFillPRDetailsProgress(fillPRDetailsProgressEvent{
 				Event:      "batch_done",
@@ -268,7 +277,7 @@ func (a *App) runFillPRDetails(ctx context.Context, args []string) error {
 		result.Remaining = 0
 	}
 	result.FinishedAt = time.Now().UTC().Format(time.RFC3339Nano)
-	return a.writeOutput("fill-pr-details", result, true)
+	return errors.Join(syncErr, a.writeOutput("fill-pr-details", result, true))
 }
 
 type fillPRDetailsProgressEvent struct {
@@ -426,8 +435,5 @@ func (a *App) syncRepository(ctx context.Context, owner, repo string, options sy
 		Logger:            logger,
 		Progress:          options.Progress,
 	})
-	if err != nil {
-		return syncer.Stats{}, target, err
-	}
-	return stats, target, nil
+	return stats, target, err
 }