Add snapshot-only storage for generated data branches by trask · Pull Request #436 · open-telemetry/shared-workflows

trask · 2026-10-02T01:39:11Z

Add opt-in snapshot-only storage for CI queue data and PR dashboard state to reduce clone size without discarding historical CI measurements or delivery acknowledgements. Addresses #434.

After promoting snapshot-capable code to all dashboard writers and completing the documented paused-writer cutover, enable it with:

gh variable set DATA_BRANCH_SNAPSHOTS --body true --repo open-telemetry/shared-workflows

Each generated branch then retains one complete, parentless snapshot. Explicit push leases prevent stale workers from overwriting accepted updates, and snapshot generations prevent stale fetches from rolling state backward.

Activation defaults to disabled. The cutover procedure includes an external backup, tree verification, and recovery guidance. Existing clones do not shrink automatically, and other refs to old data can retain history.

Refresh schedules and the stalled-run watchdog remain unchanged.

Preserve current CI measurements and dashboard delivery state while allowing generated branches to keep a single parentless snapshot. Gate activation until all dashboard writers support snapshot ordering and explicit push leases. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

opentelemetry-pr-dashboard · 2026-10-02T01:39:53Z

Pull request dashboard status

Merged · refreshed 2026-10-13 15:21 UTC

Status above doesn't look right?

Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

Copilot

Copilot review overview

🟡 Changes recommended

Fresh checkouts can accept history-bearing commits after cutover and reset snapshot generations.

Review effort: Balanced
Findings: 1

Open (1)

Fresh checkout accepts history-bearing heads as generation 0 · New

What changed in this PR

Adds opt-in parentless snapshots for generated dashboard and queue-data branches to reduce clone size while preserving current state.

Changes:

Adds snapshot generation, explicit push leases, and retry handling.
Wires activation into workflows and the queue collector.
Adds tests and operational cutover/recovery documentation.

File	Description
`pull-request-dashboard/README.md`	Documents dashboard snapshot storage.
`github-actions-queue/README.md`	Documents queue-data snapshots.
`.github/workflows/pull-request-dashboard-repo.yml`	Passes the activation variable.
`.github/workflows/pull-request-dashboard-refresh-author-nudges.yml`	Passes the activation variable.
`.github/workflows/pull-request-dashboard-drain.yml`	Passes the activation variable.
`.github/workflows/github-actions-queue-test.yml`	Expands queue test triggers.
`.github/workflows/github-actions-queue-collector.yml`	Uses explicit leases and snapshot commits.
`.github/scripts/pull-request-dashboard/WEBHOOK_SETUP.md`	Documents cutover and recovery.
`.github/scripts/pull-request-dashboard/test_state_branch.py`	Updates state-branch tests.
`.github/scripts/pull-request-dashboard/test_rollout.py`	Tests workflow activation wiring.
`.github/scripts/pull-request-dashboard/test_data_snapshots.py`	Tests snapshot behavior and concurrency.
`.github/scripts/pull-request-dashboard/state_branch.py`	Implements the snapshot protocol.
`.github/scripts/github-actions-queue/DATA_BRANCH_README.md`	Describes generated branch semantics.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot review overview

🔵 Needs a closer look

The distributed Git snapshot protocol and irreversible operational cutover warrant final human validation despite comprehensive tests.

Review effort: Balanced
Findings: None

Resolved since last review (1)

Fresh checkout accepts history-bearing heads as generation 0

Copilot

Copilot review overview

🟡 Changes recommended

History appended to a snapshot can reset its generation and undermine stale-state protection.

Review effort: Balanced
Findings: 1

Open (1)

Reject snapshot ancestry to preserve monotonic generations · New

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot review overview

🔵 Needs a closer look

The force-pushed parentless-history protocol and coordinated operational cutover warrant final human validation despite comprehensive safeguards and tests.

Review effort: Balanced
Findings: 1

Open (1)

Reject snapshot ancestry to preserve monotonic generations

Copilot

Copilot review overview

🟡 Changes recommended

The collector unnecessarily downloads the entire multi-gigabyte legacy history on every pre-cutover run.

Review effort: Balanced
Findings: 1

Open (1)

Avoid fetching full queue history when snapshot mode is disabled · New

Resolved since last review (1)

Reject snapshot ancestry to preserve monotonic generations

…pshot mode Copilot comment: This deepens the legacy queue-data branch to its entire history on every hourly run even while snapshot mode is intentionally unset during the required rollout. Because that history is the multi-gigabyte data this change is meant to compact, pre-cutover collection can now consume substantial time and disk on every fresh runner. Pass the activation variable to this step and only fetch ancestry when snapshot mode is enabled; ordinary append mode only needs the depth-one head. Copilot comment:  ## Copilot review overview ### 🟡 Changes recommended The collector unnecessarily downloads the entire multi-gigabyte legacy history on every pre-cutover run. **Review effort:** Balanced **Findings:** 1 <picture><source media="(prefers-color-scheme: dark)" srcset="https://github.githubassets.com/static/images/icons/copilot-code-review/medium-v2-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://github.githubassets.com/static/images/icons/copilot-code-review/medium-v2-light.svg"><img src="https://github.githubassets.com/static/images/icons/copilot-code-review/medium-v2-light.png" alt="Medium severity" width="62" height="18" align="texttop"></picture> <details open> <summary><strong>Open (1)</strong></summary> - <picture><source media="(prefers-color-scheme: dark)" srcset="https://github.githubassets.com/static/images/icons/copilot-code-review/medium-v2-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://github.githubassets.com/static/images/icons/copilot-code-review/medium-v2-light.svg"><img src="https://github.githubassets.com/static/images/icons/copilot-code-review/medium-v2-light.png" alt="Medium severity" width="62" height="18" align="texttop"></picture> [Avoid fetching full queue history when snapshot mode is disabled](#discussion_r4175743106) · New </details> <details> <summary><strong>Resolved since last review (1)</strong></summary> - <picture><source media="(prefers-color-scheme: dark)" srcset="https://github.githubassets.com/static/images/icons/copilot-code-review/high-v2-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://github.githubassets.com/static/images/icons/copilot-code-review/high-v2-light.svg"><img src="https://github.githubassets.com/static/images/icons/copilot-code-review/high-v2-light.png" alt="High severity" width="62" height="18" align="texttop"></picture> [Reject snapshot ancestry to preserve monotonic generations](#discussion_r4171386289) </details> Analysis: Pass DATA_BRANCH_SNAPSHOTS to the checkout step and gate the full-depth fetch on it being true, since the ancestry check in commit-data only runs in snapshot mode. Upsides: Avoids downloading multi-gigabyte history on every pre-cutover hourly run. Downsides: If snapshot mode is later turned off, no ancestry is available, which commit-data already forbids for snapshot branches. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot review overview

🔵 Needs a closer look

The destructive branch compaction and distributed concurrency protocol require final human validation of the operational cutover.

Review effort: Balanced
Findings: None

Resolved since last review (1)

Avoid fetching full queue history when snapshot mode is disabled

Co-authored-by: trask <218610+trask@users.noreply.github.com>

Preserve queue-owned live and maintenance execution alongside opt-in data snapshots. Pass the activation variable to the owned drain and guard its propagation. Reconcile deployment guidance so snapshot cutover pauses both lanes and preserves promoted readers, leases, and delivery receipts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

trask requested a balanced review from Copilot October 2, 2026 01:39

Copilot started reviewing on behalf of trask October 2, 2026 01:39

View session

opentelemetry-pr-dashboard Bot mentioned this pull request Oct 2, 2026

Pull Request Dashboard #123

Open

Copilot AI reviewed Oct 2, 2026

View reviewed changes

Comment thread .github/scripts/pull-request-dashboard/state_branch.py Resolved

Fix review findings in #436

9ff8bf7

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

trask requested a balanced review from Copilot October 3, 2026 00:51

Copilot started reviewing on behalf of trask October 3, 2026 00:52

View session

Copilot AI reviewed Oct 3, 2026

View reviewed changes

trask requested a balanced review from Copilot October 3, 2026 02:37

Copilot started reviewing on behalf of trask October 3, 2026 02:37

View session

Copilot AI reviewed Oct 3, 2026

View reviewed changes

Comment thread .github/scripts/pull-request-dashboard/state_branch.py Resolved

Fix review findings in #436

52331a0

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

trask requested a balanced review from Copilot October 3, 2026 17:15

Copilot started reviewing on behalf of trask October 3, 2026 17:16

View session

Copilot AI reviewed Oct 3, 2026

View reviewed changes

trask requested a balanced review from Copilot October 4, 2026 01:45

Copilot started reviewing on behalf of trask October 4, 2026 01:45

View session

Copilot AI reviewed Oct 4, 2026

View reviewed changes

Comment thread .github/workflows/github-actions-queue-collector.yml Resolved

trask requested a balanced review from Copilot October 4, 2026 02:14

Copilot started reviewing on behalf of trask October 4, 2026 02:14

View session

Copilot AI reviewed Oct 4, 2026

View reviewed changes

trask commented Oct 12, 2026

View reviewed changes

Comment thread .github/scripts/pull-request-dashboard/state_branch.py Resolved

Copilot started work on behalf of trask October 12, 2026 18:12

View session

Prevent stale fetches from replacing snapshot refs

b3611c9

Co-authored-by: trask <218610+trask@users.noreply.github.com>

Copilot finished work on behalf of trask October 12, 2026 18:15

trask marked this pull request as ready for review October 12, 2026 22:33

trask requested a review from a team as a code owner October 12, 2026 22:33

trask requested a review from mx-psi October 12, 2026 22:33

mx-psi approved these changes Oct 13, 2026

View reviewed changes

trask added 2 commits October 13, 2026 08:08

Merge branch 'main' into trask-deployment-volume

81ad21c

Merge branch 'main' into trask-deployment-volume

ed1de29

trask merged commit 2695653 into main Oct 13, 2026
7 checks passed

trask deleted the trask-deployment-volume branch October 13, 2026 15:21

Navigation Menu

Repository navigation

Add snapshot-only storage for generated data branches - #436

Add snapshot-only storage for generated data branches#436
trask merged 8 commits into
mainfrom
trask-deployment-volume

trask commented Oct 2, 2026

Uh oh!

opentelemetry-pr-dashboard Bot commented Oct 2, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

trask commented Oct 2, 2026

Uh oh!

opentelemetry-pr-dashboard Bot commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request dashboard status

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

opentelemetry-pr-dashboard Bot commented Oct 2, 2026 •

edited

Loading