Repository navigation
Make an idle tick ask only whether anything changed - #5
Merged
Merged
Conversation
The clock remembered what it had sent in a local state file, which tied delivery to one host: while that host slept every slot waited for GitHub's backstop, and a second host would have sent every slot again. A clock now claims a slot before sending it, by creating a ref under refs/github-cron-trigger-clock/ in the served repository. Ref creation is atomic, so of any number of clocks ticking at once exactly one sends each slot. A failed send releases its claim, so the next tick on any host retries. A line with no marks is baselined; a workflow that stops being enrolled loses its marks, so enrolling it again baselines it again; each line keeps its two newest marks. Marks of a workflow file that could not be read this tick are left alone. `--state` is gone: a host keeps nothing. Without `--send` a tick is a dry run that reports what it would do and writes nothing. The ledger's ref primitives (create with a note, delete, list) are shared with the marks. CI's live check runs two clocks at once against a workflow enrolled for it, clock-check.yml, and checks each slot is sent once. Closes #2.
Every tick downloaded the full text of every workflow file on the default branch and parsed each with yq, though workflow files change a few times a day and a clock ticks every few minutes. For a repository with 52 workflow files that is about 1.3 MB and 52 yq runs per tick. A tick now asks for the id of the default branch's `.github/workflows` tree. When the tree is one it has read before, everything comes from a cache keyed by content id - the tree's entries, each file's parse - and nothing else is fetched or parsed. When it changed, only the files whose blob is new are fetched, in one batched query, and parsed. Cached texts are checked against their git blob id on every read, and anything unreadable is a miss, so the cache can never be stale and deleting it changes no decision. Entries unused for 30 days are pruned. Measured on that 52-file repository: a cold read is 4 queries, 1.3 MB and 3.1 s; an idle read is 1 query, 79 bytes, 0.6 s and no yq. Closes #4.
jeffwall-curlewlabs
changed the base branch from
feat/clock-state-in-repo
to
main
October 5, 2026 20:34
jeffwall-curlewlabs
enabled auto-merge
October 5, 2026 20:52
jeffwall-curlewlabs
disabled auto-merge
October 5, 2026 20:54
Contributor
Author
|
Merged with |
Contributor
Author
|
Branch protection on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4. Stacked on #3: it builds on #3's clock and is based on its branch, so this diff shows only the change below. When #3 merges, GitHub retargets this to
main.Why
Every tick downloaded the full text of every workflow file on the default branch and parsed each with yq, though the files change a few times a day and a clock ticks every few minutes. On a repository with 52 workflow files that is about 1.3 MB and 52 yq runs per tick, per host.
What changes
default_branch.pyreplaces the all-text read, for the clock and formissed:.github/workflowstree. That id changes only when a workflow file changes.$XDG_CACHE_HOME/github-cron-trigger.PARSE_FORMAT, not yq's version. The yq floor exists so supported releases read workflows alike, and an idle tick should start no process.Checked
Unit tests, with canned GraphQL answers and real yq:
The live check now reads this repository twice and requires the second read to be one query with no parse.
Measured against a real repository with 52 workflow files:
The cache held 1.9 MB.