feat(slides): automated homepage highlights refresh - #312
Draft
yasinmiran wants to merge 42 commits into
Draft
Conversation
There was a problem hiding this comment.
Pull request overview
Introduces a Node-based automation pipeline (plus tests and a scheduled GitHub Actions workflow) to keep the homepage highlights carousel (src/data/slides.json + src/data/slides/) refreshed automatically using recent site content, with validation and a build gate before auto-merge.
Changes:
- Added a full slides refresh pipeline (collect → rank → select → optional captioning → apply → validate) with unit tests.
- Added a scheduled + manually dispatchable GitHub Actions workflow to run the pipeline, open a PR, and auto-merge after
pnpm build. - Bootstrapped ownership metadata in
src/data/slides.jsonand addedgray-matterdependency for frontmatter parsing.
Reviewed changes
Copilot reviewed 26 out of 27 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/data/slides.json | Adds ownership tags (evergreen / sourceArticle) to bootstrap automation. |
| scripts/slides/validate-slides.test.mjs | Tests slide validation rules, including legacy vs bot image quality gates. |
| scripts/slides/validate-slides.mjs | Implements slide + image validation and diff-scope checks. |
| scripts/slides/slides.AGENTS.md | Defines strict JSON-in/JSON-out contract for optional caption agent. |
| scripts/slides/select.test.mjs | Tests selection logic (hysteresis, swap cap, untracked retention). |
| scripts/slides/select.mjs | Implements deterministic selection/rotation of slides. |
| scripts/slides/refresh.mjs | Orchestrates the end-to-end refresh run and sets workflow outputs. |
| scripts/slides/README.md | Documents ownership model, safety guarantees, and operator commands. |
| scripts/slides/rank.test.mjs | Tests ranking behavior (recency, event decay, anti-repeat). |
| scripts/slides/rank.mjs | Implements scoring + ranking with editorial topic weighting. |
| scripts/slides/opencode.json | Sandboxes OpenCode tool permissions for the caption agent step. |
| scripts/slides/image-probe.test.mjs | Tests header-only probing for PNG/JPEG. |
| scripts/slides/image-probe.mjs | Adds dependency-free image header probing (dimensions/format/bytes). |
| scripts/slides/frontmatter.test.mjs | Tests article discovery and frontmatter parsing. |
| scripts/slides/frontmatter.mjs | Reads article frontmatter (title/summary/date/cover) via gray-matter. |
| scripts/slides/dates.test.mjs | Tests parsing of the repo’s free-text English date format. |
| scripts/slides/dates.mjs | Implements strict free-text date parsing to UTC-midnight. |
| scripts/slides/constants.mjs | Centralizes thresholds, regexes, and topic weighting constants. |
| scripts/slides/collect-candidates.test.mjs | Tests candidate collection + cover quality filtering. |
| scripts/slides/collect-candidates.mjs | Collects current slides + ranks eligible content candidates. |
| scripts/slides/caption-agent.test.mjs | Tests agent fallback and hallucination safeguards. |
| scripts/slides/caption-agent.mjs | Adds optional agent captioning with strict validation + fallback. |
| scripts/slides/apply-slides.test.mjs | Tests transient-field stripping and bot-file pruning rules. |
| scripts/slides/apply-slides.mjs | Applies selected slides, copies covers, and prunes stale bot images. |
| pnpm-lock.yaml | Locks the added gray-matter dependency and transitive deps. |
| package.json | Adds slides:* operator scripts and gray-matter dependency. |
| .github/workflows/refresh-highlights.yml | Adds scheduled workflow to run refresh + build gate + PR/merge + failure issue reporting. |
Files not reviewed (1)
- pnpm-lock.yaml: Generated file
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Member
|
This looks like a great idea. |
korbinib
approved these changes
Jul 28, 2026
The change detector compared only src/alt/caption, so a run whose sole effect was stamping an untracked (CMS-added) entry with `evergreen: true` reported no-op and never wrote the tag back, leaving the entry untracked run after run. Ownership keys are now part of the comparison. Validation flagged the neither-key case but accepted slides carrying both `evergreen` and `sourceArticle`, which reads as bot-managed while silently behaving as a pin. Also drops the README's claim that untracked entries are logged; the stamp shows up in the run's PR diff instead.
…ducers The image quality thresholds and the alt/caption text rules were each written twice, once as an accept predicate in the producer and once as a reject list in the validator, down to a copy-pasted illegal-character regex. Nothing kept the pairs in sync, and drift there is expensive: selection would pick a cover, or the caption agent would emit text, that the gate then rejects, aborting every subsequent run. The validator now owns both predicates and the producers import them. Also has apply() return the array it just wrote so refresh no longer reads the file back to validate it, and unexports REPO_ROOT, which nothing imported.
Six ways an unattended run could damage the carousel or wedge itself: Bot image names were `<year>-<slug>`, which is not unique across collections (news and events both hold 2025/elixir-industry-engagement-day) and collides with CMS uploads, which slugify the alt text and so can also start with a year. A collision made apply() overwrite a human's image and then fail validation on a duplicate src for good. Names now carry the collection, and apply() refuses to write over any file a retained slide still points at. An article pinned by an entry that also carried sourceArticle was eligible for selection again, appearing twice under two filenames, and cleanEntry normalized the evidence away before validation ran. Refs held by pins are now claimed. Pins exceeding the slot limit produced an over-length set that failed the count check on every subsequent run. The run now stops and says which state it is in rather than dropping bot slides to make room. Survivors were matched by ref instead of identity, so two entries sharing one sourceArticle both survived and pushed the set past budget. An article whose summary repeats its title yields alt === caption from the fallback captions, which validation rejects; one such article is in the pool today. Candidates are now screened with the same text rules the gate applies, which subsumes the old non-empty-summary check. Covers whose extension disagrees with their bytes are screened there too. Also closes the bot's PR when an automated merge fails, since it carries a whole-file slides.json that would revert newer slides if merged later, and runs slide validation and the pipeline tests on PRs.
An entry carrying both ownership keys was resolved by dropping sourceArticle on write, which unclaimed the ref and let the next run pick the same article up again as fresh, putting one article on two slides with output that validates clean. Neither key can be assumed to be the right one, so the run now stops and names the entry instead. Two incumbents naming one article both survived, indefinitely and silently. Refs and generated filenames are now claimed once, which also stops two candidates in a single run from generating the same filename: the year comes from the frontmatter date rather than the directory, so two entries can agree on collection, slug and year. apply() tracks what it has already written for the same reason. The warning for a fully pinned carousel claimed there was nothing to rotate while the run went on to delete a bot slide and its image, and exited 0. It now distinguishes a purge from a genuine no-op. An article with no summary and a title over the alt limit passed the candidate screen, since clamping made alt and caption differ by an ellipsis. The one bot-managed image left over from bootstrapping was named outside the bot convention, so it could never be pruned once its article rotated out. It is a hand-made composite rather than the article's cover, so it cannot be regenerated; renaming is what puts it under the bot's ownership. Also stops a merge that failed only at branch cleanup from reporting a red job, deletes the pushed branch when PR creation fails, and decouples the two tests that pr-test.yml now runs from specific articles.
Renaming eosc-entrust.png into the bot convention was wrong. It is a hand-made 960x540 composite, not the article's cover, and the rename opted it out of the filename protection that keeps human assets safe: the next rotation deleted it outright, and re-entry regenerated the slide from the article's raw group photo, a different image. Tagging it evergreen matches what it actually is, and the orphan the rename was chasing cannot occur for a slide that is never rotated. Two incumbents naming one article were deduped by dropping the later entry, which silently deleted a slide and its image on a guess, and left the purge count understating what went to disk. That is the same unanswerable question as a dual-key entry, so it now halts the same way. The restored summary check tested truthiness on a value that need not be a string; YAML yields a number for an unquoted `summary: 2024`, and the bot runs before the build that would reject it, so it crashed with a bare TypeError. Also retries the branch delete when a merge succeeds but its cleanup does not.
… ref Pinning the composite left slides.json with no bot-managed entry, which made the test that walks them assert nothing and left the incumbent rescue with no coverage at all. That path keeps an on-screen article scored once it drops below the ranked pool, so without it hysteresis has nothing to compare and the slide leaves the moment it stops ranking. Driven from a synthetic current now, which is also what it was really testing. A non-string sourceArticle slipped past the duplicate check, since a Set compares objects by reference, and then died inside the comparator on a missing localeCompare. It is hand-edited-JSON only, but an opaque crash is the one outcome this pipeline should never have.
…lts on The README claims `slides:validate` catches all four, and pr-test.yml gates every PR on that claim, but a duplicate sourceArticle and a non-string one both passed: the duplicate check only looked at src, and an object is truthy so the ownership check waved it through. A human PR could land either one and the breakage would surface later, inside a bot run, as a halt nobody asked for.
Nobody sees what the carousel becomes until it is live on the homepage. The selection is deterministic but the captions are not, and an editorial mistake on the front page is worth more than the two days a review costs. Dropping the merge means an open PR can now outlive its run, so each run closes the previous one. The branch carries a whole slides.json computed against main as it stood then, and merging two of them can restore a slide the newer run deliberately dropped.
opencode reads rules from AGENTS.md or from paths under the instructions key, and this file is slides.AGENTS.md with no instructions key, so the model was running on the one-line prompt in caption-agent.mjs alone. The schema, the length caps and the ban on inventing names never reached it. Nothing failed visibly because every failure in this path degrades to summary-derived captions on purpose, which is also why no test caught it: the unit tests inject a fake runAgent and never spawn the binary.
Twenty .mjs files for 1160 lines meant most of them existed to re-import each other, and the split had started to lie: the shared acceptance rules lived in validate-slides.mjs because that is who enforces them, while collect-candidates and caption-agent imported them to screen their own output. One file, sectioned in dependency order, drops that indirection and the three separate CLI entrypoints with it. slides.js collect|refresh|validate is now the only way in, so package.json, pr-test.yml and the refresh workflow all name the same command. No behaviour change: the same 52 tests pass and slides.json validates unchanged.
"Here is the input. Return only the JSON array" reads as "echo the array you
were given", and that is what came back: {id, title, summary} unchanged, every
time. validAgentText then rejected it for missing alt and caption, so every run
silently used summary text and the agent looked like it was working.
Verified against opencode/big-pickle: the old prompt echoes the input, the new
one returns {id, alt, caption} that passes the gate.
The module consolidation left a reference to select.mjs and a filename shape that predates the collection prefix.
…rays Two halt states told an operator less than the docs promised. Exceeding the pin limit reported only a count, and a duplicated ref named the article but not the two slides claiming it, so resolving either meant hunting through slides.json by hand. Naming the src is the entire point of halting instead of guessing. Tests now assert it so the messages cannot quietly regress. The diff-scope guard ran on `git diff`, which cannot see untracked files, so a stray file written outside src/data/slides* passed a check named for catching exactly that.
A bare assignment from a command substitution carries that command's exit status, so under set -e a hiccup listing open PRs ended the step outright. It sat between the branch push and the PR creation, and the cleanup lives in the create failure path, so the branch stayed on the remote with no PR and nothing to ever collect it. Reading the list before the push means a failure there leaves nothing behind. Closing a superseded PR can also fail for an ordinary reason: someone merged it by hand in the seconds since the list was read. That aborted the step and filed a failure issue while a perfectly good PR sat open, so a failed close now warns instead. Failures above these steps went unreported entirely, since a plain if is implicitly ANDed with success(). A broken install stops the carousel just as dead as a broken pipeline, so failure() now reports it, with GH_REPO set because a failure before checkout leaves gh no remote to read the repo from. Also bound the PR list, which silently pages at 30.
The rules told the model that alt describes what the image shows, while the rule above it said to derive every word from the title and summary. Only the title and summary are ever sent, so the first instruction was unfollowable and the model resolved the contradiction by inventing. It captioned a text infographic as a group photo of 59 people, and every gate passed it, because nothing in the pipeline can check a claim about an image against the image. Alt now names the subject from the title, and the rule says outright that the image is not part of the input.
The blocker is not the collection schema. Frontmatter is read straight off the file with gray-matter and never goes through Astro, so the schema has no say; the entries simply have no cover. Adding one is enough, and dateless entries already fall back to editorial weight.
Add Slide seeded a new entry as {src, alt, caption}, which carries no ownership
key, and slide validation rejects exactly that. Every PR the editor opened for
a new slide would have failed CI on a rule the editor itself broke.
A slide added by hand is a human's pick, so evergreen is the honest tag, and it
matches what the refresh job would stamp on it anyway. The Slide interface was
missing both keys as well: they survived a round trip only because the editor
spreads the object it was given.
Order was rebuilt every run as pins, then survivors, then new. The CMS offers up/down reordering and advertises that order matters, so promoting a bot slide above a pin was undone twice a week by a PR whose only content was the reversion. Retained entries now hold the position they had. A stable selection was taken as proof of a sound file: refresh returned success without validating whenever nothing rotated, so a deleted image or an emptied caption would have reported healthy indefinitely. It now validates on that path too, which is the one thing this job is best placed to notice. A null array element threw a TypeError out of both the validator and the selector. Both are contracted to name the offending slide, and `[null]` is valid JSON that a hand-edit can produce. Agent text was judged before trimming but stored after, so a trailing newline, the commonest artifact in model output, silently discarded good text in favour of the fallback. The JSON extractor spanned the first bracket to the last, so any preamble containing one made the whole batch fall back. It now scans for a balanced array, honouring strings.
Porcelain C-quotes any path holding a space or a non-ASCII byte, wrapping it in double quotes, and a quoted path matches none of the prefixes the scope guard checks. A slide image with a space in its name would have been reported as an out-of-scope change and failed the run. -z turns quoting off. Also move the comment about which ownership key wins back above the check it explains, and drop a long dash.
The caption agent is sent a title and a summary, never the image, so any alt asserting what is pictured is invented whatever the article says. It captioned a text infographic as a group photo of 59 people, and when the rules banned that opening it wrote "Group photo from" instead. Banning phrasings is a game of whack-a-mole, so the check now lives in code: alt cannot use photo, image, picture or shown unless the source itself does, which leaves an article about a photo competition free to say photo. Proper nouns were guarded for people only, and only against the summary, while the check behind it reads title and summary both. It now covers cities, countries, organisations and projects, which is where an invented name is likeliest and hardest to spot.
An article with a cover that fails the gates was dropped without a word, so an editor who uploaded one saw the bot ignore their post and had nothing to act on. Twenty-four articles are in that state today, and the reasons are all mundane: a photo straight off a camera is portrait and eight times the size limit, older covers are under 800px wide, two point at SVG logos. collect now carries a rejected list with the reason per article. The gates already computed these strings and threw them away. usableCandidate is now the empty case of that list rather than a parallel implementation, so the two cannot drift.
Pinning every slide at bootstrap left one rotating slot in six, so the swap cap and the hysteresis margin could never come into play, and twelve candidates queued for a single tile. EOSC-ENTRUST and the All Hands 2025 group photo were pinned because they happened to be there, not because they are timeless. A pinned event cannot correct itself: the 2025 photo has been on the homepage for four months since the 2026 All Hands. NeLS, GDI and RDMkit stay, since those are standing service promos. The two images stay on disk. They match no bot filename, so nothing can delete them, and either can be pinned again by adding the entry back.
yasinmiran
marked this pull request as draft
August 24, 2026 08:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Adds a twice-weekly GitHub Actions job that keeps the homepage highlights carousel (
src/data/slides.json) a fresh, rolling set of up to 6 slides drawn from recent news and events. It exists because nobody currently takes the initiative to update the slides.The job does not merge anything. It opens a PR and stops.
How it works
altandcaptionfor new slides. It is sandboxed to JSON in, JSON out with every tool denied, and falls back to summary-derived text on any failure, so a flaky free model cannot break a run.src/data/slides/, rewrites the JSON, and prunes only stale bot-created images.pnpm buildare the machine sanity gate before the PR opens.Ownership model
Every slide carries exactly one tag:
"evergreen": true, pinned. Kept always, text frozen, image never deleted by the bot."sourceArticle": "collection/year/slug", bot-managed. Rescored each run, rotated, dropped when it ages out.A slide added later through the CMS carries neither. It is retained and stamped
evergreenon the next run (fail closed), never dropped. The CMS now pins the slides it creates, so this should only arise from a hand-edit.This PR tags the existing slides and unpins two of them. Pinning all five left one rotating slot in six, which made the swap cap and the hysteresis margin unreachable. NeLS, GDI and RDMkit stay pinned, since they are standing service promos. EOSC-ENTRUST and the All Hands 2025 photo do not: a pinned event cannot correct itself, and that photo sat on the homepage for four months after the 2026 All Hands. Both image files stay on disk, match no bot filename, and can be pinned again by restoring the entry.
Bot-created images are named
<collection>-<year>-<slug>.<ext>, and the bot only ever deletes unreferenced files matching that shape. The collection is part of the name because a slug is unique only within its collection: news and events both hold2025/elixir-industry-engagement-day. Legacy and human-uploaded images cannot match that shape, so they are structurally safe, and the image quality gates (min width 800, max 3 MB, landscape-ish aspect) apply only to bot-created files so grandfathered pins do not trip them.The bot never guesses
Four states are ones a human can author but no bot run can produce. Each halts the run with the offending
srcnamed, writes nothing, and reports on theslides-botissue:sourceArticlesourceArticlethat is not a stringResolving any of them silently would cost a slide or put one on screen twice, so the bot stops and a human edits
slides.json.pnpm slides:validatecatches all four before a merge, which is whypr-test.ymlnow runs it on every PR.Failure is loud
On any abort (validation, build, or PR creation, or a step before them such as a broken install) the workflow opens or updates a single
slides-botlabelled issue rather than failing quietly, and the job exits non-zero.The carousel is validated even on runs that rotate nothing, so a deleted image or an emptied caption raises the issue rather than sitting unnoticed until someone looks at the homepage.
Layout
Four files under
scripts/slides/, plus a README:slides.jsslides.test.jsslides.AGENTS.mdopencode.jsonRequired repo settings (before enabling)
GITHUB_TOKENget no workflow runs, andmainhas no required status checks, so a bot PR is mergeable on one approval with an empty Files-changed checks list. Adding secretSLIDES_BOT_TOKEN(a fine-grained PAT with contents and PR write) makespr-testrun on them. Without it the PR still opens, and the same validation pluspnpm buildhave already run inside the workflow run that created it.SLIDES_AGENT_MODELand secretOPENROUTER_API_KEY. Without them the pipeline uses summary-derived captions and is fully functional.No merge token is needed. A human merges the PR.
Testing
57 unit tests (
pnpm slides:test), passing on Node 24.pr-test.ymlnow runs both the suite andslides validateon every PR. End to end dry runs (SLIDES_AGENT=off pnpm slides:refresh) confirm the pipeline produces a valid carousel, touches onlysrc/data/slides.jsonandsrc/data/slides/, and passespnpm build.refreshandvalidateboth take--diff-scopeto assert nothing outside those two paths changed.Operator commands:
pnpm slides:collect(dry),pnpm slides:refresh,pnpm slides:validate,pnpm slides:test.slides:collectalso reports arejectedlist: every article that has a cover but cannot be used, with the reason. 24 articles are in that state today, mostly covers under 800px wide or phone photos that are portrait and over the 3 MB limit. That is the first place to look when an article never reaches the homepage.Schedule
Mon 07:00 UTC and Fri 15:00 UTC (roughly 08:00 and 16:00 Europe/Oslo, drifting an hour with DST since GitHub cron is UTC only), plus manual dispatch.
Only one refresh PR is open at a time: opening a new one closes any older one. Each run recomputes the whole carousel against main as it stands then, so an older PR is a competing answer rather than an earlier instalment, and merging both could restore a slide the newer run deliberately dropped. Closed PRs keep a Restore branch button, so nothing is lost.
Known limits
funding-and-projectsis in the collection list but nothing from it surfaces yet, because no entry declares acoverand the selector only considers articles that have one. Frontmatter is read straight off the file rather than through the collection schema, so adding acoverto an entry is all it takes; dateless entries are already handled and fall back to editorial weight.altnames the article's subject rather than describing the picture. Wording varies between runs, which is why the text is worth reading on each bot PR.