Rework plugin submission into an async, Minion-driven pipeline - #19
Merged
mrenvoize merged 17 commits intoAug 13, 2026
Merged
Conversation
mrenvoize
force-pushed
the
worktree-repo-picker
branch
from
August 10, 2026 08:15
23c7498 to
5fe7421
Compare
3 tasks
mrenvoize
force-pushed
the
worktree-submission-pipeline
branch
2 times, most recently
from
August 10, 2026 13:00
1d01d36 to
84b7425
Compare
mrenvoize
force-pushed
the
worktree-repo-picker
branch
from
August 10, 2026 13:05
5fe7421 to
8b6f514
Compare
…ebase Rebasing onto the updated repo-picker branch left one call site (new_plugin_confirm's ownership re-check, added by a later commit here) and three test mocks still referring to the removed fetch_public_repos. Also adjusts the new-repo-beyond-page-1 ownership test to mock fetch_releases instead of the now-removed _get_latest_release_from_github, matching this branch's reshaped new_plugin.
fetch_releases/fetch_release_by_tag/download_kpz/fetch_contributors all read public data, which GitHub serves fine without a token -- just at a much lower rate limit (60/hr vs 5000/hr authenticated). Previously these functions either skipped the call entirely when github_app_token was unset, or sent it as a literal Bearer header when left at the .conf.example placeholder, both of which broke submission processing outright rather than degrading gracefully. github_app_token remains supported for when request volume actually warrants the higher rate limit -- it's just no longer required to get basic functionality working.
The 'My Plugins'/'All Plugins' tables had no way to check how a submission was progressing after landing there from somewhere other than the initial post-submission redirect -- no status shown, and no link back to /plugins/:slug (which already shows per-version status with auto-refresh while checks are running). Adds a colour-coded status badge for the latest version (by submission order, not release date, since a re-submitted older release shouldn't sort behind a newer one) and links the plugin title to its detail page.
_parse_metadata already extracts 'author' from the plugin's $metadata block, but process_plugin_version never wrote it to the plugins row -- only name/description/class_name were saved, leaving the Author column on the listing pages always empty regardless of what the plugin declared.
KohaPluginStore::startup registers the Minion plugin against $self->pg *before* any test gets a chance to call $t->app->pg(test_pg()) -- Minion's backend keeps its own independent Mojo::Pg reference from that point on, permanently. Every test that touches Minion (enqueue, perform_jobs, job info) was silently running against whatever pg_dsn happens to be in the developer's own koha_plugin_store.conf, not the test database. This was never actually about forking or async races (both perform_jobs and perform_jobs_in_foreground were reliable once pointed at the right database) -- it was a plain cross-database mismatch, which surfaces as pure timing-dependent flakiness whenever the wrong database happens to have anything else touching it (a real Minion worker attached to a docker dev environment, for example). Adds TestDB::test_app(), which repoints both $app->pg and $app->minion->backend->pg to the test database, and switches every test file from the old two-line Test::Mojo->new + $t->app->pg pattern to it. Also switches task_process_plugin_version.t from perform_jobs to perform_jobs_in_foreground, since there's no reason for a test to fork when it doesn't need to.
mrenvoize
force-pushed
the
worktree-submission-pipeline
branch
from
August 10, 2026 14:37
02145c3 to
be3efe8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the synchronous "fetch latest release → download → parse → confirm → insert" submission pipeline with an async one:
plugins/plugin_versionsrows are created immediately (status = 'submitted'), then a Minion background job (process_plugin_version) downloads the.kpz, extracts it, parses$metadata, computes a SHA-256 digest, and records author/contributor info — updatingstatustopublishedorchanges_requested(with an error message) as it completesGET /plugins/:slugdetail page shows live status, auto-refreshing while a version is still processinggithub_user_access_tokenrenamed togithub_app_token— a dedicated, public-repos-read-only PAT for the store's own background GitHub calls (separate from the developer's own OAuth login token).kpzcache — the job uses an ephemeral temp directoryplugin_contributorstable,plugins.slug/documentation_url,plugin_versions.status/error_message/content_digest/author_username/author_avatar_urlStacked on #18 (repo-picker) — diff here is the 12 commits for this rework.
Built from a design + implementation plan (
docs/superpowers/specs/2026-08-07-submission-pipeline-design.md/docs/superpowers/plans/2026-08-07-submission-pipeline.md, kept on a separateplanningbranch per this repo's convention of never committing plan/design docs onto code branches), executed task-by-task with an independent review (and one fix round where needed) after each task, plus a final whole-branch review.Test plan
prove -l t/— 57 tests, all passingt/plugins_new_plugin.t) and re-verified clean