[CRCR] Add multi-issuer OIDC support to jwt_helper for Buildkite - #8453
Draft
subinz1 wants to merge 8 commits into
Draft
[CRCR] Add multi-issuer OIDC support to jwt_helper for Buildkite#8453subinz1 wants to merge 8 commits into
subinz1 wants to merge 8 commits into
Conversation
Make verify_oidc_token() issuer-aware so downstream repos running on Buildkite can authenticate callbacks alongside existing GitHub Actions repos. The unverified `iss` claim selects the correct JWKS client, and an issuer-specific extractor normalizes the caller identity to the GitHub-style `owner/repo` format that the rest of the pipeline expects. For Buildkite, identity comes from a static (org_slug, pipeline_slug) → repo mapping in BUILDKITE_REPO_MAP since Buildkite OIDC tokens carry no `repository` claim. Unregistered pipelines get a 403; unknown issuers get a 401. lambda_function.py requires no changes — it reads claims["repository"] which verify_oidc_token now guarantees for every supported issuer. Closes #8326
meta-cla
Bot
added
the
CLA Signed
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Collapse short multi-line expressions to single lines per the Python formatter's preferences.
Instead of hardcoding Buildkite (org_slug, pipeline_slug) -> repo
mappings in jwt_helper.py, load them from the allowlist YAML's
buildkite_repos section. This means onboarding a new Buildkite-based
downstream repo only requires adding a line to allowlist.yml — no
Lambda redeployment needed.
Expected allowlist format:
buildkite_repos:
vllm/ci: vllm-project/vllm
acme/build: acme/repo
Replace the allowlist-coupled approach with a standalone ci_providers.yml config file in test-infra. The Lambda fetches it at runtime via CI_PROVIDERS_URL and caches it in Redis, so onboarding a new Buildkite (or future GitLab/Jenkins) downstream repo only requires editing the config file — no Lambda redeployment and no changes to allowlist.yml.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds multi-issuer OIDC support to the CRCR relay, enabling downstream repos running on any supported CI platform (currently GitHub Actions + Buildkite) to authenticate callbacks.
Architecture
Changes
config/ci_providers.ymlutils/jwt_helper.pyci_providers.ymlvia URL + Redis cacheutils/config.pyci_providers_urlfieldutils/redis_helper.pyget_cached_ci_providers/set_cached_ci_providerswith separate Redis keycallback/lambda_function.pyload_ci_providers(config)before token verificationutils/allowlist.pytests/test_jwt_helper.pyOnboarding a new Buildkite repo
Edit
config/ci_providers.yml— no Lambda redeployment needed:Onboarding a new CI provider (future)
_ISSUER_CONFIGinjwt_helper.py_extract_repo_<provider>functionload_ci_provider_mappingsci_providers.ymlCloses #8326
Test plan
CI_PROVIDERS_URLenv var and deploy to staging