-
-
Notifications
You must be signed in to change notification settings - Fork 46
docs(deployment): deploy-space is keyless via HF Trusted Publishers #238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
249f14e
554c706
f998b48
9a576d5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -81,8 +81,8 @@ flowchart TD | |
| srvbuild -->|build & push| srvimg | ||
| srvbuild -->|repository_dispatch: build-hf-space| bhs | ||
| bhs -->|FROM server image + ES/Redis/OCR| spaceimg | ||
| bhs -->|"restart (is_release=true)"| demo | ||
| bhs -->|"restart (branch=main)"| dev | ||
| bhs -->|"commit digest pin (is_release=true)"| demo | ||
| bhs -->|"commit digest pin (branch=main)"| dev | ||
| bhs -->|duplicate + retarget| prspace | ||
| ``` | ||
|
|
||
|
|
@@ -112,7 +112,7 @@ on **`pull_request`**. | |
| | `extralit/docs/` | `extralit.docs.yml` | `extralit/docs/**`, `mkdocs.yml` | Versioned docs via `mike` → `gh-pages` | No | | ||
| | *(whole repo)* | `release.yml` | manual dispatch only | version stamp on `main` + `release` + `vX.Y.Z` tag | **Yes** (indirectly) | | ||
| | *(none)* | `github-release.yml` | `vX.Y.Z` tag | GitHub Release, once PyPI serves both packages | No³ | | ||
| | `extralit-hf-space/`| `build-hf-space.yml` *(other repo)* | repository_dispatch / manual | `extralit-hf-space` Docker image + Space restart/deploy | **Yes** (terminal) | | ||
| | `extralit-hf-space/`| `build-hf-space.yml` *(other repo)* | repository_dispatch / manual | `extralit-hf-space` Docker image + Space deploy | **Yes** (terminal) | | ||
|
|
||
| ¹ The frontend's own workflow tests, lints, and uploads a prerendered SPA | ||
| artifact; the **live** UI is deployed separately by Vercel's native Git | ||
|
|
@@ -289,8 +289,13 @@ Remove it once the trunk flip has been verified. | |
|
|
||
| `env_name` selects the GitHub **Environment** (`production` vs `staging`), which | ||
| is how per-environment secrets/vars are scoped: `DOCKER_REPO`, | ||
| `EXTRALIT_SERVER_IMAGE`, `HF_SPACE_ID`, `HF_TOKEN`, | ||
| `DOCKER_USERNAME`/`DOCKER_PASSWORD`. | ||
| `EXTRALIT_SERVER_IMAGE`, `HF_SPACE_ID`, | ||
| `DOCKER_USERNAME`/`DOCKER_PASSWORD` (plus `HF_TOKEN` on `staging`, which only | ||
| `deploy-pr-space` still uses — see §4). | ||
|
|
||
| Note the environments carry **no protection rules and no branch policy**, so | ||
| `environment:` here is a scoping mechanism, not an approval gate. What actually | ||
| confines production access is the per-job `permissions:` block (§3). | ||
|
|
||
| ### Job `build` | ||
| Builds the self-contained Space image **on top of the server image**: | ||
|
|
@@ -309,13 +314,56 @@ multi-process runtime (elastic + redis + RQ workers + FastAPI). Pushed to | |
| (staging), tagging `:latest` when `tag_latest=true`. | ||
|
|
||
| ### Job `deploy-space` — *non-PR builds only* (`pr_space_slug == ''`) | ||
| Restarts the live Space so it pulls the freshly pushed image: | ||
| Deploys by **committing to the Space repo**, not by restarting it. The Space is a | ||
| thin `FROM <pushed image>` Dockerfile; the job rewrites that `FROM` line to the | ||
| **digest** `build` just pushed and commits it, and HF rebuilds on the new commit. | ||
|
|
||
| ```bash | ||
| curl -X POST "https://huggingface.co/api/spaces/${HF_SPACE_ID}/restart" \ | ||
| -H "Authorization: Bearer $HF_TOKEN" | ||
| ```yaml | ||
| permissions: | ||
| contents: read | ||
| id-token: write # mint the OIDC token; see below | ||
| env: | ||
| HF_OIDC_RESOURCE: spaces/${{ vars.HF_SPACE_ID }} | ||
| IMAGE_DIGEST: ${{ needs.build.outputs.image_digest }} | ||
| ``` | ||
| ```python | ||
| after = re.sub(r"^FROM\s+\S+", f"FROM {repo}@{digest}", before, count=1, flags=re.M) | ||
| api.upload_file(path_or_fileobj=after.encode(), path_in_repo="Dockerfile", | ||
| repo_id=space, repo_type="space", commit_message=f"Deploy {pin}") | ||
| ``` | ||
|
|
||
| **Why a commit and not `restart_space()`.** The restart API answers **401** to an | ||
| OIDC token: a repo publisher grants *write access to that repo*, and restarting is | ||
| a runtime operation rather than a repo write. Committing is what the credential is | ||
| for, and is HF's own | ||
| [documented GitHub Actions pattern](https://huggingface.co/docs/hub/en/spaces-github-actions). | ||
| A 401 here is specifically the restart endpoint — a misconfigured publisher fails | ||
| earlier and differently, as `OIDCError`/`invalid_grant` from the exchange. | ||
|
|
||
| **Why a digest and not a tag.** When the `FROM` line was `:latest`, HF reused the | ||
| base image it had already built and never re-pulled — the job went green while the | ||
| Space served the old build. That is the v0.7.0 failure: it cycled | ||
| `RUNNING_APP_STARTING → RUNNING` and still reported 0.6.1. A digest cannot resolve | ||
| to a previously-built image, which retires the `factory_reboot=True` workaround | ||
| that used to paper over this. | ||
|
|
||
| Re-deploying an unchanged digest is a no-op commit and so triggers no rebuild; that | ||
| path skips the wait and asserts the Space's *current* stage is `RUNNING`, so an | ||
| earlier failed build is never reported as a green redeploy. | ||
|
|
||
| **This job holds no HF credential.** It authenticates with | ||
| [Trusted Publishers](https://huggingface.co/docs/hub/en/trusted-publishers): | ||
| GitHub Actions mints a short-lived OIDC id token, and HF exchanges it (RFC 8693) | ||
| for a token scoped to that one Space for ~1h. `huggingface_hub` does the whole | ||
| dance inside `get_token()` when `HF_OIDC_RESOURCE` is set, and raises `OIDCError` | ||
| rather than falling back to an ambient credential. | ||
|
|
||
| Each Space registers a publisher pinned to repo `Extralit/extralit-hf-space`, | ||
| branch `main`, workflow `build-hf-space.yml`. Those claims are satisfied by | ||
| *every* job in this file, so the `id-token: write` grant is deliberately scoped to | ||
| this job alone — `build` and `deploy-pr-space` inherit only the workflow-level | ||
| `contents: read` and therefore cannot mint a token to exchange at all. | ||
|
|
||
| `HF_SPACE_ID` is the environment-scoped Space (see §5): the **`production`** | ||
| environment points at `extralit/public-demo` — the live public demo served at | ||
| **<https://extralit-public-demo.hf.space>** — while the **`staging`** | ||
|
|
@@ -382,10 +430,26 @@ resolve **per environment**. | |
|
|
||
| | Secret | Repo-level | `production` env | `staging` env | | ||
| | ----------------- | :--------: | :--------------: | :-----------: | | ||
| | `HF_TOKEN` | ✅ (default) | ✅ (override) | ✅ (override) | | ||
| | `HF_TOKEN` | — | — | ✅ | | ||
| | `DOCKER_USERNAME` | — | ✅ | ✅ | | ||
| | `DOCKER_PASSWORD` | — | ✅ | ✅ | | ||
|
|
||
| `HF_TOKEN` survives **only** on `staging`, and only for `deploy-pr-space`. | ||
| Trusted Publishers scope a token to an *existing* repo, so they cannot cover | ||
| `duplicate_space()`, which creates `extralit-dev/pr-N` on demand. That leaves the | ||
| remaining token's write access confined to the `extralit-dev` org — nothing can | ||
| reach `extralit/public-demo` with a stored credential. | ||
|
Comment on lines
+437
to
+441
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- workflow files ---'
git ls-files .github/workflows
printf '%s\n' '--- HF_TOKEN and staging references ---'
rg -n -C 4 'HF_TOKEN|deploy-pr-space|deploy-space|environment:\s*staging|duplicate_space|extralit/public-demo|extralit-dev/pr-' .github/workflows docs/architecture/deployment.mdRepository: Extralit/extralit Length of output: 24628 🏁 Script executed: #!/bin/bash
set -e
repo='extralit/extralit-hf-space'
printf '%s\n' '--- workflow listing ---'
curl -fsSL "https://api.github.com/repos/$repo/contents/.github/workflows" |
jq -r '.[] | .path' |
sort
printf '%s\n' '--- HF_TOKEN references in workflow files ---'
for path in $(curl -fsSL "https://api.github.com/repos/$repo/contents/.github/workflows" |
jq -r '.[].path'); do
body=$(curl -fsSL "https://raw.githubusercontent.com/$repo/main/$path")
if printf '%s' "$body" | rg -n -q 'HF_TOKEN|deploy-pr-space|deploy-space|environment:|repository_dispatch|workflow_dispatch'; then
printf '\n### %s\n' "$path"
printf '%s\n' "$body" | rg -n -C 5 'HF_TOKEN|deploy-pr-space|deploy-space|environment:|repository_dispatch|workflow_dispatch'
fi
doneRepository: Extralit/extralit Length of output: 324 🏁 Script executed: #!/bin/bash
set -e
url='https://raw.githubusercontent.com/extralit/extralit-hf-space/main/.github/workflows/build-hf-space.yml'
printf '%s\n' '--- build-hf-space.yml ---'
curl -fsSL "$url" |
rg -n -C 6 'HF_TOKEN|deploy-pr-space|deploy-space|environment:|repository_dispatch|workflow_dispatch'Repository: Extralit/extralit Length of output: 6239 🏁 Script executed: #!/bin/bash
set -e
url='https://raw.githubusercontent.com/extralit/extralit-hf-space/main/.github/workflows/build-hf-space.yml'
tmp=$(mktemp)
trap 'rm -f "$tmp"' EXIT
curl -fsSL "$url" > "$tmp"
python3 - "$tmp" <<'PY'
import re, sys
text = open(sys.argv[1]).read()
checks = {
"build_has_HF_TOKEN": bool(re.search(r"(?ms)^ build:.*?(?=^ [A-Za-z0-9_-]+:|\Z).*?HF_TOKEN", text)),
"deploy_space_environment": re.search(r"(?ms)^ deploy-space:.*?^ environment:\s*(.+)$", text),
"deploy_space_condition": re.search(r"(?ms)^ deploy-space:.*?^ if:\s*(.+)$", text),
"deploy_space_HF_TOKEN": bool(re.search(r"(?ms)^ deploy-space:.*?HF_TOKEN:\s*\$\{\{\s*secrets\.HF_TOKEN\s*\}\}", text)),
"deploy_pr_environment": re.search(r"(?ms)^ deploy-pr-space:.*?^ environment:\s*(.+)$", text),
"deploy_pr_condition": re.search(r"(?ms)^ deploy-pr-space:.*?^ if:\s*(.+)$", text),
"deploy_pr_HF_TOKEN": bool(re.search(r"(?ms)^ deploy-pr-space:.*?HF_TOKEN:\s*\$\{\{\s*secrets\.HF_TOKEN\s*\}\}", text)),
}
for key, value in checks.items():
if hasattr(value, "group"):
value = value.group(1).strip()
print(f"{key}={value!r}")
print("\n--- resolve-env and job headers ---")
for i, line in enumerate(text.splitlines(), 1):
if i <= 95 or 140 <= i <= 195:
print(f"{i}:{line}")
PYRepository: Extralit/extralit Length of output: 21558 Sensitive Data Exposure (CWE-732): Incorrect Permission Assignment for Critical Resource Reachability: Internal · Exploitability: Theoretical Align the documentation with the workflow’s token boundary.
🤖 Prompt for AI Agents |
||
|
|
||
| > Deleting the repo-level `HF_TOKEN` is part of this, not an afterthought: | ||
| > `secrets.HF_TOKEN` silently falls back to it, so leaving it in place would make | ||
| > removing the `production` override purely cosmetic. | ||
| > | ||
| > **Order matters.** This table is the state *after* a production release has | ||
| > deployed keyless end-to-end. Until then both overrides stay, so reverting | ||
| > `deploy-space` to a stored token remains a one-commit rollback — deleting them | ||
| > early costs nothing on the deploy path (nothing reads `secrets.HF_TOKEN` there | ||
| > anymore) but throws away that escape hatch. | ||
|
|
||
| ### `extralit/extralit` (monorepo) | ||
|
|
||
| - **Repo variables:** none. | ||
|
|
@@ -442,7 +506,7 @@ The cross-repo `client-payload` carries the handoff state: | |
| | `image_tag` *(output)* | payload tag, or `latest` (manual) | Docker tag built & deployed | | ||
| | `tag_latest` *(output)* | `true` on release/trunk | also tag/push `:latest` | | ||
| | `platforms` *(output)* | `amd64` (staging) / `amd64,arm64` (release) | buildx target platforms | | ||
| | `pr_space_slug` *(output)* | `pr-N` / slug for preview refs | empty → restart live Space; set → PR preview | | ||
| | `pr_space_slug` *(output)* | `pr-N` / slug for preview refs | empty → deploy live Space; set → PR preview | | ||
| | `DOCKER_TAGS` | `${DOCKER_REPO}:${IMAGE_TAG}[,:latest]` | tags pushed by build job | | ||
| | `EXTRALIT_SERVER_IMAGE` *(build-arg)* | `vars.EXTRALIT_SERVER_IMAGE` | base image the Space is built `FROM` | | ||
| | `EXTRALIT_VERSION` *(build-arg)* | `image_tag` | base image tag (→ Dockerfile `ARG`) | | ||
|
|
@@ -508,7 +572,7 @@ The cross-repo `client-payload` carries the handoff state: | |
| 4. `repository_dispatch(build-hf-space, {tag: main, is_release: false, branch: main})`. | ||
| 5. `build-hf-space.yml` → `resolve-env` (env=`staging`) → builds | ||
| `extralitdev/extralit-hf-space:main` `FROM` the server image. | ||
| 6. `deploy-space` → `POST /spaces/extralit-dev/develop/restart`. | ||
| 6. `deploy-space` → commits the digest pin to `extralit-dev/develop`, which rebuilds. | ||
| 7. Live at **<https://extralit-dev-develop.hf.space>**. **Production untouched.** | ||
|
|
||
| ### Release → public demo (`release` + tag) | ||
|
|
@@ -521,8 +585,8 @@ The cross-repo `client-payload` carries the handoff state: | |
| (`is_release=true`) → `extralit/extralit-server:vX.Y.Z` (+`:latest`), amd64+arm64. | ||
| 4. `repository_dispatch(build-hf-space, {tag: vX.Y.Z, is_release: true})`. | ||
| 5. `build-hf-space.yml` → `resolve-env` (env=`production`) → builds | ||
| `extralit/extralit-hf-space:vX.Y.Z` → `deploy-space` restarts | ||
| `extralit/public-demo`. | ||
| `extralit/extralit-hf-space:vX.Y.Z` → `deploy-space` commits that image's | ||
| digest to `extralit/public-demo`, which rebuilds. | ||
| 6. In parallel, the **tag** push drives PyPI (`extralit`, `extralit-server`), | ||
| versioned docs (`mike deploy X.Y` + `stable`), and the GitHub Release. | ||
| 7. Public demo live at **<https://extralit-public-demo.hf.space>**. | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: Extralit/extralit
Length of output: 25782
🏁 Script executed:
Repository: Extralit/extralit
Length of output: 22451
🏁 Script executed:
Repository: Extralit/extralit
Length of output: 423
🏁 Script executed:
Repository: Extralit/extralit
Length of output: 1226
Security Misconfiguration (CWE-16)
Exploitability: Theoretical
State the production authorization boundary correctly.
id-token: writeonly allowsdeploy-spaceto mint a GitHub OIDC token.HF_OIDC_RESOURCEand the Trusted Publisher claims scope the exchanged Hugging Face token to the target Space.Proposed wording
📝 Committable suggestion
🤖 Prompt for AI Agents