From 249f14ea1f9d98e3c8f67b3d96f2696daaa6f14d Mon Sep 17 00:00:00 2001 From: JonnyTran Date: Fri, 7 Aug 2026 14:59:43 -0700 Subject: [PATCH 1/4] docs(deployment): deploy-space is keyless via HF Trusted Publishers Documents the OIDC token exchange replacing HF_TOKEN in `deploy-space`, why the `id-token: write` grant is job-scoped, and why `factory_reboot=True` is load-bearing (the snippet was still showing a plain restart). Secrets table now reflects that HF_TOKEN survives only on the `staging` environment, for `deploy-pr-space` alone. --- docs/architecture/deployment.md | 54 ++++++++++++++++++++++++++++----- 1 file changed, 47 insertions(+), 7 deletions(-) diff --git a/docs/architecture/deployment.md b/docs/architecture/deployment.md index a01e23656..4e2089cd1 100644 --- a/docs/architecture/deployment.md +++ b/docs/architecture/deployment.md @@ -289,8 +289,13 @@ Remove it once the trunk flip has been verified. `env_name` selects the GitHub **Environment** (`production` vs `staging`), which is how per-environment secrets/vars are scoped: `DOCKER_REPO`, -`EXTRALIT_SERVER_IMAGE`, `HF_SPACE_ID`, `HF_TOKEN`, -`DOCKER_USERNAME`/`DOCKER_PASSWORD`. +`EXTRALIT_SERVER_IMAGE`, `HF_SPACE_ID`, +`DOCKER_USERNAME`/`DOCKER_PASSWORD` (plus `HF_TOKEN` on `staging`, which only +`deploy-pr-space` still uses — see §4). + +Note the environments carry **no protection rules and no branch policy**, so +`environment:` here is a scoping mechanism, not an approval gate. What actually +confines production access is the per-job `permissions:` block (§3). ### Job `build` Builds the self-contained Space image **on top of the server image**: @@ -309,13 +314,38 @@ multi-process runtime (elastic + redis + RQ workers + FastAPI). Pushed to (staging), tagging `:latest` when `tag_latest=true`. ### Job `deploy-space` — *non-PR builds only* (`pr_space_slug == ''`) -Restarts the live Space so it pulls the freshly pushed image: +**Factory-reboots** the live Space so it re-pulls the freshly pushed image: -```bash -curl -X POST "https://huggingface.co/api/spaces/${HF_SPACE_ID}/restart" \ - -H "Authorization: Bearer $HF_TOKEN" +```yaml +permissions: + contents: read + id-token: write # mint the OIDC token; see below +env: + HF_OIDC_RESOURCE: spaces/${{ vars.HF_SPACE_ID }} +``` +```python +HfApi().restart_space(space, factory_reboot=True) ``` +`factory_reboot=True` is load-bearing. The Space is a thin `FROM ` +Dockerfile, and a *plain* restart reuses the image HF already built without +re-pulling that base — the job goes green while the Space serves the old build. +That is the v0.7.0 failure: the Space cycled `RUNNING_APP_STARTING → RUNNING` and +still reported 0.6.1. + +**This job holds no HF credential.** It authenticates with +[Trusted Publishers](https://huggingface.co/docs/hub/en/trusted-publishers): +GitHub Actions mints a short-lived OIDC id token, and HF exchanges it (RFC 8693) +for a token scoped to that one Space for ~1h. `huggingface_hub` does the whole +dance inside `get_token()` when `HF_OIDC_RESOURCE` is set, and raises `OIDCError` +rather than falling back to an ambient credential. + +Each Space registers a publisher pinned to repo `Extralit/extralit-hf-space`, +branch `main`, workflow `build-hf-space.yml`. Those claims are satisfied by +*every* job in this file, so the `id-token: write` grant is deliberately scoped to +this job alone — `build` and `deploy-pr-space` inherit only the workflow-level +`contents: read` and therefore cannot mint a token to exchange at all. + `HF_SPACE_ID` is the environment-scoped Space (see §5): the **`production`** environment points at `extralit/public-demo` — the live public demo served at **** — while the **`staging`** @@ -382,10 +412,20 @@ resolve **per environment**. | Secret | Repo-level | `production` env | `staging` env | | ----------------- | :--------: | :--------------: | :-----------: | -| `HF_TOKEN` | ✅ (default) | ✅ (override) | ✅ (override) | +| `HF_TOKEN` | — | — | ✅ | | `DOCKER_USERNAME` | — | ✅ | ✅ | | `DOCKER_PASSWORD` | — | ✅ | ✅ | +`HF_TOKEN` survives **only** on `staging`, and only for `deploy-pr-space`. +Trusted Publishers scope a token to an *existing* repo, so they cannot cover +`duplicate_space()`, which creates `extralit-dev/pr-N` on demand. That leaves the +remaining token's write access confined to the `extralit-dev` org — nothing can +reach `extralit/public-demo` with a stored credential. + +> Deleting the repo-level `HF_TOKEN` is part of this, not an afterthought: +> `secrets.HF_TOKEN` silently falls back to it, so leaving it in place would make +> removing the `production` override purely cosmetic. + ### `extralit/extralit` (monorepo) - **Repo variables:** none. From 554c70641525681f9db6d675f9bc526ce92635d4 Mon Sep 17 00:00:00 2001 From: JonnyTran Date: Fri, 7 Aug 2026 17:44:54 -0700 Subject: [PATCH 2/4] docs(claude): update Vector DB migration details and streamline development workflow Replaced the Vector DB description to reflect the migration to Lancedb. Removed outdated sections on Python package management and running services, consolidating the development workflow documentation for clarity. Updated deployment and branching guidelines to enhance understanding of the release process. --- CLAUDE.md | 54 +++--------------------------------------------------- 1 file changed, 3 insertions(+), 51 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 6b00f2908..3cbfc6d96 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,7 +5,7 @@ - **extralit-frontend/**: Vue 3 / Nuxt 4 (Vite); Pinia state management - **extralit/**: Python SDK client - **extralit-hf-space/**: Self-contained HF Spaces deployment bundle (Docker; bundles Elasticsearch + Redis + OCR) — git submodule -- **Vector DB**: Elasticsearch/OpenSearch (separate service) +- **Vector DB**: Elasticsearch/OpenSearch (migrating to Lancedb) ### Key Patterns - Backend: SQLAlchemy ORM, Alembic migrations, async pytest @@ -41,44 +41,7 @@ cd extralit-server && uv run alembic -c src/extralit_server/alembic.ini upgrade docker-compose up -d ``` -## Development Workflow - -# Python Package Management with uv -Use uv exclusively for Python package management in this project. - -## Package Management Commands -- All Python dependencies **must be installed, synchronized, and locked** using uv -- Never use pip, pip-tools, poetry, or conda directly for dependency management - -Use these commands: -- Install dependencies: `uv add `, avoid adding to pyproject.toml directly -- Remove dependencies: `uv remove ` -- Sync environment: `uv sync` -- Lock dependencies: `uv lock` - -## Running Python Code -- Run a Python script with `uv run .py` -- Run Python tools with `uv run ` (e.g. `uv run pytest`, `uv run ruff check`, `uv run ruff format`, `uv run ty check`, `uv run pre-commit`, `npm run lint`, `npm run format`) -- Launch a Python REPL with `uv run python` - -### Running Services -```bash -cd extralit-server && uv run python -m extralit_server server-dev # Server + worker -cd extralit-frontend && npm run dev # Frontend -``` - -### Testing -```bash -cd extralit-server && uv run pytest tests # Server tests -cd extralit && uv run pytest tests # SDK tests -cd extralit-frontend && npm run test # Frontend tests -``` - -## Branching - -Trunk-based. **`main` is the trunk and the default branch** — every change, code or docs, -branches from `main` and squash-merges back via PR. There is no `develop` branch and no -`releases/**` branches. +## Deployment and Branching | Ref | Role | Deploys to | |---|---|---| @@ -87,11 +50,6 @@ branches from `main` and squash-merges back via PR. There is no `develop` branch | `vX.Y.Z` tag | the release itself | PyPI, versioned docs, GitHub Release | | PR (non-fork) | preview | ephemeral `extralit-dev/pr-N` | -Branch names: `feat/*`, `fix/*`, `docs/*`, short-lived. PR titles use the matching -`feat:` / `fix:` / `docs:` / `chore:` prefix. Note the generated release notes group PRs by -**label**, not by title prefix (see `.github/release.yml`) — label a PR to place it under a -heading; the title convention is for humans. - **Never push to `release` or create tags by hand.** Releases are one dispatch: ```bash @@ -103,10 +61,4 @@ That stamps the version via `scripts/bump_version.py` and pushes `main`, `releas tag atomically. The version lives in three files — always change it with `python scripts/bump_version.py set --version X.Y.Z`, never by hand. -See `docs/architecture/deployment.md` for the full pipeline and -`extralit/docs/community/release_guide.md` for the release runbook. - -## Git Workgrees -When creating a git workgree, place it at `.worktree/` relative to the repo root, normalizing `/` to `-` in the branch-name. - -Example: branch `feature/add-new-feature` should be placed at `.worktree/feature-add-new-feature`. +See `docs/architecture/deployment.md` for the full pipeline From f998b48b09c42f483e1b164d0fbba6b78ead2113 Mon Sep 17 00:00:00 2001 From: JonnyTran Date: Sat, 8 Aug 2026 01:33:52 -0700 Subject: [PATCH 3/4] docs(deployment): deploy-space commits a digest pin instead of restarting The restart API rejects OIDC tokens with a 401, caught by the staging smoke test. Documents the commit-to-rebuild mechanism, why a digest replaces the factory_reboot workaround, and how to tell a restart-endpoint 401 apart from a misconfigured publisher. --- docs/architecture/deployment.md | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/docs/architecture/deployment.md b/docs/architecture/deployment.md index 4e2089cd1..4d1f69abf 100644 --- a/docs/architecture/deployment.md +++ b/docs/architecture/deployment.md @@ -314,7 +314,9 @@ multi-process runtime (elastic + redis + RQ workers + FastAPI). Pushed to (staging), tagging `:latest` when `tag_latest=true`. ### Job `deploy-space` — *non-PR builds only* (`pr_space_slug == ''`) -**Factory-reboots** the live Space so it re-pulls the freshly pushed image: +Deploys by **committing to the Space repo**, not by restarting it. The Space is a +thin `FROM ` Dockerfile; the job rewrites that `FROM` line to the +**digest** `build` just pushed and commits it, and HF rebuilds on the new commit. ```yaml permissions: @@ -322,16 +324,32 @@ permissions: id-token: write # mint the OIDC token; see below env: HF_OIDC_RESOURCE: spaces/${{ vars.HF_SPACE_ID }} + IMAGE_DIGEST: ${{ needs.build.outputs.image_digest }} ``` ```python -HfApi().restart_space(space, factory_reboot=True) +after = re.sub(r"^FROM\s+\S+", f"FROM {repo}@{digest}", before, count=1, flags=re.M) +api.upload_file(path_or_fileobj=after.encode(), path_in_repo="Dockerfile", + repo_id=space, repo_type="space", commit_message=f"Deploy {pin}") ``` -`factory_reboot=True` is load-bearing. The Space is a thin `FROM ` -Dockerfile, and a *plain* restart reuses the image HF already built without -re-pulling that base — the job goes green while the Space serves the old build. -That is the v0.7.0 failure: the Space cycled `RUNNING_APP_STARTING → RUNNING` and -still reported 0.6.1. +**Why a commit and not `restart_space()`.** The restart API answers **401** to an +OIDC token: a repo publisher grants *write access to that repo*, and restarting is +a runtime operation rather than a repo write. Committing is what the credential is +for, and is HF's own +[documented GitHub Actions pattern](https://huggingface.co/docs/hub/en/spaces-github-actions). +A 401 here is specifically the restart endpoint — a misconfigured publisher fails +earlier and differently, as `OIDCError`/`invalid_grant` from the exchange. + +**Why a digest and not a tag.** When the `FROM` line was `:latest`, HF reused the +base image it had already built and never re-pulled — the job went green while the +Space served the old build. That is the v0.7.0 failure: it cycled +`RUNNING_APP_STARTING → RUNNING` and still reported 0.6.1. A digest cannot resolve +to a previously-built image, which retires the `factory_reboot=True` workaround +that used to paper over this. + +Re-deploying an unchanged digest is a no-op commit and so triggers no rebuild; that +path skips the wait and asserts the Space's *current* stage is `RUNNING`, so an +earlier failed build is never reported as a green redeploy. **This job holds no HF credential.** It authenticates with [Trusted Publishers](https://huggingface.co/docs/hub/en/trusted-publishers): From 9a576d567473479299a3ffb984c0c059737e03a2 Mon Sep 17 00:00:00 2001 From: JonnyTran Date: Sat, 8 Aug 2026 01:36:10 -0700 Subject: [PATCH 4/4] docs(deployment): purge stale restart references from the deploy path [skip review] Roborev #329 (High) flagged that the doc described a deploy-space that did not exist in the submodule at any ref. The workflow has since landed on extralit-hf-space main, and the mechanism changed again (commit a digest pin rather than call the restart API, which rejects OIDC tokens with a 401). Sweeps the references the earlier commit missed: the two flowchart edges, the routing and resolve-env tables, and both walkthroughs in the pipeline section. Also sequences the HF_TOKEN deletion explicitly. Nothing on the deploy path reads secrets.HF_TOKEN anymore, so deleting early breaks nothing, but it does discard the one-commit rollback to a stored token before the keyless path has deployed to production even once. --- docs/architecture/deployment.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/docs/architecture/deployment.md b/docs/architecture/deployment.md index 4d1f69abf..2cb346cf5 100644 --- a/docs/architecture/deployment.md +++ b/docs/architecture/deployment.md @@ -81,8 +81,8 @@ flowchart TD srvbuild -->|build & push| srvimg srvbuild -->|repository_dispatch: build-hf-space| bhs bhs -->|FROM server image + ES/Redis/OCR| spaceimg - bhs -->|"restart (is_release=true)"| demo - bhs -->|"restart (branch=main)"| dev + bhs -->|"commit digest pin (is_release=true)"| demo + bhs -->|"commit digest pin (branch=main)"| dev bhs -->|duplicate + retarget| prspace ``` @@ -112,7 +112,7 @@ on **`pull_request`**. | `extralit/docs/` | `extralit.docs.yml` | `extralit/docs/**`, `mkdocs.yml` | Versioned docs via `mike` → `gh-pages` | No | | *(whole repo)* | `release.yml` | manual dispatch only | version stamp on `main` + `release` + `vX.Y.Z` tag | **Yes** (indirectly) | | *(none)* | `github-release.yml` | `vX.Y.Z` tag | GitHub Release, once PyPI serves both packages | No³ | -| `extralit-hf-space/`| `build-hf-space.yml` *(other repo)* | repository_dispatch / manual | `extralit-hf-space` Docker image + Space restart/deploy | **Yes** (terminal) | +| `extralit-hf-space/`| `build-hf-space.yml` *(other repo)* | repository_dispatch / manual | `extralit-hf-space` Docker image + Space deploy | **Yes** (terminal) | ¹ The frontend's own workflow tests, lints, and uploads a prerendered SPA artifact; the **live** UI is deployed separately by Vercel's native Git @@ -443,6 +443,12 @@ reach `extralit/public-demo` with a stored credential. > Deleting the repo-level `HF_TOKEN` is part of this, not an afterthought: > `secrets.HF_TOKEN` silently falls back to it, so leaving it in place would make > removing the `production` override purely cosmetic. +> +> **Order matters.** This table is the state *after* a production release has +> deployed keyless end-to-end. Until then both overrides stay, so reverting +> `deploy-space` to a stored token remains a one-commit rollback — deleting them +> early costs nothing on the deploy path (nothing reads `secrets.HF_TOKEN` there +> anymore) but throws away that escape hatch. ### `extralit/extralit` (monorepo) @@ -500,7 +506,7 @@ The cross-repo `client-payload` carries the handoff state: | `image_tag` *(output)* | payload tag, or `latest` (manual) | Docker tag built & deployed | | `tag_latest` *(output)* | `true` on release/trunk | also tag/push `:latest` | | `platforms` *(output)* | `amd64` (staging) / `amd64,arm64` (release) | buildx target platforms | -| `pr_space_slug` *(output)* | `pr-N` / slug for preview refs | empty → restart live Space; set → PR preview | +| `pr_space_slug` *(output)* | `pr-N` / slug for preview refs | empty → deploy live Space; set → PR preview | | `DOCKER_TAGS` | `${DOCKER_REPO}:${IMAGE_TAG}[,:latest]` | tags pushed by build job | | `EXTRALIT_SERVER_IMAGE` *(build-arg)* | `vars.EXTRALIT_SERVER_IMAGE` | base image the Space is built `FROM` | | `EXTRALIT_VERSION` *(build-arg)* | `image_tag` | base image tag (→ Dockerfile `ARG`) | @@ -566,7 +572,7 @@ The cross-repo `client-payload` carries the handoff state: 4. `repository_dispatch(build-hf-space, {tag: main, is_release: false, branch: main})`. 5. `build-hf-space.yml` → `resolve-env` (env=`staging`) → builds `extralitdev/extralit-hf-space:main` `FROM` the server image. -6. `deploy-space` → `POST /spaces/extralit-dev/develop/restart`. +6. `deploy-space` → commits the digest pin to `extralit-dev/develop`, which rebuilds. 7. Live at ****. **Production untouched.** ### Release → public demo (`release` + tag) @@ -579,8 +585,8 @@ The cross-repo `client-payload` carries the handoff state: (`is_release=true`) → `extralit/extralit-server:vX.Y.Z` (+`:latest`), amd64+arm64. 4. `repository_dispatch(build-hf-space, {tag: vX.Y.Z, is_release: true})`. 5. `build-hf-space.yml` → `resolve-env` (env=`production`) → builds - `extralit/extralit-hf-space:vX.Y.Z` → `deploy-space` restarts - `extralit/public-demo`. + `extralit/extralit-hf-space:vX.Y.Z` → `deploy-space` commits that image's + digest to `extralit/public-demo`, which rebuilds. 6. In parallel, the **tag** push drives PyPI (`extralit`, `extralit-server`), versioned docs (`mike deploy X.Y` + `stable`), and the GitHub Release. 7. Public demo live at ****.