Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 5 additions & 16 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,19 +60,8 @@ jobs:
- uses: actions/upload-pages-artifact@v5
with:
path: output/site

deploy:
if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'
needs: build
runs-on: ubuntu-24.04
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- uses: actions/configure-pages@v6
- name: Publish the checked dashboard
id: deployment
uses: actions/deploy-pages@v5
retention-days: 30
- name: Explain PR preview publishing
if: github.event_name == 'pull_request'
run: |
echo 'The Pages publisher will reconcile this build after checks complete. A **Pages preview** check links the deployed page from the PR.' >> "$GITHUB_STEP_SUMMARY"
63 changes: 63 additions & 0 deletions .github/workflows/publish-pages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: Publish dashboard and PR previews

on:
workflow_run:
workflows: [Check and publish Quickdash]
types: [completed]
pull_request_target:
types: [closed]
workflow_dispatch:

permissions:
contents: read

# Reconcile all open PRs on every run: Actions may coalesce pending runs.
concurrency:
group: quickdash-pages-publish
cancel-in-progress: false

jobs:
publish:
runs-on: ubuntu-24.04
permissions:
actions: read
contents: write
pages: write
id-token: write
checks: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
# This privileged workflow only executes default-branch code. PR artifacts
# are read as data, including when the completed build came from a fork.
- uses: actions/checkout@v7
with:
ref: ${{ github.event.repository.default_branch }}
- uses: actions/setup-python@v7
with:
python-version: '3.12'
- uses: actions/configure-pages@v6
id: pages
- name: Assemble production and open PR previews
env:
GH_TOKEN: ${{ github.token }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
PAGES_BASE_URL: ${{ steps.pages.outputs.base_url }}
run: |
python3 scripts/pages_preview.py assemble --site "$RUNNER_TEMP/quickdash-pages" \
--default-branch "$DEFAULT_BRANCH" --base-url "$PAGES_BASE_URL"
- uses: actions/upload-pages-artifact@v5
with:
path: ${{ runner.temp }}/quickdash-pages
- name: Publish dashboard and previews
id: deployment
uses: actions/deploy-pages@v5
- name: Link deployed previews from PR checks
env:
GH_TOKEN: ${{ github.token }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
PAGES_BASE_URL: ${{ steps.pages.outputs.base_url }}
run: |
python3 scripts/pages_preview.py report --site "$RUNNER_TEMP/quickdash-pages" \
--default-branch "$DEFAULT_BRANCH" --base-url "$PAGES_BASE_URL"
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ Files opened here stay in your browser; they are not uploaded or included in lin
- Edit or add a self-contained eval file in [configs/evals/](configs/evals/), such as [polymath.yaml](configs/evals/polymath.yaml). Each file holds its scoring rules and language assignments together; the catalogue combines them automatically.
- Add weighting profiles to [configs/weights/](configs/weights/), or optional named eval sets to [configs/sets/](configs/sets/). Each directory has a `default.txt` choosing its startup selection. See [contributing configs](configs/README.md).

Use a pull request or GitHub’s **Add file → Upload files**. Changes on `main` trigger tests and a GitHub Pages rebuild; pull requests are checked without publishing. Invalid inputs stop the update and leave the last successful site online. The repository and dashboard are public, so use browser imports for private comparisons.
Use a pull request or GitHub’s **Add file → Upload files**. Changes on `main` trigger tests and a GitHub Pages rebuild. Successful PR builds get a separate review page: open **Details** on the PR’s **Pages preview** check, or use the [preview index](https://openeurollm.github.io/quickdash/pr-preview/). Previews update after successful builds and are removed when the PR closes. Invalid inputs leave the last successful page online. The repository, dashboard, and PR previews are public, so use browser imports for private comparisons.

The [Pages workflow](.github/workflows/pages.yml) publishes only the generated dashboard, fictional demo, and licenses. Repository maintainers configure **Settings → Pages → Source → GitHub Actions**. Check the repository’s **Actions** tab if an update fails to appear.
The [build workflow](.github/workflows/pages.yml) checks and packages the dashboard, fictional demo, and licenses. The [Pages publisher](.github/workflows/publish-pages.yml) combines the main dashboard with previews at `pr-preview/pr-<number>/`. Repository maintainers configure **Settings → Pages → Source → GitHub Actions**. Check the repository’s **Actions** tab if an update fails to appear; see [publishing and previews](docs/development.md#publish-through-github-pages).

## Build a standalone file

Expand Down
29 changes: 26 additions & 3 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ Run commands from the repository root. Install the Python package with `python -
| --- | --- |
| `quickdash/` | Native Python interpretation, analysis, diagnostics, and CLI. |
| `app/` | Python builder, DOM-independent JavaScript engine (`analysis.js`), browser renderer (`app.js`), HTML, and bundled YAML parser. |
| `tests/` | Public contract tests, browser checks, and optional private-export regressions. |
| `tests/` | Public contract tests, browser checks, publishing tests, and optional private-export regressions. |
| `scripts/` | Assemble and report GitHub Pages production and PR preview deployments. |
| `configs/` | Catalogue manifest, self-contained `evals/` files, weighting profiles, optional named sets, and fictional examples. |
| `results/` | Public CSV exports contributed to the shared dashboard. |
| `examples/` | Public sample export for Pages and parity tests, plus small fictional quickstart data. |
Expand Down Expand Up @@ -108,8 +109,30 @@ This optional check compares included rows, weights, contributions, scores and d

The [workflow](../.github/workflows/pages.yml) runs public tests on pull requests and pushes to `main`. The Python loader and Node filesystem helper both assemble the per-eval files declared by `configs/catalogue.yaml`. Shared tests check assembly, language ownership, file loading, and portable export/import as well as scoring.

After tests pass, it builds the shared dashboard and a separate fictional demo. When `results/` has no CSVs, the shared page embeds `examples/sample-evals.csv`; real shared CSVs take precedence. The sample also runs through both engines in CI for all shipped weighting profiles, eval sets, and aggregation modes, with complete and mismatched coverage. See [contributor requirements](../AGENTS.md). Only `output/site/` is uploaded as the Pages artifact: `index.html`, `demo.html`, and license files. The repository root and private local output are not published as the site.
After tests pass, it builds the shared dashboard and a separate fictional demo. When `results/` has no CSVs, the shared page embeds `examples/sample-evals.csv`; real shared CSVs take precedence. The sample also runs through both engines in CI for all shipped weighting profiles, eval sets, and aggregation modes, with complete and mismatched coverage. See [contributor requirements](../AGENTS.md). Only `output/site/` is uploaded as the build artifact: `index.html`, `demo.html`, and license files. Build artifacts are retained for 30 days. The repository root and private local output are not published as the site.

In repository **Settings → Pages**, select **GitHub Actions** as the source. Publishing uses the generated artifact rather than a checked-in root or `docs/` folder. A successful push to `main` deploys automatically; a failed build leaves the last successful site available. Review build or deployment failures in the repository’s **Actions** tab.
In repository **Settings → Pages**, select **GitHub Actions** as the source. Keep the `github-pages` environment restricted to `main`. The [publisher workflow](../.github/workflows/publish-pages.yml) runs after build completions, when a PR closes, or through **Run workflow**. It executes the publisher script from the default branch and uses successful build artifacts as static data. It never checks out or executes PR code with publishing permissions, including for fork PRs. The build workflow keeps read-only repository access.

The publisher deploys one combined site:

- `/quickdash/` serves the latest successful main build.
- `/quickdash/pr-preview/pr-<number>/` serves that PR’s latest published successful build.
- `/quickdash/pr-preview/` lists open PRs, preview links, and the exact built commits. It labels a retained preview as **Previous successful build** when the current PR head has no successful build yet.

After deployment succeeds, a **Pages preview** check on each current built PR commit links directly to its preview. No PR comments are posted. Closing or merging a PR removes its directory on the next publisher run. A failed main build preserves production; a failed PR build preserves its previous preview. Review both the ordinary `build` check and the built commit before assessing a preview. Previews are public and share the Pages origin with the main dashboard.

Generated files and their source run/commit identifiers persist on the `pages-content` branch, created automatically by the first publisher run. This is storage for the Actions publisher; **do not change Pages to deploy from this branch**. The stored files keep published previews available after their source artifacts expire. Every publisher run reconciles all open PRs and successful main builds, and publishing is serialized so concurrent builds cannot overwrite each other’s previews. Run **Publish dashboard and PR previews** manually to retry a failed deployment or reconcile missed updates. The workflow must be on `main` before automatic previews can run; existing successful artifacts can be picked up during that first deployment.

Only the four expected regular files are accepted from each build artifact, with a 100 MiB limit on each archive layer and its total file content. Unexpected paths, duplicate files, links, and malformed archives are rejected without extracting their paths. Invalid PR artifacts do not prevent production or other previews from publishing. Hidden Git/state files are excluded from the final Pages artifact.

To rehearse assembly with current GitHub artifacts, without pushing a branch, deploying Pages, or writing PR checks:

```sh
python3 scripts/pages_preview.py inspect --repository OpenEuroLLM/quickdash \
--site output/preview-rehearsal --base-url https://openeurollm.github.io/quickdash
open output/preview-rehearsal/pr-preview/index.html
```

This requires an authenticated GitHub CLI (`gh`). The publisher uses Python’s standard library and `gh`; no package installation is required. `python3 -m tests.check` includes archive validation, snapshot updates, fork build identification, stale-build handling, cleanup, and preview-check regression tests. Review build or deployment failures in the repository’s **Actions** tab.

Before pushing, run the affected tests and check the README, config reference, and contribution instructions for changed commands or behavior. Contributions to [results](../results/README.md) and [configs](../configs/README.md) are validated by the same workflow.
Loading