[arekay-nv]

What & why

Records the endpoints source commit in every run's report, so a
result_summary.json / report.txt is self-describing about which code
produced it.

resolve_git_sha() walks a channel chain (first hit wins), validating every
value against a hex-SHA grammar ([0-9a-fA-F]{7,64}, optional -dirty):

Order	Channel	Populated by	Used in
1	git	live git rev-parse of the package's own source checkout (src/ layout + toplevel guard, not cwd)	dev checkouts, bind-mounted checkouts
2	env	ENDPOINTS_GIT_SHA (docker --build-arg GIT_SHA → ENV); ignored inside a git checkout	container images, installed packages
3	none	—	nothing resolvable

A checkout reports live git or nothing. If <root>/.git exists but git can't
read it (git missing, a timeout, dubious ownership, a .git pointing outside a
bind mount), the report says unknown (source: none) and a warning is logged,
rather than recording an image's ENDPOINTS_GIT_SHA, which describes other
code.

The SHA gains -dirty when tracked files in the code that runs differ from
HEAD: src/inference_endpoint, pyproject.toml, uv.lock, and the VBench
scorer subproject under examples/. Untracked files and LFS data elsewhere are
ignored. The probe is
git -c core.checkStat=minimal status --porcelain --untracked-files=no -- <paths>
under GIT_OPTIONAL_LOCKS=0: copied/rsynced checkouts and Docker Desktop bind
mounts (synthetic inode numbers) aren't re-hashed, so they aren't falsely dirty,
and the caller's index is never written. A probe that fails or times out (5s)
reports -dirty and logs why.

Provenance is resolved once per process, at the start of setup_benchmark
(and logged), so every report the process writes — main run and audit phases —
records the same SHA, sampled before the run rather than after it.

The report always prints Git SHA: <sha> (source: <channel>), and
git_sha_source is recorded in result_summary.json.

Changes

• utils/version.py: validated git → env → none resolver; checkout gate;
  scoped checkStat=minimal dirty probe that warns on failure; bytes-safe
  rev-parse; git repo-location env scrub (GIT_DIR/GIT_WORK_TREE/
  GIT_INDEX_FILE/…) so a stray GIT_DIR can't slip a foreign SHA past the
  toplevel guard; per-process memo.
• commands/benchmark/execute.py: setup_benchmark pins and logs
  provenance before any work runs.
• metrics/report.py: new required git_sha_source field; unconditional
  Git SHA: display line.
• scripts/push_docker_image.sh (feat :Testing workflow to push images #405 publish path): forwards
  --build-arg GIT_SHA=<short sha>[-dirty] — the same --short=7 SHA as the
  image tag, with the runtime's dirty scope — so published GHCR images report
  their commit at runtime instead of source: none.
• scripts/Dockerfile.dev: ARG GIT_SHA → ENV ENDPOINTS_GIT_SHA, last so
  the per-commit value doesn't bust cached layers; documented build command
  carries -dirty.
• docs (docs/metrics/*, docs/utils/DESIGN.md) + tests.

Testing

• test_version.py: hermetic tmp repos (no ambient git config or GIT_*);
  real-git tests for dirty scope, copied checkouts, filtered (LFS-style) files
  with stat-only changes, unreadable checkouts (gitfile outside a mount,
  dangling .git symlink) and index immutability; fake-subprocess.run tests
  that decode exactly when the real call would; a test keeping the image-build
  pathspec equal to the runtime's. Mutation-checked: reverting any of the
  behaviors above fails at least one test.
• Docker Desktop with inference-endpoint-dev (no git-lfs) and the LFS parquet
  smudged in a clean bind-mounted clone: (sha, "git"), no false -dirty; a
  bind-mounted worktree whose .git points outside the mount: (None, "none")
  plus the warning, with the image's ENDPOINTS_GIT_SHA set.
• End-to-end CLI against the echo server (offline, and from-config with an
  audit: output-caching block): the setup log line precedes the run;
  report.txt and result_summary.json carry the same SHA across the main and
  audit-phase reports; git runs only at setup (2 invocations per process).
• Full suite: no new failures vs main (remaining local failures are the
  macOS-only socket-path / sched_*affinity / /proc ones, identical on
  main).

Follow-ups (not in this PR)

• A baked build-info channel for installed wheels, landing together with
  the CI step that generates it (and removes it after uv build).

Notes

• Report.git_sha_source is a new required struct field; nothing decodes
  Report back from JSON, so there is no wire/back-compat break.
• Known limitation: code mounted into the image without its .git (a
  tarball, or src/ alone) can't be told apart from the image's own copy and
  reports the image's ENDPOINTS_GIT_SHA. Mount the whole checkout.

🤖 Generated with Claude Code

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[Copilot]

🔵 Needs a closer look

Static review cannot validate the Docker attestation and bind-mounted Git runtime interactions, including channel precedence and image/build-arg behavior, without container-level execution.

No open issues 🎉

What changed in this PR

This pull request adds source-commit provenance to benchmark reports by resolving a validated SHA from the running checkout or build-time environment, propagating it through reports, and stamping Docker images with the corresponding commit.

Changes:

• Added validated Git, environment, and unresolved provenance channels with per-process caching and dirty-tree detection.
• Added git_sha_source and unconditional Git SHA display to reports, with setup-time provenance logging.
• Updated Docker publishing, development image configuration, documentation, and provenance tests.

File	Description
tests/​unit/​test_version.py	Adds coverage for SHA validation, checkout detection, dirty probing, channel precedence, caching, and edge cases.
tests/​unit/​metrics/​test_report_builder.py	Tests report provenance serialization and Git SHA display behavior.
tests/​unit/​commands/​test_benchmark.py	Verifies provenance is resolved and logged during benchmark setup.
src/​inference_endpoint/​utils/​version.py	Implements validated Git, environment, and fallback provenance resolution with dirty detection and process-level caching.
src/​inference_endpoint/​metrics/​report.py	Adds the required provenance source field and renders the Git SHA in reports.
src/​inference_endpoint/​commands/​benchmark/​execute.py	Resolves and logs source provenance before benchmark execution begins.
scripts/​push_docker_image.sh	Passes the source SHA and dirty marker as a Docker build argument.
scripts/​Dockerfile.dev	Exposes the build SHA through the runtime ENDPOINTS_GIT_SHA environment variable.
docs/​utils/​DESIGN.md	Documents provenance resolution behavior and integration points.
docs/​metrics/​report_design.md	Documents the expanded report provenance field.
docs/​metrics/​DESIGN.md	Updates the report schema documentation for source provenance.

_{Review effort: Lite}

[anandhu-eng]

LGTM