Skip to content

CI: add DSpark 16-card serving accuracy guard - #1144

Merged
zhangqi-chen merged 1 commit into
hw-native-sys:mainfrom
ndleslx:ci/add-dspark-16-card-serving-accuracy
Sep 11, 2026
Merged

zhangqi-chen merged 1 commit into
hw-native-sys:mainfrom
ndleslx:ci/add-dspark-16-card-serving-accuracy

Conversation

@ndleslx

@ndleslx ndleslx commented Sep 7, 2026

Copy link
Copy Markdown
Contributor
  • Add a separate serving-dspark PR job for changes under
    models/deepseek_v4_flash_dspark, with its own build and pip caches.
  • Run the Palace 64-prompt/128-completion HTTP guard from
    pypto-serving/main on 16 cards against the pypto-lib checkout under
    test. Print completions for review and check the response contract
    and token accounting with default Simpler timeouts.
  • Resolve PYPTO_DSV4_DSPARK_MODEL_DIR from runner configuration,
    validate the weights directory, and forward it through task-submit.
  • Bound queue and execution time and retain orphan-task and build
    artifact cleanup for the new serving job.

@coderabbitai

coderabbitai Bot commented Sep 7, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 8f476527-eea5-4b8e-b68b-8c4366619ca8

📥 Commits

Reviewing files that changed from the base of the PR and between aff70c1 and 2494855.

📒 Files selected for processing (3)
  • .github/actions/pypto-serving-tests/action.yml
  • .github/actions/setup-ci-job/action.yml
  • .github/workflows/ci.yml

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The CI detects changes to the DeepSeek V4 DSpark model and runs a dedicated serving accuracy job. The serving action resolves the DSpark model directory and executes tests on 16 devices.

Changes

DeepSeek DSpark serving tests

Layer / File(s) Summary
DSpark serving action
.github/actions/pypto-serving-tests/action.yml, .github/actions/setup-ci-job/action.yml
The serving action accepts deepseek-v4-dspark, resolves PYPTO_DSV4_DSPARK_MODEL_DIR, and runs the DSpark accuracy test on 16 devices. Runner configuration documentation includes the new environment variable.
Workflow detection and dispatch
.github/workflows/ci.yml
Change detection enables serving-dspark for DSpark model changes. The job configures the runner, caches, timeout, pinned reference, and DSpark serving action invocation.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PullRequest
  participant detect-changes
  participant serving-dspark
  participant pypto-serving-tests
  participant task-submit
  PullRequest->>detect-changes: change models/deepseek_v4_flash_dspark/
  detect-changes->>serving-dspark: set run_dspark_serving_tests=true
  serving-dspark->>pypto-serving-tests: invoke deepseek-v4-dspark tests
  pypto-serving-tests->>task-submit: run accuracy test on 16 devices
  task-submit-->>pypto-serving-tests: return test result
Loading

Merge Risk: ⚪ Minimal · up to 24948

No confirmed merge-blocking risk remains in this CI-only change.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: adding a CI serving accuracy guard for DSpark on 16 cards.
Description check ✅ Passed The description directly matches the changeset. It covers the dedicated serving job, DSpark model configuration, 16-card accuracy testing, time limits, and cleanup.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks the model trail
DSpark tests hop through the rail
Sixteen devices start to run
Logs stay quiet, work gets done
CI marks the finish line done

Comment @coderabbitai help to get the list of available commands.

@ndleslx

ndleslx commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

This draft depends on hw-native-sys/pypto-serving#213 landing first: the reusable action checks out pypto-serving/main, and that PR supplies tests/test_deepseek_dspark_accuracy.py and the DSpark serving integration.

The selected self-hosted runner needs 16 available cards and PYPTO_DSV4_DSPARK_MODEL_DIR set in its service environment to the DSpark W8A8 checkpoint directory.

Validation: all pre-commit hooks passed. Local checks parsed the YAML, validated embedded shell syntax, exercised DSpark versus unrelated change selection and missing-model handling, and captured the task-submit payload to verify the 16-card request, model path, PYTHONPATH, pytest target, and absence of Simpler timeout overrides.

The Palace 64/128 guard previously passed on the serving branch with pypto-lib 57e9d6a and default Simpler timeouts (2 passed in 500.94s). The new CI job has not been run end to end against this PR's pypto-lib base. As with the existing model-specific serving jobs, CI-only edits do not trigger the new device job.

Related: #1133

- Add a separate serving-dspark PR job for changes under
  models/deepseek_v4_flash_dspark, with its own build and pip caches.
- Run the Palace 64-prompt/128-completion HTTP guard from
  pypto-serving/main on 16 cards against the pypto-lib checkout under
  test. Print completions for review and check the response contract
  and token accounting.
- Pin the guard to PyPTO 747e1e4ba99f13dea1a609d893ac7512e85a5897.
- Resolve PYPTO_DSV4_DSPARK_MODEL_DIR from runner configuration,
  validate the weights directory, and forward it through task-submit.
- Allow 4800 seconds for device scheduling and 7200 seconds for test
  execution, with job-level headroom and cleanup for orphaned tasks and
  build artifacts.
@ndleslx
ndleslx force-pushed the ci/add-dspark-16-card-serving-accuracy branch from 831ac9d to 2494855 Compare September 11, 2026 01:56
@ndleslx
ndleslx marked this pull request as ready for review September 11, 2026 02:01
@zhangqi-chen
zhangqi-chen merged commit 14d6e1b into hw-native-sys:main Sep 11, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants