Skip to content

feat: add AFD prefill performance simulator - #269

Merged
jiangkuaixue123 merged 1 commit into
vllm-project:simulatorfrom
jiangkuaixue123:codex/add-prefill-performance-simulator
Aug 26, 2026
Merged

feat: add AFD prefill performance simulator#269
jiangkuaixue123 merged 1 commit into
vllm-project:simulatorfrom
jiangkuaixue123:codex/add-prefill-performance-simulator

Conversation

@jiangkuaixue123

@jiangkuaixue123 jiangkuaixue123 commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Purpose

Add a Python prefill performance simulator for comparing asynchronous Attention/FFN disaggregation against the merged DP4/TP4/EP16 architecture using normalized msModeling analytic profiles.

Issue

  • Related issue(s): N/A

Scope

  • In scope: fixed and continuous workloads, CSV replay, Prefix Cache, chunked prefill, round-robin and vLLM queue-aware DP scheduling, AFD CAM pipeline modeling, merged barriers and global EP phases, QPS sweeps, TTFT/SLO metrics, HTTP API, and an interactive timeline UI.
  • Out of scope: decode modeling, online serving integration, automatic msModeling installation, and runtime hardware execution.

Implementation Notes

  • Keeps config, workload generation, profile lookup/building, simulation engine, HTTP server, and web presentation in separate modules.
  • Reuses one Python simulation core from CLI and HTTP endpoints; the frontend is presentation and configuration only.
  • Builds AFD Attention and single-job FFN profiles from separate msModeling topologies, and records phase provenance in normalized profiles.
  • Models merged Attention per real DP token count, waits at the DP barrier, then uses phase-specific global/DP-max profile queries for EP phases.
  • Includes CSV timestamp/length inputs, deterministic Prefix Cache sampling, chunked prefill, CAM send/recv parameters, expert sampled Shapes, and interactive timeline zoom/pan/tooltips.
  • Generated profile JSON and raw traces remain ignored because they are device- and msModeling-revision-specific; README documents the generation command.

Test Plan

CPU-only:

  • .venv/bin/python -m pre_commit run
  • .venv/bin/python -m unittest discover -s simulator/tests -v
  • .venv/bin/python -m compileall -q simulator
  • JavaScript syntax check for the inline web UI
  • CLI simulate/sweep and HTTP defaults/page/simulate smoke tests with a generated DSV4 Flash profile

GPU/NPU-gated:

  • Not applicable. Runtime simulation is CPU-only; operator latency data is produced offline by msModeling.

Test Result

  • Pre-commit: passed
  • Unit tests: 27/27 passed
  • Ruff and diff checks: passed
  • CLI simulate/sweep: passed
  • HTTP API/page smoke: passed
  • Frontend JavaScript syntax: passed
  • Independent test subagent: PASS
  • Independent architecture review: APPROVE

Docs Impact

  • Files updated: simulator/README.md, simulator/profiles/README.md, examples, and complete configuration-field documentation.
Essential PR Checklist
  • Purpose is clear and linked to public context when possible.
  • Scope is bounded.
  • Compatibility with vLLM v0.26.0 is considered; the simulator is standalone and does not import vLLM runtime modules.
  • No changes are made to the vLLM source checkout.
  • Plugin-owned modules are used; no monkey patches are introduced.
  • No compatibility shim or monkey patch is added.
  • Imports remain CPU-safe; profile generation is an explicit offline operation.
  • Validation evidence is included; GPU validation is not applicable.
  • Documentation impact is stated.

@jiangkuaixue123
feat: add AFD prefill performance simulator
df8033d 
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
@jiangkuaixue123
jiangkuaixue123 force-pushed the codex/add-prefill-performance-simulator branch from eae9e5c to df8033d Compare August 25, 2026 12:03
@jiangkuaixue123
jiangkuaixue123 changed the base branch from main to simulator August 26, 2026 01:28
@jiangkuaixue123
jiangkuaixue123 merged commit 78b1480 into vllm-project:simulator Aug 26, 2026
2 checks passed
@ShwStone ShwStone mentioned this pull request Aug 26, 2026
Merged
9 tasks
jiangkuaixue123 pushed a commit that referenced this pull request Aug 26, 2026
@ShwStone
feat(simulator): expand topology and workload exploration (#272)
1d4338b 
## Purpose

Extend the experimental DeepSeek-V4-Flash prefill simulator introduced
in #269 so AFD and merged deployments can be compared across equal-die
topologies, independent batching policies, chunked FIFO waves, and
burst-preserving production-like length traces.

This also checks in the normalized analytic profiles and prompt-free
MoonConv arrival/length fixtures so a fresh clone can use the WebUI
without installing msModeling or downloading prompt token IDs.

## Issue

- Related issue(s): follow-up to #269
- Closing keyword: none

## Scope

- In scope:
- Simulator-only scheduling, workload, profile, HTTP, and WebUI changes.
- Independent AFD and merged max-batch/chunk controls and DPLB policies.
  - FIFO wave estimators using token sum or token-square work.
  - Equal-die dynamic topology selection and profile composition/reuse.
  - Canvas/LOD timeline rendering for large event sets.
  - QPS-scaled MoonConv V4 traces that preserve zero-gap bursts.
  - Checked-in 64K analytic profiles for 16/24/32/40/48-die comparisons.
- Out of scope:
  - Production AFD runtime behavior changes.
  - Decode, KV-transfer, capacity/eviction, EPLB, or real expert skew.
  - GPU/NPU execution or msModeling calibration against hardware.

## Implementation Notes

- The experimental scheduler configuration is intentionally breaking:
the shared policy and batch-token fields are replaced by
architecture-specific AFD and merged settings.
- Current-runtime policy models AFD as round-robin on the present async
engine path and merged as vLLM queue-aware DPLB.
- Chunked requests retain FIFO order; unstarted waves are not
arbitrarily reordered.
- AFD DP replication reuses per-DP msModeling points when TP and FFN EP
are unchanged. Merged TP/global-EP changes use independently generated
profiles.
- Every comparison validates equal physical die budgets from profile
topology metadata.
- scaled_trace preserves all relative intervals and simultaneous bursts,
rescales the complete cycle to the configured mean QPS, and repeats the
cycle across long warmup/measurement windows.
- MoonConv fixtures contain only arrival_time_ms and input_length.
Source revision, hashes, ODC-By attribution, and exclusions are
documented under simulator/data/README.md.
- No vLLM checkout, compatibility shim, monkey patch, or plugin runtime
class is changed.

## Test Plan

- python -m ruff check simulator
- python -m unittest discover -s simulator/tests -v
- Load every checked-in profile and validate its AFD/merged die budget.
- Start the HTTP server with all profiles and smoke each newly added
topology.
- Replay MoonConv formal_0 at target 4 QPS with 32K chunking.

## Test Result

- Ruff: passed.
- Simulator unit tests: 47 passed.
- Profiles: all 9 checked-in JSON bundles loaded; all equal-die checks
passed.
- Dynamic topology API: 6 AFD choices and 8 merged choices returned; all
three newly requested pairings completed simulation smoke tests.
- MoonConv 4-QPS smoke: 284 generated arrivals, including 255 preserved
zero-gap arrivals; 249 requests landed in the 60-second measured window.
Finite-window offered QPS was 4.15 because burst boundaries are
preserved; each complete repeated cycle averages exactly 4 QPS.
- GPU/NPU E2E: not run because this PR changes only the dependency-free
analytic simulator.

## Docs Impact

- Files updated: simulator/README.md, simulator/profiles/README.md, and
simulator/data/README.md.
- The docs cover topology budgets, device/die terminology, scheduler
semantics, scaled trace behavior, profile generation/reuse, dataset
provenance, and WebUI operation.

---

<details>
<summary>Essential PR Checklist</summary>

- [x] Purpose is clear and linked to public context when possible.
- [x] Scope is bounded.
- [x] Compatibility with vLLM v0.26.0 is considered.
- [x] No changes are made to the vLLM source checkout.
- [x] Plugin-owned classes or explicit dotted class paths are preferred
over monkey patches; this PR changes no runtime classes.
- [x] No compat shim or monkey patch is added.
- [x] Imports remain CPU-safe; the simulator uses the standard library
at runtime.
- [x] Validation evidence is included, including the GPU/NPU skip
reason.
- [x] Documentation impact is stated.

</details>

---------

Signed-off-by: ShwStone <HaowenShi@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hsliuustc0106 hsliuustc0106 Awaiting requested review from hsliuustc0106 hsliuustc0106 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jiangkuaixue123