Skip to content

Docs: RNGD end to end and real models served from onggi volumes (#54, #55) - #65

Merged
gspark-etri merged 1 commit into
mainfrom
docs/rngd-and-engines
Oct 22, 2026
Merged

gspark-etri merged 1 commit into
mainfrom
docs/rngd-and-engines

Conversation

@gspark-etri

Copy link
Copy Markdown
Contributor

Background

ATOM was verified end to end, but RNGD had only storage numbers (#54), and no engine had served a model from an onggi volume (#55).

Changes

  • docs/evaluation.md:
    • a summary matrix of the three levels (store less, node delta, NPU delta) and real-model serving, for ATOM, RNGD and the A100;
    • an RNGD section: four furiosa-llm artifacts, onggi diff --ranges, CSI cold and delta apply;
    • a section on serving real models from onggi volumes on ATOM and RNGD: tokens and pod-to-first-token times.

Verification

Cluster runs on npu-node-1 in a test namespace, deleted afterwards. Each engine Job held one card and ran to completion.

  • RNGD storage: 32.44 GB → 10.79 GB. pfbatch → preset: 99.77 % at the same offset, 4.3 MB new. CSI preset by delta apply: Running in 4 s, 16.15 GB in place; every file identical.
  • ATOM real model: Llama-3.1-8B from an onggi volume gives 48 greedy tokens identical to the original directory, cold, warm and after delta apply. First token 28.4 s via delta apply, 22.5 s from the original directory.
  • RNGD real model: furiosa-llm 2026.3.0 gives tokens identical in every run. First token 21.8 s via delta apply (0 B from the store), 18.9 s from the original directory.
  • Plugin fixes: the two problems the run found are fixed and merged (CSI: an exiting plugin removes the live plugin's socket #61 socket race, CSI: files of mode 0600 in an artifact are unreadable for non-root pods #62 file modes).

Compatibility and limits

  • Docs only.
  • A cold ATOM pull (11.58 GB from the store) puts the first token at 191 s.
  • One RNGD cold run waited 248.8 s for a card that another host job held. That wait is reported separately.

Related Issue

Closes #54
Closes #55

🤖 Generated with Claude Code

…ary matrix (#54, #55)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gspark-etri
gspark-etri merged commit daf9884 into main Oct 22, 2026
3 checks passed
@gspark-etri
gspark-etri deleted the docs/rngd-and-engines branch October 22, 2026 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant