[Alex-Wengg]

What

Vela3Manager runs vllm-sr/vela3-fast (Vela 3.0, UltraBERT-380M, Apache-2.0) from the new
FluidInference/vela-3.0-coreml snapshot:

• Decisions: choice or yes / no questions about a state, on the decision backbone (Vela3Decision.mlpackage, L128–L1024).
• Spans: the release's span_sets adapter (LoRA merged, Vela3SpanSets.mlpackage, L512 / L1024): 17 PII types, and the words of an answer its context does not support.

One encoder pass per question on the GPU. The Neural Engine is slower for this encoder at every length and drifts further from fp32 (15 / 1,200 decisions flipped), so it is not used.

How

• Vela3Tokenizer: byte-level BPE reproducing Hugging Face tokenizers offsets.
  • No offset trimming, because the post-processor is TemplateProcessing.
  • Pieces of one multi-byte character share its offsets.
  • lstrip / rstrip special tokens absorb the whitespace next to them.
  • Merges use a (rank, position) min-heap, so long pieces don't run in quadratic time.
• Decision rendering and readout follow vela3_encoder.DecisionEncoder.
• Span assembly, word units and the decoder reuse the Vela 2.0 Swift code. The release copied that Python verbatim; I diffed every helper and they are identical.
• Long span inputs follow the release's assemble / _windows within a 1,024-token pass:
  • the other part (an unsupported-claim context) is cut first;
  • then the target is read in overlapping windows, each window re-fitted;
  • per-word logits are averaged across windows.
• Thresholds come from the release's vela3_config.json: set-adapter temperature, PII thresholds by input length, and the unsupported-claim threshold.
• Vela2Heads.init(weights:span:) was added so both models share the readout. It validates tensor shapes, and Vela 2 values are unchanged.
• Vela3ModelStore is pinned to revision deaf348c with a SHA-256 for every file.

Checks

swift run -c release Vela3Check parity <model dir> <fixtures.json> against the Python runtime of the HF repo (vela3_coreml.py, same encoders, the release's own code). Fixtures come from conversion/fixtures.py in the HF repo. They cover emoji ZWJ sequences and flags, CJK, Arabic, Cyrillic, CRLF, URLs, <|mask|> and a 3,000-letter word; 2- and 9-window PII; and long contexts and answers.

tokens: 23 texts, 0 mismatches
decisions: 120 questions, 0 sequence mismatches, 0 answers differ, max |dp| 0.00000
spans: 16 calls (29 windows, 599 spans), 0 sequence mismatches, 0 span sets differ, max span |dp| 0.00000
median call: decision 9.7 ms, span 18.1 ms (GPU, end to end, M5 Pro)

The check now fails on probability drift above 1e-3, not only on changed answers. The Python runtime itself matches fp32 PyTorch: 0 / 1,200 typed decisions differ, and PII span sets are identical on 25 passages (HF reports/verify.txt).

Built in release mode: Vela3Check, Vela2Check, GuardrailDemo, and FrontDoorDemo (with the Xcode toolchain; the CommandLineTools toolchain lacks the SwiftUI macro plugin, and that predates this PR).

Core ML cold-start abort (fixed here)

When Core ML allocated the output, the first GPU prediction on a cold compile cache aborted inside MPSGraphTensorData (shape.count = 0 != strides.count = 3). This happened in about 1 of 3 cold runs, inside MLE5Engine, with either the sync or the async prediction API (macOS 27.0).

With a preallocated output backing and synchronous prediction it happened 0 of 20 times: 12 cold runs, plus 8 under a launcher that had crashed 3 of 3. Vela2Manager uses the same async, Core ML-allocated output pattern; it may be worth the same change in a follow-up.

Review

A /code-review pass found 15 issues. Fixed in 7792562:

• long-input handling for unsupported claims and PII;
• <|mask|> lstrip;
• quadratic BPE;
• silent span-temperature default;
• safetensors and hidden-shape bounds;
• pii_buckets validation;
• duplicate option names;
• Vela2 error type in Vela3 load;
• parity exit ignoring probability drift;
• per-call schema tokenization;
• redundant compile check.

Not changed: the model-store, encode and byte-table duplication, which follows the existing per-model pattern across FluidUse (a shared helper would be a repo-wide refactor), and the hard-coded function lengths (the snapshot is pinned).

🤖 Generated with Claude Code