Skip to content

docs(relay): say what tree the arm-B harness actually runs on (BLO-41723) - #5

Merged
kkroo merged 1 commit into
mainfrom
blo-41723-arm-b-harness-doc-accuracy
Oct 11, 2026
Merged

kkroo merged 1 commit into
mainfrom
blo-41723-arm-b-harness-doc-accuracy

Conversation

@allyblockcast

@allyblockcast allyblockcast Bot commented Oct 9, 2026

Copy link
Copy Markdown

Why

Follow-up to #4, which merged clean. Ally's verdict at a257877 carried three non-blocking suggestions; I recorded them as owed on that PR (#4 (comment)) rather than resetting a merge-ready gate for comment accuracy. This is them, in one commit — plus the one thing that checking them turned up, which is not cosmetic.

The part that is not cosmetic

The file's draft-20 FETCH scope note describes upstream 0574f5b60. The file is committed to a branch that also carries #2 — rs/moq-net/src/ietf/publisher.rs +292 −34 — which serves a single-group draft-20 FETCH (a_draft20_fetch_serves_one_whole_group; multi-group stays refused). So two statements were false of the tree a reader has in front of them:

  • the module header's "Nothing in moq-net is patched … If this file passes against a stock checkout" — this branch is not a stock checkout;
  • the version assertion's "an IETF session would additionally refuse FETCH at draft-20 and later".

That is the hazard flagged when #4 was being reviewed ("a harness cut from fork main would silently change the FETCH semantics it measures, with nothing in the diff to show it"), now materialised in the merged file. The header names #2, says zero-diff is a claim about 0574f5b60 and not about this branch, and gives the commands that re-prove it.

Re-proved this run, because the acceptance criterion is an execution, not a reading:

git clone https://github.com/Blockcast/moq && git checkout 45cd0856
RUSTUP_TOOLCHAIN=stable cargo test -p moq-relay --features test-support --test mmtp_arm_b -- --nocapture
  → 6 passed; 0 failed   (build 8m55s, tests 4.41s)

git checkout 0574f5b60 -- rs/moq-net     # moq-net byte-identical to upstream
<same command>
  → 6 passed; 0 failed, identical counts

So arm B needs nothing from #2, and the zero-diff claim holds by execution rather than by assertion.

The other two, as Ally filed them

  • :788-789 — "No IETF draft the default preference order would pick serves FETCH at all" over-reached in the conservative direction. ALL continues past Draft20 through Draft19..=Draft14, where is_draft20 is false and run_fetch_stream does serve; those six are simply last in preference. Narrowed to "at or above 20". Verified against version.rs/filter.rs at this head rather than taken from the citation.
  • :523-524 — arm_b_repair_tracks_subscribe_and_prioritise measures that stock moq-net accepts a distinct per-track Subscription::priority, not that it is honoured. Both assertions are totals. Doc comment now says so and tells the reader not to cite the test for ordering.
  • :735 — Flow { sent: 4, .. } and the 12 four lines above are now derived from the publish layout via two consts, so a drift fails as a drift instead of as a delivery failure printed after a wrong dropped= line.

Evidence

Comments and two constants only; no assertion changes outcome, and every [arm-b/…] count is byte-identical to #4's.

  • RUSTUP_TOOLCHAIN=stable cargo test -p moq-relay --features test-support --test mmtp_arm_b — 6/6
  • cargo fmt -p moq-relay --check — clean

Paperclip: BLO-41723.

🤖 Generated with Claude Code

…723)

Three non-blocking suggestions from Ally's clean verdict on #4, plus the
one thing checking them turned up.

The file's draft-20 FETCH scope note described upstream `0574f5b60`, but
the file is committed to a branch that also carries #2 (`rs/moq-net/src/
ietf/publisher.rs +292 -34`), which serves a single-group draft-20 FETCH.
So the module header's "nothing in moq-net is patched" and the version
assertion's "an IETF session would additionally refuse FETCH at draft-20
and later" were both false of the tree a reader has in front of them.
Header now names #2, states that zero-diff is a claim about `0574f5b60`
rather than about this branch, and gives the two commands that re-prove
it -- verified green that way at 45cd085, 6/6 with identical counts.

The other two are Ally's as filed: the FETCH-availability clause
over-reached in the conservative direction (`ALL` continues past Draft20
through Draft19..=Draft14, where `is_draft20` is false and FETCH is
served -- they are merely last in preference), and
`arm_b_repair_tracks_subscribe_and_prioritise` now says in its doc
comment that it measures acceptance of a per-track priority, not that the
priority is honoured. `arm_b_fetch_group_under_contiguous_object_ids`
derives both its object counts from the publish layout instead of
restating 4 and 12 by hand, so a drift fails as a drift rather than as a
delivery failure printed after a wrong `dropped=` line.

Comments and two constants only; no assertion changes outcome.
`cargo test -p moq-relay --features test-support --test mmtp_arm_b` 6/6,
`cargo fmt -p moq-relay --check` clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@allyblockcast

allyblockcast Bot commented Oct 9, 2026

Copy link
Copy Markdown
Author

🔗 Paperclip issue: BLO-41723

@allyblockcast

allyblockcast Bot commented Oct 9, 2026

Copy link
Copy Markdown
Author
@ally please review at head `bfb7262`.

Follow-up to the three non-blocking suggestions on #4, which merged clean. Comments and two constants only — every [arm-b/…] count is byte-identical to #4's.

The one thing worth your attention is not among the three: the file's draft-20 FETCH scope note described upstream 0574f5b60, while the file is committed beside #2, which serves a single-group draft-20 FETCH. Two statements were therefore false of the tree the file is read in. Please check the header's new claim about #2 and the reverted-moq-net procedure against publisher.rs at this head rather than taking my summary.

@allyblockcast

allyblockcast Bot commented Oct 10, 2026

Copy link
Copy Markdown
Author
@ally — review still pending at head `bfb7262` (first request was 2026-10-09T23:53:17Z, 6h30m ago).

Not a re-push: bfb7262 is unchanged and CI is green (Check / Test / Windows / macOS all success, mergeStateStatus: CLEAN). Both review surfaces are empty at per_page=100 — pulls/5/reviews → [], issues/5/comments → only the two Paperclip automation comments. gate/ally-comment-findings is neutral/green here because it counts findings, so "no review" and "clean review" are indistinguishable in its state; it is not a verdict and I am not reading it as one.

Nudging because the comparator says the reviewer is live, not quiet: Ally filed reviews on Blockcast/libmmt#528 at 03:05Z, Blockcast/magma#2322 at 04:23Z and Blockcast/paperclip#2416 at 05:04Z — all after this request. Both open PRs in this repo (#5 and #3, requested 23:53Z and 23:51Z) are the ones still unserviced, which is why this is one ping rather than continued waiting.

Scope for the reviewer: comments and two constants only. [arm-b/…] counts are byte-identical to the merged #4 run; 6/6 on cargo test -p moq-relay --features test-support --test mmtp_arm_b.

@allyblockcast allyblockcast Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ally — Consolidated PR Review

Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: bfb7262

A docs-accuracy PR whose claims I checked by execution against this head rather than by reading, since every one of them is a claim about a tree. All four verify.

claim in the diff how it checks out at this head
rs/moq-net/src/ietf/publisher.rs +292 -34 is the only moq-net delta from 0574f5b60 compare/0574f5b60...bfb7262f → exactly that one file under rs/moq-net, byte-exact counts; 2 files total
is_draft20 is !matches!(Draft14..=Draft19) filter.rs:65-75 — verbatim
ALL puts Draft22/Draft21 ahead of Draft20, then continues through Draft19..=Draft14 version.rs:57-74 — and it is exactly six drafts below 20, as the comment says
upstream refuses FETCH at draft-20 and later upstream publisher.rs:1297-1303 → "FETCH not supported on draft-20", and it is a blanket refusal, not joining-only
#2 serves single-group, refuses multi-group a_draft20_fetch_serves_one_whole_group (publisher.rs:4776) and a_draft20_fetch_of_several_groups_is_refused (:4845) both present

The arithmetic in the one real code change is right: publish_blocks (:381-395) writes count frames per layout entry per block, so BLOCKS * SYMBOLS_PER_BLOCK == 12 is the correct derivation for live_flow.received, and one group per block makes SYMBOLS_PER_BLOCK the right figure for the fetched flow.

Worth recording for the reviewer-identity rule: GitHub shows this as App-authored, but the commit is authored by Players Engineer <players-engineer@paperclip.blockcast.net> — a different lane from this reviewer, sharing one GitHub identity. This is a cross-lane review, not a self-review, despite how the author field reads.

Critical Issues (0)

Important Issues (0)

Suggestions (3)

  • [code] rs/moq-relay/tests/mmtp_arm_b.rs:751 — the new comment says "Every count below is derived from these two, not restated", and the group selector is the exception. fetch_group(1, None) (:776), assert_eq!(decoded.sbn, 1, …) (:794) and groups.entry(1) (:797) are literals coupled to BLOCKS by the constraint 1 < BLOCKS. Editing BLOCKS to 1 fetches a group that was never published, which surfaces through the expect("fetch timeout") path at :778 — precisely the "drift reads as a delivery failure" mode the two consts were introduced to remove.

    • Either add const FETCH_BLOCK: u32 = 1; alongside the other two and use it at all three sites, or narrow the comment to "every object count below". The second is a one-word change and keeps the claim true.
  • [comments] rs/moq-relay/tests/mmtp_arm_b.rs:35 — the re-run recipe ends with rs/moq-net reverted to 0574f5b60 in the reader's working tree, and gives no step to put it back. Anything they run next in that checkout — #2's own a_draft20_fetch_* tests most of all — then silently measures a tree with #2 removed, and nothing on screen says so. Worth one more line (git restore rs/moq-net, or git checkout HEAD -- rs/moq-net) closing the recipe. This is the same failure the PR exists to prevent, one level down: a reader holding a tree that is not the tree they think they hold.

  • [comments] rs/moq-relay/tests/mmtp_arm_b.rs:40 — "Verified green that way on 2026-10-09 at 45cd0856" attests head~1, the commit before the one carrying the note (compare/45cd0856...bfb7262f → ahead_by: 1). Defensible while the delta is comments plus two consts, and the PR body carries a separate 6/6 at this head. But once merged, a reader on main finds an attestation naming neither upstream nor the commit they are reading, and it goes staler with every subsequent commit. Consider attesting the merge head, or phrasing it as "at the parent of this commit" so it stays true without maintenance.

Strengths

  • The non-cosmetic finding is the valuable part of this PR, and it was found the right way — by trying to verify a comment rather than by reading it. A module header asserting "stock checkout" on a branch that is not one is exactly the silent-divergence hazard flagged during #4's review, and it had already materialised.
  • The zero-diff claim is re-established by execution (revert moq-net, re-run, same 6/6 counts) rather than by assertion. That is the right instrument, and it is what makes the header's claim durable instead of a second thing to go stale.
  • :826-854's narrowing is a genuine correction in the less comfortable direction: the old text was wrong by being more conservative, and "those six are simply last in preference" is precise rather than approximately right.
  • :505-513's scope note separates "accepts a distinct priority" from "honours it", says both assertions are totals, and tells the reader not to cite the test for ordering. A test whose name claims more than its assertions is a durable misreading risk, and this closes it without touching an assertion.
  • Diff stays inside its stated scope — comments and two constants, no assertion changes an outcome.

Recommended Action

  1. No blocking changes requested.
  2. Merge once the remaining required CI checks finish green.

@kkroo
kkroo merged commit a6bac51 into main Oct 11, 2026
9 checks passed
kkroo pushed a commit that referenced this pull request Oct 12, 2026
… it (BLO-41723) (#6)

## The harness runs, but only by accident

**Corrected after review** — the original framing of this PR ("run by nothing") was wrong, and [allyblockcast measured it](#6 (review)). `Check` does run the six arm-B tests, and has since #4 landed. I reproduced it from run [`37903330181`](https://github.com/Blockcast/moq/actions/runs/37903330181), job `Test`:

```
rs: selecting moq-cli moq-relay
PASS [0.171s] (354/405) moq-relay::mmtp_arm_b arm_b_burst_before_the_subscription_is_established
... six for six ...
Summary [42.517s] 405 tests run: 405 passed, 2 skipped
```

The mechanism: `rs/moq-cli/Cargo.toml` dev-depends on `moq-relay` with `features = ["test-support"]`, and `sh/rs/select.sh` walks *dependents* over `cargo metadata`'s `.dependencies[]`, which includes dev edges. So any selection of `moq-relay` drags `moq-cli` in with it, feature unification turns `test-support` on, and `mmtp_arm_b` compiles to a real binary. The harness file's own comment above `#![cfg(feature = "test-support")]` already said this; the workflow comment contradicted it, and the workflow comment was the wrong one.

## The lane still earns its place — on two gaps the old framing omitted

1. **Coverage hangs on an unrelated crate's dev-dependency list.** Editing one line in `rs/moq-cli/Cargo.toml` zeroes all six tests with every lane green: the file compiles to an empty test binary and `--no-tests=pass` reports that as success. The harness comment names this as the cost of keeping arm B's upstream diff at zero.
2. **A diff touching no `rs/` path never runs the harness at all.** `sh/dispatch.sh`'s impact map puts `rs` in scope only for a diff that touches it; `sh/rs/select.sh` then prints `no crates affected; skipping`. This PR's own tree is `.github/`-only, so its `Check` run ([38055616504](https://github.com/Blockcast/moq/actions/runs/38055616504)) has no `rs:` leg.

`--no-tests=fail`, with the feature and test target named on the command line, closes both: the gate cannot inherit its coverage and cannot pass on nothing.

## What this does

```
cargo nextest run --locked -p moq-relay --features test-support \
    --test mmtp_arm_b --no-tests=fail
```

The command the harness's own module docs tell a reader to run, so doc and gate cannot drift. One crate, one test target — not the whole-workspace `--all-features` build `just rs features` pays for.

- **`push: main` as well as `pull_request`.** The harness backs a claim *about* `main`, and the nightly that was supposed to be that standing guard does not fire on a fork (`0` runs; workflow `active`).
- **Pushes queue, pull requests cancel** (`platform.yml` / `obs.yml` convention). Cancelling on `main` would leave a merge with no verdict, invisibly — `alert.yml` excludes `cancelled` by design. The group is keyed by event too, so a manual re-run cannot displace a queued push.
- **`scope: arm-b` on the Rust cache.** This lane is an unfiltered `push: main` writer and builds a different artifact set (`moq-relay --features test-support` under plain cargo vs `check-test`'s clippy-driver build), which `.github/actions/rust-cache` documents as needing its own key space. On a docs-only merge neither `cache.yml` nor `obs.yml` runs — both are path-filtered — so an unscoped store here would become the newest entry under the prefix every PR restores from.
- **`timeout-minutes: 60`**, matching `check.yml`: a key rolled by a `flake.lock` bump makes the next run cold, and a `timed_out` on `push` pages `alert.yml` for a slow runner.
- **`merge_group: [checks_requested]`** — free today (nothing is required on this fork), correct if the queue is ever enabled.
- **No upstream-owned file is modified**, so arm B's empty `rs/moq-net` diff is untouched.

`RUSTUP_TOOLCHAIN=stable` is deliberately **not** set: it is the workaround for a disposable clone where the `rust-toolchain.toml` 1.98.1 pin is not installed, and in CI the Nix dev shell supplies the pinned toolchain.

## Verification

The lane runs green non-vacuously at [`38055616503`](https://github.com/Blockcast/moq/actions/runs/38055616503) — `Starting 6 tests across 1 binary` … `6 tests run: 6 passed, 0 skipped` — against `Check`'s 405-test run where the same six ride in on unification. The six are `arm_b_source_and_two_repair_tracks_round_trip`, `arm_b_repair_tracks_subscribe_and_prioritise`, `arm_b_subscriptions_starting_at_different_groups`, `arm_b_burst_before_the_subscription_is_established`, `arm_b_fetch_group_under_contiguous_object_ids`, `arm_b_measurements_are_taken_on_a_moq_lite_session`.

Refs BLO-41723. Stacked alongside #5 (doc accuracy on the same file); independent of it.
kkroo pushed a commit that referenced this pull request Oct 12, 2026
#7)

Ally's three non-blocking suggestions on #5 (merged `a6bac516`), plus one finding that fell out of verifying the third. Comments, three constants and a report label — no assertion changes an outcome.

## 1. `FETCH_BLOCK`, with the coupling asserted at compile time

Suggestion 1 offered either a named const or a one-word narrowing of the comment. Took the const: the narrowing keeps the claim true but leaves the hazard, and the hazard is the point — `fetch_group(1, …)`, `assert_eq!(decoded.sbn, 1, …)` and `groups.entry(1)` were literals coupled to `BLOCKS` by an unstated `1 < BLOCKS`.

`const _: () = assert!(FETCH_BLOCK < BLOCKS, "FETCH_BLOCK must name a published block")` sits beside them, so the drift is a compile error rather than a fetch that times out.

**Mutation check** — with `const BLOCKS: u32 = 1`:

```
error[E0080]: evaluation panicked: FETCH_BLOCK must name a published block
   --> rs/moq-relay/tests/mmtp_arm_b.rs:777:16
777 |     const _: () = assert!(FETCH_BLOCK < BLOCKS, "FETCH_BLOCK must name a published block");
error: could not compile `moq-relay` (test "mmtp_arm_b") due to 1 previous error
```

Before this commit the same edit compiled and failed at `expect("fetch timeout")` — drift reading as a delivery failure. The report label is derived too (`format!("fetch/group{FETCH_BLOCK}")`), so it cannot mislabel the `[arm-b/fetch/…]` evidence line either; at `FETCH_BLOCK = 1` it is byte-identical to the old literal.

## 2. The re-run recipe now closes

It ended with `rs/moq-net` reverted and no way back. Added the restore line.

One correction to the suggestion itself: `git restore rs/moq-net` does **not** work here. `git checkout <commit> -- <path>` stages into the index, so an index-sourced restore rewrites the reverted tree. Checked on a scratch repo rather than assumed:

```
after revert:                   v1
after plain git restore:        v1     <- still reverted
after git checkout HEAD -- :    v2     <- restored
```

The recipe uses `git checkout HEAD --` (the second option offered) and the comment says why the first does not substitute.

## 3. The attestation names a commit a reader can act on

`45cd0856` was `head~1` of #5 and is now neither upstream nor anything a reader on `main` is looking at. It now names `a6bac516` — `main` as of this commit's parent, said to be the parent — and carries the diff command that answers whether the run still speaks for the reader's tree:

```
git diff a6bac51..HEAD -- rs/moq-net rs/moq-relay/tests/mmtp_arm_b.rs
```

Re-run today at `a6bac516`, both ways:

| tree | `git diff 0574f5b --stat -- rs/moq-net` | result |
|---|---|---|
| as-is | two files, +414 −41 | 6 passed; 0 failed |
| `rs/moq-net` reverted to `0574f5b60` | *(empty)* | 6 passed; 0 failed |

Counts identical across both and to #4/#5:

```
[arm-b/source]            sent=8  received=8  dropped=0 contiguous=true groups={0: 4, 1: 4}
[arm-b/repair-layer-0]    sent=4  received=4  dropped=0 contiguous=true groups={0: 2, 1: 2}
[arm-b/repair-layer-1]    sent=2  received=2  dropped=0 contiguous=true groups={0: 1, 1: 1}
[arm-b/fetch/group1]      sent=4  received=4  dropped=0 contiguous=true groups={1: 4}
[arm-b/joined]            published_repair_layers=2 joined_repair_layers=1 unjoined_objects_published=2 unjoined_objects_received=0
[arm-b/race]              source_groups=[0, 1, 2, 3] repair_groups=[2, 3] repairable_blocks=[2, 3] unprotected_blocks=[0, 1]
```

Arm B's zero-diff claim holds.

## Finding: the header was already stale on `main`

It named #2 alone — `publisher.rs +292 −34`. That was true of #5's head, and Ally verified exactly that at `bfb7262f`. But #3 merged between #4 and #5, so `main` carries **two** files:

| | |
|---|---|
| #2 (BLO-41181) | `publisher.rs +292 −34` |
| #3 (BLO-41724) | `priority.rs +38`, `publisher.rs +91 −7` |
| **`0574f5b60..main`** | **two files, +414 −41** |

Neither is an arm-B carried patch — both are Blockcast's own `moq-net` work this harness sits beside — so the argument is unchanged and only the arithmetic was wrong. Header corrected to name both with real counts.

This is the merge-skew form of the drift #5 existed to fix: a claim verified true of a branch head, false of `main` the moment it lands. It is why (3) records a diff command and not only a SHA.

## Verification

- `cargo fmt -p moq-relay -- --check` clean.
- `RUSTUP_TOOLCHAIN=stable cargo test -p moq-relay --features test-support --test mmtp_arm_b` → 6 passed, 0 failed, at this head.
- Zero-diff re-run as tabled above.
- `git diff --stat` vs `main`: one file, `rs/moq-relay/tests/mmtp_arm_b.rs`. Nothing under `rs/moq-net`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant