Skip to content

Proposal: a test's deadline branch should report what it observed, not diagnose a cause #968

Closed

Description

@santoshkumarradha

Proposal, not a defect report. Nothing here is failing today. It is about what our test failures SAY when they do fail, and it comes out of #967 — one helper whose five-second deadline reports no node ran when the truth is that the box was busy.

The shape

A test waits for something on a channel with a deadline, and the deadline branch asserts a cause the test did not observe:

select {
case node := <-ran:
	return node
case <-time.After(5 * time.Second):
	t.Fatal("no node ran")          // observed: nothing arrived in 5s
}                                   // asserted: the node never started

Those are different claims. "Nothing arrived within five seconds" is what happened; "the node never started" is a diagnosis, and on a loaded box it is the wrong one. The reader then spends their first hour in the task graph and the wall law rather than in the clock — which is exactly what #967 cost.

How widespread

internal/session's test files alone carry over 200 deadline branches (grep -rn 'case <-time.After(' internal/session/*_test.go). Sampling eighteen of their messages at random:

the node never started · the parked call was never handed its answer · the step never streamed · the node never reached the provider · closing the watch did not come back; its reading outlived it · a holder mid-reply was never told another window wants this conversation · the heartbeat never picked the answer up · holding the proposal did not stop its engine timer

Seventeen of the eighteen assert a cause. internal/tui3 and internal/remote carry the same shape at the same density.

This is the size of a convention, not a bug list, and the number must not be read as 229 defects. A ten-second wait that says "the node never started" is honest on a quiet box and only misleads when the deadline is tight relative to the work — which is what makes ranNodes.await's five seconds for divide-then-run the one that actually fails. Credit to lane F for both the sweep and for insisting on that distinction; the method is a grep of deadline branches minus the messages whose wording already reports a timeout, and it is crude enough that some of the matches are false positives.

What is proposed

A convention, written down once, plus mechanical help:

  1. A deadline branch says what it observed, then what it expected. waited 5s for a node to run; the graph had 3 nodes and none started rather than no node ran. The second clause is the diagnosis and it is only honest when the test actually checked it.
  2. Prefer waiting on the event to polling for a slot. Where a seam can be waited on, the deadline disappears rather than moving. Both of lane F's races on 2026-09-11 were this: a poll reporting 0 frames owed when the truth was the desk had not run yet.
  3. A deadline is never raised to fix a flake. Five seconds to thirty makes the same lie take six times as long to tell.

PERF.md:24 already states the principle for gates and its reasoning does not change for fixtures: "A stopwatch is a fact about the weather. A suite whose red means the box was busy is a suite people learn to re-run instead of read, and one flaky perf test costs more trust than the regression it was guarding against."

Scope, deliberately

#967 stays as it is — one helper, six tests, a reproducible red and an acceptance a stranger can run. This proposal is the convention, and #967's helper is its worked example. They are split because merging them would bury a real failing test under a wording change to two hundred call sites, and because there is no reason to change a message that has never misled anybody.

The honest cheap version of this is: write the convention into the tests' own guidance, fix #967 as the example, and let the remaining call sites come into line when somebody is in the file for another reason. A sweep of all of them is not obviously worth its review.


Drafted with CodeAF · reviewed and owned by the author

Activity

  1. santoshkumarradha commented on Sep 11, 2026

    @santoshkumarradha
    MemberAuthor

    Lane F, whose sweep this proposal came out of, endorses the cheap version and would review from it:

    Write the convention down, fix #967 as the worked example, let the rest come into line when somebody is in the file anyway. A mechanical sweep of two hundred messages would cost a reviewer real attention and buy nothing — none of them is red, and what changes the next one written is the convention existing, not the old ones being rewritten.

    Recording it here so that whoever picks this up does not read the two-hundred figure as a work item. It is the size of the convention. The deliverable is the convention plus one worked example; everything else is opportunistic.

  2. AbirAbbas commented on Sep 16, 2026

    @AbirAbbas
    Collaborator

    Closed by #976, which landed the convention and the one worked example this proposal settled on (#967's helper): a wait that runs out reports what it waited for and what it observed, and never claims the scheduler did not run.

    The wording and the run-budget-derived patience are pinned by TestAWaitThatRanOutSaysItTimedOutRatherThanNamingACause and TestTheWaitScalesWithTheRunsOwnTimeout in internal/session/awaitword_test.go, both green on dev at 77293538c. The remaining historical call sites come into line when somebody is in the file anyway, as the scope section here and Lane F's comment agreed.

  3. added
    featureWork that adds a capability; developers break it into tasks
    on Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureWork that adds a capability; developers break it into tasks

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions