Skip to content

test: the contract tests sometimes fail at the end: the trial broker did not stop cleanly #214

Closed

Description

@groupsky

What happened

The contract tests (#209) passed, and then the package failed when the trial broker was stopped. The CI job test (go.mod) failed this way twice on 2026-11-18:

contract: the trial broker did not stop cleanly: exit status 1
FAIL	github.com/code6x/nodkey/test/contract

No test in the package failed (no --- FAIL line). The error comes from runTests in test/contract/harness_linux_test.go, after m.Run(): tr.stop() closes the stdin of nodkey try, sends SIGTERM, and nodkey try ends with exit status 1. In both runs test (stable) passed, and only test (go.mod) failed.

The harness keeps what nodkey try wrote (tr.out: its stdout lines and its stderr), but it does not print it when the stop fails. So the log does not say why nodkey try ended with 1.

Possible causes, not checked yet:

  • nodkey try sees the closed stdin (EOF) before the SIGTERM, and ends with an error;
  • a race in the shutdown of nodkey try after SIGTERM (cmd/nodkey/try.go, signal.NotifyContext);
  • the test expects exit status 0 after SIGTERM, but nodkey try sometimes reports the signal as an error.

Done when

  • When the stop of the trial broker fails, the harness prints what nodkey try wrote (tr.output()), so a later failure says why.
  • The cause is found and written in the pull request: the harness, or nodkey try itself.
  • If nodkey try is wrong (for example a clean SIGTERM gives exit status 1), it is fixed in cmd/nodkey, and a test in cmd/nodkey covers it.
  • The package passes 20 separate runs in a row on a CI runner, with the Go of go.mod and with Go stable. Separate runs, for example for i in $(seq 20); do go test -race -count=1 ./test/contract/ || exit 1; done, because -count=20 starts and stops the trial broker only once.

Activity

  1. added a commit that references this issue on Nov 18, 2026
    10404ba
  2. groupsky commented on Nov 19, 2026

    @groupsky
    ContributorAuthor

    Fixed in #218 (squash commit a2fac55).

    • The harness prints what nodkey try wrote when the stop fails. runTests in test/contract/harness_linux_test.go now writes the trial broker did not stop cleanly: <error>; nodkey try wrote: and then tr.output(). The two timeouts of stop() have their own text too.
    • The cause is the harness, not nodkey try. tr.stop() closed the stdin of nodkey try and then sent SIGTERM. When its input ends, nodkey try stops with exit status 1 ("the terminal approver stopped: no answer can come any more", serveTry in cmd/nodkey/try.go). This is on purpose. So the harness started a race between the end of the input (exit 1) and the signal (exit 0), and sometimes the end of the input came first. Now stop() sends only SIGTERM, waits until the stdout of nodkey try is closed, and then waits for the process. This is written in test(contract)!: stop the trial broker with SIGTERM alone #218.
    • nodkey try is not wrong, so cmd/nodkey does not change. Its tests already cover both ways: TestTryStopsWhenTheInputEnds (end of input → exit 1), and TestTryApproveWindow / TestTryExpiry (SIGTERM → 0), TestTryDeny (SIGINT → 0), TestTryStopsOnHangup (SIGHUP → 0).
    • 20 separate runs in a row on a CI runner, with for i in $(seq 20); do go test -race -count=1 ./test/contract/ || exit 1; done: 20 of 20 with the Go of go.mod and 20 of 20 with Go stable in run 44398083961 (jobs 135133323503 and 135133323613). The go.mod job also passed 20 of 20 earlier, in run 44394191315.

    A note on the PR: the README of test/contract/ says that a change to the harness is a change in behavior for callers. So #218 has ! in its title and a section in docs/updating.md, which says that no caller of nodkey run is affected. The release notes will list it under "BREAKING CHANGES".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions