Skip to content

test: TestLoginRememberedTimer is flaky in CI with Go stable #205

Closed

Description

@groupsky

TestLoginRememberedTimer/the_broker's_timer (internal/broker/remembered_test.go:614) fails often in the test (stable) CI job, with Go 1.27.4:

remembered_test.go:614: value 2 that the store gave out is not cleared
remembered_test.go:614: value 3 that the store gave out is not cleared

Seen on 2026-11-16 and 2026-11-17, on the release pull request branch and on #204, which do not touch internal/broker. On #204 it failed three runs in a row and passed on the fourth. It passes locally, also in the full go test -race ./.... TestFloodOfBadAnswers (internal/ntfy/approver_test.go:249, "the approver keeps 1 values, want 0") failed once in the same runs.

The test likely checks the cleared values before the code that clears them has run (a race in the test, not in the broker), but this is not checked yet.

Done when

  • The cause is found and written in the pull request.
  • TestLoginRememberedTimer and TestFloodOfBadAnswers pass 20 runs in a row with go test -race -count=20 -run '...' on a CI runner.
  • If the cause is in the broker, not the test, a test covers the deny path.

Activity

  1. groupsky commented on Nov 18, 2026

    @groupsky
    ContributorAuthor

    It fails with the Go of go.mod too, not only with Go stable. The CI job test (go.mod) failed in TestLoginRememberedTimer/the_broker's_timer (remembered_test.go:614, values 2 and 3 not cleared) in:

    #210 is the same test and the same error.

  2. groupsky commented on Nov 18, 2026

    @groupsky
    ContributorAuthor

    Fixed in #219 (squash commit 4175660).

    The cause is found and written in the pull request.

    • TestLoginRememberedTimer: the cause was in the broker. preapproveLogin cleared its own copy of the login (values 2 and 3: the password and the TOTP seed of the fetch after the tap) with a defer. The defer ran after startedGrant had written and flushed the answer. So the test sometimes checked before the clear. Now the request clears its copy before it answers.
    • TestFloodOfBadAnswers: the cause was the test. The fake core gives the answer to the test inside answer(), and handle() forgets the value only after answer() returns. The test now waits for it (with a limit of 10 s).

    20 runs in a row on a CI runner: go test -race -count=20 -run 'TestLoginRememberedTimer|TestFloodOfBadAnswers' ./internal/broker/ ./internal/ntfy/ passed with Go stable and with the Go of go.mod, in run 44395774524 (jobs 135125115061 and 135127611297).

    The cause is in the broker, so a test covers the deny path: TestLoginPreapproveClearsBeforeTheAnswer (internal/broker/remembered_test.go) checks that the values are cleared before the answer: when the pre-approval is allowed, and on three deny paths (the fetch took too long, the item changed, a reload took the pin away). All four cases fail on the old code.

    A finding of the review is filed as #220 (two denied entries after a reload deny).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions