Make fuzz budgets deterministic

Makefile

@@ -3,9 +3,6 @@
 # Keep release evidence in the specified order, including with make -j.
 .NOTPARALLEL:
 
-# Caller-provided Make variables must not replace release test commands.
-FUZZ_TIME ?= 10s
-
 test:
 	go run ./internal/testcheck ./...
 
@@ -33,9 +30,9 @@ net:
 	go run ./internal/testcheck -tags net ./examples/shadow/...
 
 fuzz:
-	go run ./internal/testcheck ./transport -run '^$$' -fuzz '^FuzzReadFrame$$' -fuzztime=$(FUZZ_TIME)
-	go run ./internal/testcheck . -run '^$$' -fuzz '^FuzzDecodeRequestContext$$' -fuzztime=$(FUZZ_TIME)
-	go run ./internal/testcheck ./internal/codegen -run '^$$' -fuzz '^FuzzParseGrainMarker$$' -fuzztime=$(FUZZ_TIME)
+	go run ./internal/testcheck ./transport -run '^$$' -fuzz '^FuzzReadFrame$$' -fuzztime=1600000x
+	go run ./internal/testcheck . -run '^$$' -fuzz '^FuzzDecodeRequestContext$$' -fuzztime=225000x
+	go run ./internal/testcheck ./internal/codegen -run '^$$' -fuzz '^FuzzParseGrainMarker$$' -fuzztime=2250000x
 
 resource:
 	go run ./internal/testcheck ./internal/runtime -run '^TestRuntime_ActivationChurnReleasesOwnedResources$$' -count=1

ROADMAP.md

@@ -9,9 +9,10 @@ Grain Timers, State failure recovery, bounded Reminder delivery, the Silo
 Activation admission limit, and bounded resource tests are complete for 0.1.0.
 Portable benchmark probes are also complete. The fail-closed local release
 gate and fuzz checks are also complete. The external restart proof and public
-document cleanup are complete. The exact tagged-module command is ready. The
-hosted supported-system run and the final same-commit candidate proof are
-complete. The successful tagged-module proof remains open. Action recovery
+document cleanup are complete. The exact tagged-module command is ready. A
+release-gate follow-up invalidated the earlier candidate proof. The hosted
+supported-system run, final same-commit candidate proof, and successful
+tagged-module proof remain open. Action recovery
 confirms Grain State before it commits an Application
 receipt. Pending actions recover in save order. An Invalid Reminder now gets
 one Terminal Result. Direct lifecycle proof for a mutable State alias and a
@@ -188,14 +189,15 @@ The implementation batches are:
    before it commits an Application receipt. A fourth follow-up gives an
    Invalid Reminder one Terminal Result. A fifth follow-up proves the direct
    State failure lifecycle. A sixth follow-up migrates the exact `v0.0.5`
-   schema and proves the external upgrade path. The hosted platform run and
-   final same-commit candidate proof are complete. The successful
-   tagged-module run is still open.
+   schema and proves the external upgrade path. A seventh follow-up gives
+   each fuzz target a fixed iteration budget. The hosted platform run, final
+   same-commit candidate proof, and successful tagged-module run are open for
+   the new candidate.
 
 The exact scope and proof for each batch are in
 [design/release-0.1.0.md](design/release-0.1.0.md). Batches 1 through 7 and
-Batch 8a through 8d are complete. Batch 8e candidate proof is complete; its
-exact tagged-module proof remains open.
+Batch 8a through 8d are complete. Batch 8e candidate and exact tagged-module
+proofs remain open.
 
 ## Cluster preview
 

design/release-0.1.0.md

@@ -391,8 +391,18 @@ Add these fuzz targets:
 
 Each fuzz target must contain valid and invalid seed inputs. A successful decode
 must satisfy the same size and round-trip rules as normal tests. `make fuzz`
-must run each target separately for 10 seconds by default. Go must save a
-failure in the normal fuzz corpus.
+must run each target separately with a fixed iteration budget:
+
+- `FuzzReadFrame`: 1,600,000 iterations;
+- `FuzzDecodeRequestContext`: 225,000 iterations;
+- `FuzzParseGrainMarker`: 2,250,000 iterations.
+
+The commands must use Go's `-fuzztime=Nx` form and must hard-code these budgets.
+A caller-provided Make variable must not reduce them. A fixed count gives every
+runner the same minimum work and avoids the Go fuzz coordinator's racy
+wall-clock deadline. A real target failure must still fail the command and Go
+must save it in the normal fuzz corpus. The measured basis is in
+[the issue 118 research note](../research/release-0.1.0-issue-118-fuzz-budget.md).
 
 Do not wrap a fuzz package in a process-wide goroutine count. The Go fuzz
 runner owns background goroutines, and their lifetime is not a package
@@ -423,6 +433,7 @@ Proof:
   `ci`, `external`, or `external-tagged`;
 - a mutation that removes the zero-test rejection fails its behavior test;
 - each fuzz target runs its seed corpus and one focused fuzz session;
+- the internal test command preserves a real fuzz callback failure;
 - `make ci` passes on Linux amd64;
 - all hosted platform jobs pass on the same commit.
 
@@ -473,8 +484,9 @@ Proof:
 
 #### Batch 8e: Final candidate proof
 
-**Status: In progress.** The final `master` candidate commands and hosted
-platform jobs are complete. The exact tagged-module proof remains pending.
+**Status: In progress.** A release-gate change invalidated the earlier final
+candidate proof. The new final `master` candidate commands, hosted platform
+jobs, and exact tagged-module proof remain pending.
 
 Before the final proof, complete the release audit for the upgrade from
 `v0.0.5`. That version stores Reminder identity in the `entity_type` and
@@ -567,13 +579,15 @@ Batch 7 hardens generation and moves implementation code under `internal/`.
 Batch 8a adds Activation admission. Batch 8b adds bounded resource proof and
 portable benchmark probes. Batch 8c implements fail-closed local release
 checks, fixed tools, fuzz checks, and the hosted platform workflow. The hosted
-workflow has passed on the final `master` candidate. Batch 8d implements the
+workflow passed before the fixed fuzz-budget follow-up. Batch 8d implements the
 external restart proof and public document cleanup. A release audit follow-up
 passes dynamic Reminder names to typed methods. A second follow-up prevents a
 caller-provided Make variable from replacing the internal test command. A
 third follow-up restores Grain State before it commits an Application receipt.
 An Invalid Reminder now gets one Terminal Result. A fifth follow-up proves
 the mutable State alias and Reminder-turn State failure lifecycle. Batch 8e's
-candidate commands, hosted proof, and final same-commit proof are complete. It
-still owns the exact tagged-module proof. A sixth release audit follow-up
-migrates the exact `v0.0.5` Reminder schema and adds the external upgrade proof.
+candidate commands, hosted proof, final same-commit proof, and exact
+tagged-module proof are pending for the new candidate. A sixth release audit
+follow-up migrates the exact `v0.0.5` Reminder schema and adds the external
+upgrade proof. A seventh follow-up gives every fuzz target a fixed iteration
+budget above the earlier successful work count.

design/release.md

@@ -90,8 +90,9 @@ The 0.0.x release-note question has a zero-maintenance answer: none is written.
 The earlier functional readiness work is complete. It includes public API
 comments, Runtime shutdown, errors, lifecycle, the example, observability, and
 benchmarks. The 0.1.0 contract adds composition and release evidence gates.
-Public document cleanup, hosted jobs, and the final same-commit candidate proof
-are complete. The final tagged-module proof remains open. See
+Public document cleanup is complete. A fixed fuzz-budget follow-up invalidated
+the earlier candidate proof. Hosted jobs, the final same-commit candidate
+proof, and the final tagged-module proof remain open. See
 [ROADMAP.md](../ROADMAP.md).
 
 Cluster support remains a preview. A partition can misjudge healthy Silos.

docs/release-0.1.0.md

@@ -308,6 +308,7 @@ internal test command. Action recovery confirms requested State before it
 commits the Application receipt. The clean external-module restart proof
 passes locally on Linux. The exact `v0.0.5` migration and external upgrade
 proof also pass locally on Linux. An Invalid Reminder now gets one Terminal
-Result. Public document cleanup is complete. Hosted
-supported-system proof and the final same-commit candidate proof are complete.
-The exact tagged-module run remains open. ROADMAP.md tracks the batches.
+Result. Public document cleanup is complete. A fixed fuzz-budget follow-up
+invalidated the earlier candidate proof. Hosted supported-system proof, the
+final same-commit candidate proof, and the exact tagged-module run remain open.
+ROADMAP.md tracks the batches.

internal/testcheck/main_test.go

@@ -38,3 +38,19 @@ func TestRunKeepsGoTestFailure(t *testing.T) {
 		t.Fatalf("stdout = %q, want child test failure", stdout.String())
 	}
 }
+
+func TestRunKeepsFuzzFailure(t *testing.T) {
+	var stdout, stderr bytes.Buffer
+	code := run([]string{
+		"-run", "^$",
+		"-fuzz", "^FuzzFail$",
+		"-fuzztime=1x",
+		"./testdata/fuzzfail",
+	}, &stdout, &stderr)
+	if code != 1 {
+		t.Fatalf("run exit = %d, want 1\nstdout:\n%s\nstderr:\n%s", code, stdout.String(), stderr.String())
+	}
+	if !strings.Contains(stdout.String(), "intentional fuzz failure") {
+		t.Fatalf("stdout = %q, want child fuzz failure", stdout.String())
+	}
+}

internal/testcheck/makefile_release_test.go

@@ -38,9 +38,9 @@ var releaseCommandContracts = []struct {
 			"go run ./internal/testcheck -tags net ./examples/shadow/...",
 			"go run ./internal/testcheck ./internal/runtime -run '^TestRuntime_ActivationChurnReleasesOwnedResources$' -count=1",
 			"go run ./internal/testcheck ./internal/timer -run '^TestPoller_LargeBacklogKeepsPagesAndWorkersBounded$' -count=1",
-			"go run ./internal/testcheck ./transport -run '^$' -fuzz '^FuzzReadFrame$' -fuzztime=10s",
-			"go run ./internal/testcheck . -run '^$' -fuzz '^FuzzDecodeRequestContext$' -fuzztime=10s",
-			"go run ./internal/testcheck ./internal/codegen -run '^$' -fuzz '^FuzzParseGrainMarker$' -fuzztime=10s",
+			"go run ./internal/testcheck ./transport -run '^$' -fuzz '^FuzzReadFrame$' -fuzztime=1600000x",
+			"go run ./internal/testcheck . -run '^$' -fuzz '^FuzzDecodeRequestContext$' -fuzztime=225000x",
+			"go run ./internal/testcheck ./internal/codegen -run '^$' -fuzz '^FuzzParseGrainMarker$' -fuzztime=2250000x",
 		},
 	},
 	{
@@ -70,7 +70,9 @@ func TestMakefile_ReleaseCommandsKeepTestcheck(t *testing.T) {
 				{"GO_TEST=true"},
 				{"MAKE=true"},
 				{"GO_TEST=true", "MAKE=true"},
+				{"FUZZ_TIME=1x"},
 				{"-j8"},
+				{"-j8", "FUZZ_TIME=1x"},
 				{"-j8", "GO_TEST=true", "MAKE=true"},
 			} {
 				overridden := makeDryRun(t, repository, contract.target, settings...)

internal/testcheck/testdata/fuzzfail/fuzz_test.go

@@ -0,0 +1,10 @@
+package fuzzfail
+
+import "testing"
+
+func FuzzFail(f *testing.F) {
+	f.Add("seed")
+	f.Fuzz(func(t *testing.T, value string) {
+		t.Fatal("intentional fuzz failure")
+	})
+}

research/release-0.1.0-issue-118-fuzz-budget.md

@@ -0,0 +1,85 @@
+# Issue 118 fuzz budget evidence
+
+## Observation
+
+The final `master` proof for
+`884351d1b130ece20ede99c300995677ff202f9b` ran in GitHub Actions workflow
+[31499798759](https://github.com/suraciii/gor/actions/runs/31499798759).
+Linux stable used Go 1.26.5. `make ci` reached the last fuzz target and then
+failed:
+
+```text
+fuzz: elapsed: 10s, execs: 421658 (38686/sec), new interesting: 22 (total: 75)
+--- FAIL: FuzzParseGrainMarker (10.08s)
+    context deadline exceeded
+```
+
+No product assertion failed. Go wrote no failing corpus input. The earlier two
+targets completed 1,356,179 and 32,992 executions. The second target stopped
+making progress before its wall-clock deadline but still passed.
+
+PR workflow
+[31499526676](https://github.com/suraciii/gor/actions/runs/31499526676)
+passed on `be9a3e02aae7b26a86d375f817bd5cca75a3261d`. The PR head and squash commit
+have the same tree, `fe4ff904865aaf5fe259b9b7a5ea8d0103012d2b`. The result
+therefore changed without a source-tree change.
+
+## Cause
+
+The release gate used `-fuzztime=10s`. Go implements a duration budget with a
+context deadline. The coordinator can race with propagation of that normal
+deadline and return `context deadline exceeded` as a test failure. The Go
+project tracks this defect in
+[golang/go#75804](https://github.com/golang/go/issues/75804). The upstream fix
+in [golang/go#79199](https://github.com/golang/go/pull/79199) suppresses the
+parent deadline as well as the child deadline. It was merged to Go master and
+backported to the Go 1.27 branch, but the required Go 1.25 and Go 1.26.5 lanes
+do not contain it.
+
+Retrying the workflow would not remove this race. Ignoring the error would also
+hide real target timeouts.
+
+## Decision
+
+Use Go's fixed-iteration `-fuzztime=Nx` mode. Set each budget above both the
+failed hosted run and the earlier successful ten-second run:
+
+| Target | Failed hosted run | Earlier successful run | Fixed budget |
+| --- | ---: | ---: | ---: |
+| `FuzzReadFrame` | 1,356,179 | 1,508,765 | 1,600,000 |
+| `FuzzDecodeRequestContext` | 32,992 | 200,792 | 225,000 |
+| `FuzzParseGrainMarker` | 421,658 | 2,185,654 | 2,250,000 |
+
+The fixed counts preserve a real fuzz session, require more work than either
+recorded run, and remove only the wall-clock stop race. The commands remain
+fail-closed for a panic, assertion, worker timeout, or invalid corpus entry.
+
+## Proof
+
+Working-tree validation passed before review:
+
+- `make fuzz` passed with exit 0 in 23.82 seconds at the first proposed fixed
+  counts;
+- changing the first expected count by one made the release contract test fail
+  with exit 1, and restoring it made the test pass;
+- `make ci` passed with exit 0 in 48.98 seconds with a cold test cache;
+- `make external` passed with exit 0 in 31.82 seconds with a cold test cache,
+  including 20 external tests and the exact `v0.0.5` upgrade.
+
+Review found that the first proposed counts were lower than an earlier
+successful run. The final counts above correct that defect. After the
+correction:
+
+- `make fuzz` passed with exit 0 in 22.18 seconds and completed 1,600,000,
+  225,000, and 2,250,000 executions;
+- changing the first expected count from 1,600,000 to 1,599,999 made the
+  release contract test fail with exit 1;
+- the internal test command preserved exit 1 and output from an intentional
+  fuzz callback failure;
+- `make ci` passed with exit 0 in 107.83 seconds with a cold test cache,
+  including 639 default and race tests and all three fixed fuzz budgets;
+- `make external` passed with exit 0 in 34.81 seconds with a cold test cache,
+  including 20 external tests and the exact `v0.0.5` upgrade.
+
+The mutation was restored. Exact-commit local proof and hosted proof remain
+pending.