test: wait on real signals in two load-flaky tests

[ScriptedAlchemy]

Two tests failed under host load and passed alone. Both are test races. Production ordering is correct in both cases. Each test now waits on the signal that ends the work it checks.

prepared_generation_uses_bounded_parallelism_and_retained_bytes

Root cause. The test starts a prefetch for every path and then spawns a demand task for every path. It awaited only the demand tasks before it asserted observed_builds.active() == 0. A prefetch that waits on a demand build loops back to the process-global page cache when that build finishes. If the page has already left the cache, the prefetch builds the page again. The test does not wait for that rebuild, so it was still running at the assertion (left: 1, right: 0 at tests.rs:710).

Why the #2811 fix stopped holding. #2852 and #2971 fixed a different assertion, peak() > 8. They replaced it with a rendezvous barrier. The active() == 0 assertion was not affected by that change. It became exposed when cache misses grew more frequent. On 2026-10-09, 4479556 added release_unpinned_shared_jsonl_pages, and idle_release_drops_unpinned_prepared_pages_and_keeps_pinned_ones (same module, same suite binary) calls it. That call drops every unpinned page, including this test's pages, which are deliberately left unpinned. So in a full-suite run, a waiting prefetch now often misses the cache and rebuilds.

Fix. start_shared_jsonl_page_prefetch_with_cancellation now returns each spawned page task's TaskHandle alongside its AbortHandle. The production caller (SharedJsonlPathPin::start_prefetches) keeps only the abort handle and drops the task handle. Dropping a task handle detaches the task, which is the same as the old cx.spawn(..).is_ok(). The test-only start_shared_jsonl_page_prefetch returns the task handles, and the test joins every prefetch before it asserts.

Evidence. All runs use --runs_per_test=50, --run_under a taskset -c 0-1 wrapper, and --test_arg=<test name>. A temporary test-env switch forced every built page not to fit the cache, which reproduces the full-suite eviction. The switch is not part of this PR.

• Master, forced: 34 of 50 failed with the gate's assertion. Values ranged from left: 1 to left: 39.
• This branch, forced: 50 of 50 passed.
• This branch, unforced: 50 of 50 passed.

registered_persistence_failure_releases_only_after_join_and_writer_fence

Root cause. The producer had a 250 ms shutdown budget. After the worker joins with the terminal write failed, stop runs the writer fence. The fence is a begin_write_transaction and rollback that waits for the test's held writer. The test commits that writer only after it observes shutdown_joined(). If the test task ran late, the fence used up the rest of the 250 ms and timed out. Shutdown then returned observability_shutdown_deadline and the producer stayed JOINED, not SETTLED. That produced the gate failure assertion failed: retained.shutdown_settled().

Production ordering. SETTLED is stored only after the worker joins and the fence completes, or after a terminal write proves the writer settled. A fence that cannot finish leaves the producer JOINED, and the registry entry stays AwaitingWriter. That is the documented retry path. A mutation that stores SETTLED in place of JOINED (skipping the fence) fails this test 5 of 5 runs with assertion failed: !retained.shutdown_settled(). So the test still detects the ordering defect its name describes.

Fix. The shutdown budget is now 30 s, so the fence ends when the writer is released and not when the budget runs out. The pre-shutdown sleep(persistence * 6) is removed. It only made sure the blocked event failed early enough for the worker to reply within the 250 ms budget. With the new budget, the event fails with observability_persistence_deadline whether the run loop or the shutdown drain takes it.

Evidence. The same flags as above apply. A temporary 400 ms sleep before blocker.commit() stood in for a starved test task. That sleep is not part of this PR.

• Master, forced: 50 of 50 failed with assertion failed: retained.shutdown_settled().
• This branch, forced: 50 of 50 passed.
• This branch, unforced: 50 of 50 passed.

Clippy (--config=clippy) and rustfmt pass on both touched targets.

Fixes #3950

---

[changeset-bot]

⚠️ No Changeset found

Latest commit: c97790b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[devin-ai-integration]

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)