damacy rewrite PR-2: planner consumes prefetch handles

[nclack]

Reviewer: read dev/damacy_rewrite_pr2.md first. Function-by-function plan for the whole PR.

PR-2 collapses the duplicate metadata fetch path the planner inherited from PR-1: the prefetcher already pinned array_meta / shard_index / chunk_layout in the prefetch caches, but the planner was independently calling the legacy synchronous zarr_meta_cache / zarr_shard_cache. After this PR, the planner reads everything by handle from the prefetch caches, the legacy caches are deleted, and the public config + stats surfaces are reshaped to match.

Glossary

• prefetch_handle — { slot, generation } pair the prefetcher uses to address a pinned cache entry. Looked up via prefetch_cache_try_get.
• planner_sample — internal input type to planner_plan. Carries (uri, aabb) plus h_meta / h_shards[] / h_layout handles. Replaces the bare damacy_sample* array.
• array_meta_cache / shard_index_cache / chunk_layout_cache — the three prefetch_cache instances inside damacy. Pinned by prefetcher_release_batch via the batch gate.
• advance_from_shard — prefetcher state-machine stage that waits on all of a sample's shard-index handles before promoting the slot to chunk_layout. Used to fail the whole sample on any shard error; now tolerates NOTFOUND.
• zarr_meta_cache / zarr_shard_cache — legacy synchronous caches the planner used to call from inside planner_plan. Deleted by this PR.

Summary

Look first at src/planner/planner.c. planner_plan now reads zarr_metadata via prefetch_cache_try_get(array_meta_cache, sample->h_meta), walks sample->h_shards[] for the per-shard shard_index_value, and pulls chunk_layout the same way — no copies, no pin management inside the planner. The handles are threaded through a new planner_sample type that damacy_plan.c::plan_reserve builds by stealing the relevant fields from each prefetcher_ready.

Non-obvious decisions worth a second look:

1. Prefetcher tolerates missing shards now. advance_from_shard used to fail the entire sample if any shard handle errored. After this PR, DAMACY_NOTFOUND per-shard is treated as "shard file absent — planner emits fill"; other errors still fail. The planner reads each h_shards[i] per-shard inside the iterator loop and calls prefetch_cache_query on the NULL-return case to distinguish fill vs. real failure. Test: tests/test_prefetcher.c::test_missing_shard_reaches_ready.

2. chunk_layout_fetch degrades to success/NULL. When the shard_index_cache has no usable entry (missing shard, or non-blosc codec), the chunk_layout entry resolves to success with NULL value rather than DAMACY_DECODE. The planner reads try_get → NULL as "no blosc1 layout, decoder uses caps". The non-blosc path was already shaped that way after PR damacy rewrite PR-1: file split + prefetcher producer #121; this PR extends the same NULL-value contract to the no-shard case so layout never gates on missing shard data.

3. prefetch_handle moved to its own header. prefetch_cache.h carries a _Atomic uint64_t in prefetch_gate that nvcc rejects in CUDA TUs. planner.h only needs the POD prefetch_handle, so it gets a minimal new header (src/prefetch/prefetch_handle.h); prefetch_cache.h is no longer transitively pulled into CUDA TUs through the planner.

4. Public ABI break in damacy_config.tuning and damacy_stats. n_zarrs_meta_cache / n_shards_meta_cache → n_array_meta_cache / n_shard_index_cache / n_chunk_layout_cache (new). zarr_meta_hits/... / shard_idx_hits/... collapse into nested array_meta.{hits,misses} / shard_index.{hits,misses} / chunk_layout.{hits,misses}. Python binding, bench JSON output, and bench report column names all updated in the same commits.

5. Three legacy tests deleted. test_chunk_layout.c / test_zarr_meta_cache.c / test_zarr_cache_threading.c (PR-2 part 1) and test_zarr_shard_cache.c (PR-2 part 2). Their behavior is exercised by test_chunk_layout_cache.c / test_array_meta.c / test_shard_index.c / test_prefetcher.c against the prefetch implementations.

Test plan

• test_missing_shard_reaches_ready (new): unlinks a shard file, drains the prefetcher, asserts the sample reaches PREFETCHER_READY and the per-shard handle queries as ERROR/NOTFOUND.
• test_missing_shard_becomes_fill and test_missing_shard_fills continue to pass, exercising the planner's fill emission downstream of the now-tolerant prefetcher path.
• test_planner.c fixture rewritten: builds array_meta_cache + shard_index_cache + chunk_layout_cache with a synchronous executor; mk_sample warms one shard handle per shard the AABB touches and tracks them for cleanup.
• 27/27 ctest pass.