simplify(maintenance): unify storage-report sizing on a bounded, coverage-aware filesystem walk #4303

Priority / scope

P2 · medium effort · medium reporting-compatibility risk. Static audit of master 064973de6e32bb2f31475d65bac9d184094fa470 (2026-10-18). This is a read-only reporting simplification and resource-bound fix, not a change to retention eligibility or a request to delete profile data.

Concrete source evidence

• build_storage_report_page_from_registered_global_db and project_storage_report_page, storage_report.rs:590–750 page registered projects and top-level directories, but each selected subtree is still sized in full. A page limit of 64 projects does not bound the files inside one project.
• append_project_report, storage_report.rs:895–947 calls walk_regular_files(&data_root, usize::MAX, ...), disabling the existing entry-count bound and ignoring its truncation result.
• The unregistered-directory phase instead calls orphan_stores::dir_size_bytes. walk_store_stats and dir_size_bytes, orphan_stores/pages.rs:32–66,141–150 use an independent recursive walk with no entry ceiling and silently skip directory-entry/metadata failures. The controlled counterpart exists, but this report does not call it.
• footprint.rs:25–28,155–210 already owns an entry ceiling, no-follow metadata traversal, unavailable-entry accounting, and partial coverage when truncated.
• append_project_report also runs generation_digest_scan_exceeds_budget before checking whether the supplied retention-protection set is already unavailable. Offline callers deliberately pass unavailable protection, so that scan cannot produce an actionable retention plan in that case.

Confirmed from control flow: daemon page size does not cap nested metadata work, and the two sizing paths have different error accounting. No wall-time, CPU, or production-outage attribution is claimed.

Exact implementation instructions

1. Use the existing footprint metadata walker for both registered and unregistered storage-report sizing. Remove the report's use of the unbounded dir_size_bytes convenience path; do not delete the age/mtime semantics needed by actual retention callers.
2. Pass one request-owned entry budget across every subtree sampled in a daemon call. Do not reset a full budget for each of up to 64 projects. Thread the caller's existing cancellation/deadline checks into this traversal rather than creating another scheduler, generic executor, or persistent census cache.
3. Return measured bytes together with explicit complete/partial/unavailable accounting. Preserve unavailable-entry counts and a distinct truncation/budget reason. Propagate this through per-store records, unregistered totals, profile_total_size, and CLI/dashboard rendering. Partial bytes must be presented as a lower bound, never as an exact complete size or a fabricated zero.
4. Keep the current opaque top-level directory cursor implementation. Do not replace it with repeated full-directory sorting. Distinguish 'more projects remain' from 'this subtree was only partially sampled'; never mint an outer cursor that falsely claims the unsampled subtree was fully covered. A documented partial sample is sufficient for the bounded daemon route—do not add a durable recursive traversal engine just for exact totals.
5. Keep the explicit full-profile CLI census a separate deliberate operation. Its existing truncation state and unavailable evidence must survive aggregation; do not silently relabel it complete or promise exact full totals after hitting its ceiling.
6. In append_project_report, check a known unavailable retention-protection result before enumerating generation files for the digest-size preflight. Still emit the same named unavailable reason, and do not plan or collect against an empty guessed protection set. Any remaining daemon generation-directory scan needs an entry/work bound as well as the byte threshold; many zero-byte files must not evade the budget.
7. Preserve metadata-only sampling, read-only free-page probes, source-directory side-effect rules, no-follow behavior, and the existing registry snapshot/lease ownership. This issue does not permit copying every graph database to calculate sizes or changing retention decisions.

Required behavioral tests

• A single project containing more entries than a small injected budget yields bounded visits and explicit partial coverage even when outer page size is one.
• Several projects share one call budget; later projects cannot restart unlimited work.
• Deep trees, zero-byte files, unreadable entries and symlinks produce honest totals/coverage without recursive stack growth or following another store.
• Cancellation/deadline checkpoints stop further traversal, including work running on the blocking pool; dropping only the awaiting future is not sufficient evidence.
• Full and partial totals survive serialization and the existing report aggregation/rendering paths.
• Missing retention protection performs no generation digest-size preflight and reports the original unavailable reason.
• Use deterministic visit counters and small isolated temporary trees, not flaky elapsed-time thresholds or a real operator profile.

Acceptance

• Both storage-report sizing paths use one bounded accounting implementation.
• Nested filesystem work is bounded per daemon call, not merely per top-level page.
• Read failures and truncation cannot disappear into apparently exact totals.
• Existing cursor semantics and retention protection remain intact.
• Obsolete report-specific traversal is deleted rather than wrapped by a new framework.

Run the affected maintenance and storage-report integration tests through their existing Bazel targets; no zero-test/compile-only acceptance. Related to #4099 observability, but distinct from its profile-growth/retention investigation: this ticket owns the report's traversal and accounting path. No Rust tests were executed during this audit.