fix(tracking): honor tracking.history_days instead of the hardcoded default - #3440
Open
iliaal wants to merge 2 commits into
Open
fix(tracking): honor tracking.history_days instead of the hardcoded default#3440iliaal wants to merge 2 commits into
iliaal wants to merge 2 commits into
Conversation
…efault `history_days` was declared on TrackingConfig, given a default, written into the generated config.toml and documented in src/core/README.md — but never read. `cleanup_old` used the DEFAULT_HISTORY_DAYS constant directly, so the retention window was fixed at 90 days and the setting was inert. Resolve the window once at construction and store it on Tracker rather than loading the config inside cleanup_old, which runs on every record() and record_parse_failure() and must not touch the filesystem per call. A non-positive value falls back to the default instead of being honored: a typo'd 0 would otherwise delete the entire history on the next write. The same fallback covers an unparseable config, including a [tracking] section that omits history_days — the field has no serde default, so a partial section fails the whole parse. The pruning test is asserted in both directions on purpose. A hardcoded 90 still passes the "retained within window" half, so only the "pruned beyond window" half catches the regression, and only the pair shows the field is what drives the cutoff.
These databases have no auto_vacuum, so retention only marks pages reusable and the file never shrinks. Shortening history_days deletes millions of rows and leaves the file at its old size indefinitely: on one box, dropping 90 to 30 pruned 3.09M rows and left 1.9GB on disk with 445,466 free pages. cleanup_old now counts what it deleted and, when anything went, vacuums if enough of the file is free pages. VACUUM cannot run unconditionally here. cleanup_old runs once per tracked command, and VACUUM rewrites the whole database under an exclusive lock, which would both blow the startup budget and stall concurrent rtk processes sharing the DB. Gating on the free-page ratio keeps it dormant in steady state, where inserts reuse freed pages, and lets it fire on the bulk deletes that actually strand space. Measured on a 196MB database: 20 consecutive tracked commands leave the freelist at 1-4 pages of 50,067, so the gate stays closed and page_count is unchanged. Injecting 300k out-of-window rows takes it to 320MB, and the next tracked command prunes and reclaims back to 196MB with quick_check ok. Failure is ignored. VACUUM needs scratch space near the size of the live data and cannot run inside a transaction; neither is a reason to fail a user's command.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tracking.history_dayshas no effect.Tracker::cleanup_oldcomputes its cutoff from theDEFAULT_HISTORY_DAYSconstant, so the retention window stays at 90 days no matter what the config says.The setting looks live from every other angle. It's a field on
TrackingConfig, it gets a default,Config::savewrites it into the generatedconfig.toml, andsrc/core/README.mddocuments it. Nothing reads it.I found this with a 1.9 GB
history.db(3.4Mcommandsrows, 3.2Mparse_failures) and no way to shrink the window.Fix
Trackerresolves the window once at construction and stores it on the struct.cleanup_oldreads that field.Config::load()stays out ofcleanup_olddeliberately: it runs on everyrecord()andrecord_parse_failure(), so loading the config there would put a file read in the per-command path.A non-positive value falls back to the default rather than being honored. A typo'd
history_days = 0would otherwise delete the whole history on the next write. That fallback also covers an unparseable config, including a[tracking]section that omitshistory_days, since the field has no serde default and a partial section fails the whole parse.Tests
test_resolve_history_dayscovers the clamping. Twocleanup_oldtests assert both directions against a backdated row: pruned at a 3 day window, retained at 30.Both directions are needed. A hardcoded 90 still passes "retained within window", so only "pruned beyond window" catches the regression. I checked by reverting the cutoff line on its own, and that test fails (
left: 1, right: 0) while its sibling stays green.That asymmetry is probably why the bug lasted. Every existing config test uses the default value, and a dead knob is invisible to any test that never sets it to something else.
Verification
cargo fmt --check,cargo clippy --all-targets --all-features -D warnings, andcargo test --bin rtk core::trackingare clean on this branch.cargo test --allfails two tests,unattestable_passthrough::test_plain_command_still_rewritesandtest_fd_dup_redirect_still_rewrites. They fail the same way on unmodifieddevelop(checked in a detached worktree at3044911), so they're unrelated to this change. They read the host's realsettings.json, which is why the result depends on the machine.End to end on a copy of the real database, with
history_days = 30: 3,087,798 rows older than the window pruned on the next write, 3,369,148commandsdown to 280,984.Second commit: return the pruned space
Fixing the setting on its own doesn't shrink anything. There's no
auto_vacuumon these databases, so a delete only marks pages reusable. Dropping 90 to 30 pruned 3.09M rows and left the file at 1.9 GB with 445,466 free pages, which makes the setting look broken even once it works.So
cleanup_oldnow counts what it deleted and, when anything went, vacuums if enough of the file is free pages.VACUUMcan't run unconditionally here.cleanup_oldfires once per tracked command, andVACUUMrewrites the whole database under an exclusive lock, which would blow the startup budget and stall concurrent rtk processes sharing the DB. The free-page ratio gate keeps it dormant in steady state, where inserts reuse freed pages, and lets it fire on the bulk deletes that strand space. Thresholds are 4096 free pages (~16 MB at the default page size) and 25% of the file, both named constants.Measured on a 196 MB database:
The gate stays shut across normal use and opens on the bulk prune.
quick_checkok afterward.VACUUMfailure is ignored, same as the existing WAL pragma. It needs scratch space near the size of the live data and can't run inside a transaction, and neither is a reason to fail someone's command.If you'd rather keep this PR to the config fix alone, say so and I'll split the second commit out.