Skip to content

Commit 768abc4

Browse files
authored
fix(toxgen): Preserve caches on --only and --skip (#8240)
### Description QoL improvement to make `--only`/`--skip` more useful. Running `populate_tox.py` with `--only` (or `--skip`) wiped most of the jsonl caches. At the end of each run, both `releases.jsonl` and `package_dependencies.jsonl` are rewritten to contain only entries that were accessed during the run. Skipped integrations are read from the existing `tox.ini` and never touch their `releases.jsonl` entries or their free-threading dependency entries, so all of those got pruned. Now unused entries are only pruned on full runs. Partial runs still sort the files and add newly fetched entries, but don't drop anything. Stale entries for the updated packages stick around until the next full run.
1 parent 843958d commit 768abc4

1 file changed

Lines changed: 9 additions & 2 deletions

File tree

‎scripts/populate_tox/populate_tox.py‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1330,6 +1330,11 @@ def main() -> dict[str, list]:
13301330

13311331
write_tox_file(packages)
13321332

1333+
# Unused cache entries are only pruned on full runs. With --only/--skip,
1334+
# skipped integrations don't touch (most of) their cache entries, so pruning
1335+
# would wipe them.
1336+
prune_cache = not (only_update or skip_update)
1337+
13331338
# Sort the release cache file
13341339
releases = []
13351340
with open(RELEASES_CACHE_FILE) as releases_cache:
@@ -1338,7 +1343,8 @@ def main() -> dict[str, list]:
13381343
with open(RELEASES_CACHE_FILE, "w") as releases_cache:
13391344
for release in releases:
13401345
if (
1341-
CACHE[_normalize_name(release["info"]["name"])][
1346+
not prune_cache
1347+
or CACHE[_normalize_name(release["info"]["name"])][
13421348
release["info"]["version"]
13431349
]["_accessed"]
13441350
is True
@@ -1353,7 +1359,8 @@ def main() -> dict[str, list]:
13531359
with open(DEPENDENCIES_CACHE_FILE, "w") as releases_cache:
13541360
for release in releases:
13551361
if (
1356-
DEPENDENCIES_CACHE[release["python_version"]][
1362+
not prune_cache
1363+
or DEPENDENCIES_CACHE[release["python_version"]][
13571364
release["install_targets_hash"]
13581365
]["_accessed"]
13591366
is True

0 commit comments

Comments
 (0)