[ThomasWaldmann]

Fixes #10450 for master. #10454 is the 1.4-maint PR.

On master, borg2's own repository access has no such lists. The quadratic calls.pop(0) only survives in LegacyRemoteRepository.call_many (src/borg/legacy/remote.py). get_many() passes all ids of a fetch_many() at once, so borg transfer --from-borg1 --chunker-params from a remote (ssh://) borg 1.x repository was quadratic in the chunk count of an item. Plain borg transfer --from-borg1 (without --chunker-params) fetches chunks one by one via get(), so it is not affected.

This PR uses collections.deque with popleft() for calls and waiting_for.

Measurements

Measured on a Ryzen 5 8500GE (Zen 4), Python 3.13. The source is a borg 1.4 repo (repokey-blake2, lz4) with one stdin item of N unique 4 KiB chunks (fixed,4096). The data compresses to almost nothing. The command is borg transfer --other-repo SRC --from-borg1 --chunker-params fixed,4096 --compression lz4 into a fresh aes256-ocb/blake3 repo. For remote runs, SRC is ssh://__testsuite__//... (borg serve over pipes).

chunks	local SRC	remote SRC, this PR	remote SRC, master
250 k	10.5 s	11.8 s	14.9 s
500 k	21.3 s	23.9 s	34.3 s
1 M	43.2 s	48.7 s	88.6 s
2 M	87.4 s	97.3 s	271.4 s
4 M	175.9 s	192.7 s	1878.8 s

The time master spends beyond this PR grows 3.3×, 3.9×, 4.4× and 9.7× per doubling (quadratic, worse once the list no longer fits in L3; the 4 M master run partly overlapped an unrelated build on the same box, so it may be a bit high). With this PR, remote transfer is linear at about 48 µs per chunk. The borg 1.4 client version in #10454 had a second front-popped list (preload_ids) and was affected much more (1M chunks: 115 s vs 15.6 s).

Tests

pytest src/borg/testsuite/legacyrepository_test.py src/borg/testsuite/repository_test.py src/borg/testsuite/archiver/transfer_cmd_test.py src/borg/testsuite/archiver/serve_cmd_test.py: 334 passed, 7 skipped.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.72%. Comparing base (8274470) to head (4897f70).
⚠️ Report is 3 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #10456      +/-   ##
==========================================
- Coverage   88.73%   88.72%   -0.02%     
==========================================
  Files         103      103              
  Lines       19387    19388       +1     
  Branches     3023     3023              
==========================================
- Hits        17204    17202       -2     
- Misses       1514     1518       +4     
+ Partials      669      668       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.