You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
core: resolve the output directory once a pass, not once a file
Checking that a manifest entry stays inside the output directory resolved
the whole directory from the root down, and then resolved the file from the
root down, for every entry. A run of 3000 entries walked the same ancestors
6000 times. On Windows the cost of resolving a path grows with its depth,
about 0.24 ms per component here, so that was most of what verify did.
Measured on 3000 files of 1 kB, same machine, same moment:
working out paths 12475 ms
opening and hashing all 626 ms
the walk and the stats 257 ms
-------
the whole of verify 13358 ms
verify now takes 0.90-1.09 s where it took 16.9-17.3 s from a ten component
path, and 0.50-0.59 s where it took 2.40-2.56 s from a three component one.
cleanup goes through the same code and gets the same saving. Linux was
already fast and is unchanged. No concurrency was added.
What was rejected, and why it matters. The recorded plan was to hash in
parallel, on the reading that the loop already did the minimum so the cost
had to be the operating system opening files. A probe that timed the real
verify beside its own loop showed 869 ms against 17295 ms on the same files
at the same moment, which is how the missing 95% was found. Hashing in
parallel would have addressed 4.7% of the run. It scales 4.75x at eight
threads, so the number exists if the reason ever does.
The first version of this was wrong in a way worth recording. Answering from
the walk below the boundary alone is quick and it is not the rule: a link
inside the output directory pointing at another file inside it has left
nothing, and this tool accepted it before. Refusing it would have been a
change to what the tool accepts, smuggled in as a change to how fast it
answers. The quick reading now only ever answers "inside" - it cannot
wrongly refuse, only wrongly accept - and anything else falls through to the
original reading. Wrongly accepting is what the junction guards already
catch.
Nothing about what verify and cleanup accept or refuse has changed. Only the
boundary is settled once. Every entry still gets its own walk below it,
asked of the filesystem at that moment, so a link planted mid-run is still
caught. Swapping the named directory itself was never caught, before or
after: the old reading resolved both ends through the new link and found
them agreeing.
Four new guards, all proven by mutation. Two of them read the source,
because there is no behaviour to assert on: putting the resolution back
inside the loop, or comparing against the directory as typed rather than its
absolute spelling, both leave every verdict correct and quietly give the
cost back. filepath.Rel refuses a relative path against an absolute one, so
the second mistake would have been slow in the field and green in the suite,
for exactly the people who type "./out" - every other containment guard here
builds its directory with t.TempDir, which is absolute.
The README carried the same wrong reading as advice - it told people the cost
was the antivirus opening each file, and quoted 22 seconds. It is corrected
here rather than left standing, because it was the one place the mistake was
being handed to somebody else.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
0 commit comments