Repository navigation
Add many-object scenarios to the filesystem benchmark suite #1317
Description
Part of #1315.
Summary
Add scenarios with many objects to the filesystem suite of the benchmark project. The suite transfers one object per trial, which measures the per-file parallelism of range requests and parts. The batch operations of fsspec (cat, get, put, rm, copy, find, and info fan-out) are where the implementations differ most: s3fs runs them as coroutines with batch_size concurrency on its IO loop, AioS3FileSystem gets the same concurrency from AsyncFileSystem with its thread pools (#1159), and the synchronous S3FileSystem inherits the sequential loops of AbstractFileSystem (copy and get in pyathena/filesystem/s3.py fall through to the base class).
Prepared objects
prepare gains a many-object set beside the single objects, shared by every host through the manifest:
files: 1,000 objects of 1 MiB under one prefix, forcat,get,copy, andinfo.listing: 100,000 zero-byte objects under one prefix with 100 keys per "directory", forls,find, andrm.
Writes (put, pipe) create their objects in the trial and delete them after checking the count. The S3 request cost is below $0.01 per trial at these counts.
Cases
The transport is the operation and the scale is the object count, as the single-object scales are sizes. Each case runs with sync, aio, and s3fs (#1316), and the concurrency sweep uses batch_size for the aio and s3fs APIs and max_workers for the sync API, so that the three run the same number of requests at the same time where the API allows it:
| Transport | Operation | Result checked |
|---|---|---|
cat |
cat(paths) of the files set |
total bytes |
get |
get(paths, local_dir) |
total bytes on disk |
put |
put(local_paths, prefix) of 1,000 local files of 1 MiB |
object count |
pipe |
pipe({path: bytes}) of 1,000 buffers |
object count |
copy |
copy(paths, prefix) within the bucket |
object count |
rm |
rm(paths) of 1,000 objects created by the trial |
zero remaining |
find |
find(prefix) of the listing set |
key count |
info |
info(path) of each of the files set, through the API's batch (gather for aio and s3fs, a thread pool for sync) |
entry count |
The metrics stay those of the suite: median wall time (and objects or bytes per second), peak RSS, peak thread count, and the event loop lag of the aio API. Add process CPU time (user and system) to the trial record, since the implementations spend their time differently (threads against coroutines), and the single-object cases benefit from it too.
Expected use of the results
If S3FileSystem's sequential batch operations are far behind the other two, parallel batch operations for the synchronous filesystem become the first improvement filed from #1315.
Activity
Deduplicate the transport choices and keep the summary notes to one s…
Keep the timing of failed operations and remove what a failed trial c…
Fail a trial whose local files cannot be removed and fix the loop lag…
laughingman7743 commentedon Oct 23, 2026
Done in #1320, merged as 7b20b49. The PR body referenced this issue with "Refs", so it did not close automatically.