You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Run fsspec's conformance tests against the S3 filesystems #1329
Run fsspec's conformance tests, fsspec.tests.abstract, against S3FileSystem and AioS3FileSystem, and classify the failures.
Background
fsspec ships abstract test classes for copy, get, put, mv, open, and pipe (46 test functions in fsspec 2026.9.0, more with parametrization) that a filesystem runs by subclassing AbstractFixtures. s3fs runs them in s3fs/tests/derived against a moto server. PyAthena does not run them, and #1315 lists them as a precondition of step 5.
Plan
Add a fixture class for S3FileSystem and one for AioS3FileSystem, with paths below the test bucket's prefix, and run the classes against S3.
PyAthena's filesystem tests use real S3 rather than moto; keep the number of objects and the cost bounded, as for the other S3 integration tests.
Classify each failure as a defect of the filesystem, an intended difference from fsspec's local semantics (for example empty directories, which S3 does not have), or a limit of the fixtures, and record the intended differences as skips with their reason.
File the defects as grouped issues.
Done when
The conformance classes run in the filesystem tests, every skip names its reason, and the defects are filed.
Activity
Expect only the missing FileNotFoundError in the GH-1330 tests (GH-1329)
Close the aio filesystems, join keys with "/", and drop the aio pipe …