Skip to content

Make two S3FS result-set tests fail without the behavior they name #1255

Closed

Description

@laughingman7743

Summary

Part of #1239. Two offline tests in tests/pyathena/s3fs/test_result_set.py pass when the behavior they name is removed. Both tests predate #1252; the independent review of #1252 found them, and #1252 does not change them.

1. TestAthenaS3FSResultSet::test_managed_results_preserve_api_values

The test patches _get_query_results to return the whole response, rows included, for every request, including the first one with MaxResults=1. If the managed-results branch at pyathena/s3fs/result_set.py:129 is deleted, the rows of the first page restore both values, and both parameters still pass. With real managed results, the first page has only the header row, so the rows would be lost.

The fake should answer the first request and the later ones differently, for example through connection.client.get_query_results with a first page that has only the header and a later page with the data rows.

2. TestAthenaS3FSResultSet::test_reader_closes_at_end_of_results

The test asserts only that the underlying BytesIO is closed. If self._csv_reader.close() at pyathena/s3fs/result_set.py:237 is deleted and self._csv_reader = None is kept, CPython releases the reader and its TextIOWrapper, which closes the stream, so the assertion still holds. A reader that something else keeps a reference to would stay open.

The test should keep a reference to the reader or the wrapper until the assertion, so that only an explicit close passes it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions