You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Closing a PolarsCursor result set during the first chunked CSV read makes that read raise ProgrammingError #1188
Closing a chunked PolarsCursor result set while another thread is in the first read of a CSV result makes that read raise ProgrammingError: output_location is not available. instead of returning its chunk.
AthenaPolarsResultSet.close() (pyathena/polars/result_set.py:1023) calls super().close() first, which clears the query execution, so output_location becomes None.
The CSV chunk generator _iter_csv_chunks() (pyathena/polars/result_set.py:880) reads that state lazily in its first next().
It calls _is_csv_readable(), whose _get_content_length() sends an S3 request, and then checks self.output_location (pyathena/polars/result_set.py:896).
A close that lands before or during that request leaves the read without the output location.
This happens with AioPolarsCursor: a fetch runs in asyncio.to_thread() and keeps reading after its task is cancelled, while leaving async with closes the result set through close_in_thread() (pyathena/aio/common.py).
A synchronous cursor closed from another thread while it is read is outside threadsafety = 2.
Line numbers refer to 38dc8f86.
Before #1182, the close itself raised ValueError: generator already executing when the generator was running (#1172); after #1182, the close succeeds and the reader fails instead.
Found by an independent review of #1182.
Expected behavior
As in #1172, a fetch that is still reading when the result set is closed ends with the chunk it is reading.
The state that the chunk readers need (the output location, the CSV parameters, the column names) should not depend on the result set staying open until their first read.
Activity
Freeze the credentials of the chunk readers at their first read