PolarsCursor holds a copy of the CSV result file while reading it whole with Polars 1.x

Summary

Since #1169, PolarsCursor holds an extra copy of the CSV result file while it reads a whole CSV result with Polars 1.x. The peak memory of an unchunked read grew by about the size of the file.

Evidence

The combined benchmark of the 4.2.0 release commit (#1288) measured unchunked PolarsCursor CSV reads on r7g.2xlarge, with the same library versions as the 4.1.0 run (Polars 1.44.2, pyarrow 25.0.1, fsspec 2026.9.0):

Rows, shape	4.1.0 peak RSS	4.2.0 peak RSS, Polars 1.44.2	4.2.0 peak RSS, Polars 2.0.0
1 million, flat	1,004 MiB	1,259 MiB	1,181 MiB
1 million, nested	1,561 MiB	2,105 MiB	not measured
10 million, flat (2.9 GB of CSV)	6,854 MiB	9,641 MiB	7,547 MiB
10 million, nested	13,006 MiB	18,923 MiB	not measured

The three trials of each case agree within 1%. The client result time did not change, and chunked reads are not affected.

Cause

AthenaPolarsResultSet._read_csv() reads the result file with fsspec.open(...).read() and passes the bytes to pl.read_csv() (#1169, also on 3.x through #1198). Before, it passed the path with storage_options.

A local check reading a 425 MiB CSV file shows the difference in Polars 1.x:

Source given to `pl.read_csv()`	Polars 1.44.2	Polars 2.0.0
local path	1,074 MiB	1,269 MiB
`bytes` of the whole file	1,530 MiB	1,303 MiB
local file object	1,105 MiB	1,293 MiB

A local file object avoids the copy in Polars 1.x, but the result file is an S3 object opened through fsspec, so this check does not show what Polars does with an S3File.

Proposal

Read the CSV result without holding a second copy of the file in Polars 1.x, for example by passing the open file, if Polars 1.x reads it without copying, or by keeping the path read for Polars 1.x. Measure the peak with a result of several hundred MB before and after.