PolarsCursor fails with Polars 2.0 #1168

Problem

PolarsCursor and AioPolarsCursor fail with Polars 2.0.0 (released 2026-10-06).
The polars extra requires polars>=1.39.0 without an upper bound, so a fresh pip install "PyAthena[polars]" now resolves Polars 2.0.0 on every supported Python version.

With Polars 2.0.0, 50 of 147 tests in tests/pyathena/polars/ and tests/pyathena/aio/polars/ fail on both CPython 3.12 and 3.15 (master 6ea5587d, otherwise the locked dependencies):

1. CSV reads from S3 reject the fsspec storage options (46 failures).
   AthenaPolarsResultSet._csv_storage_options (pyathena/polars/result_set.py:358) passes the PyAthena Connection and other non-string values, which Polars 2.0's read_csv/scan_csv rejects:

   ValueError: invalid value for 'connection': '<pyathena.connection.Connection object at 0x...>' (expected str)
   

   Every non-UNLOAD query whose result is read from the S3 output location fails with OperationalError, including plain execute() + fetchone().

2. new_columns shorter than the file's columns raises SchemaError (2 failures, test_txt_new_columns_with_schema_overrides).

   provided `new_columns` does not match number of columns in file (1 != 2 in file). Ensure the number of names match, or pass extra_columns='ignore' to ignore (1) extra columns.
   

3. Headerless CSV defaults changed (2 failures, test_csv_read_kwargs_replace_defaults).
   Generated column names start at column_0 instead of column_1, and the test's string expectation for the first column no longer holds.

This also blocks Polars support on Python 3.15 (#1163): Polars 1.x Series methods such as is_null() return None on 3.15 (pola-rs/polars#29419), and the fix (pola-rs/polars#29489) is released only in Polars 2.0.0.

Reproduction

uv sync --group dev -p 3.12
uv pip install polars==2.0.0
uv run --no-sync --env-file .env pytest -n 4 tests/pyathena/polars tests/pyathena/aio/polars

Environment

• PyAthena master 6ea5587d (4.1.0 + unreleased commits)
• Polars 2.0.0 (polars-runtime-32 2.0.0)
• CPython 3.12.14 and 3.15.0, macOS arm64

Proposed fix (optional)

Support Polars 2.0 in the Polars result set (storage options, new_columns, and the affected tests), keeping Polars 1.x working for the supported floor, and test both in CI.
The approach for passing S3 access to Polars 2.0 needs investigation before implementation.