Backport #1116 to 3.x: Read Arrow results with the connection session's credentials

[laughingman7743]

WHAT

This PR backports #1116 to the 3.x maintenance branch. The master merge commit is cherry-picked with -x as one commit, and the commit message records how the conflicts were resolved.

On 3.x, the pyarrow S3FileSystem that the Arrow cursors (ArrowCursor, AsyncArrowCursor, AioArrowCursor) use to read result files is now built from connection.session.get_credentials().get_frozen_credentials() in every case. The region and the connect_timeout/request_timeout arguments are passed as before.

File	3.x handling
pyathena/arrow/result_set.py	The credential change applies unchanged. The method keeps its 3.x name __s3_file_system; the rename and docstring come from #881.
docs/arrow.md	The role assumption bullet and comment are removed unchanged. The credentials sentence is added to 3.x's paragraph after the timeout examples, which predates #940's rewrite; the PyArrow 10 note stays (removed on master by #885).
tests/pyathena/arrow/test_result_set.py	New on 3.x, with master's license header and #1116's test only; it calls the name-mangled _AthenaArrowResultSet__s3_file_system.

Release notes (behavior changes):

• Profiles without static keys (credential_process, IAM Identity Center, role_arn with source_profile) read result files with the profile's credentials instead of PyArrow's default credential chain.
• With the role_arn connection argument, PyArrow no longer assumes the role a second time; it reads with the role credentials the connection already obtained (including MFA via serial_number). These credentials expire after duration_seconds, as they do for the connection's boto3 clients.
• With session=, the session's credentials are used, not aws_access_key_id/aws_secret_access_key connection arguments that the session was not built from.
• A failure to resolve the session's credentials is raised instead of falling back to PyArrow's default credential chain.

The connection's s3_config, verify, and use_ssl are still not applied to the pyarrow filesystem, as on master.

WHY

Fixes #1115 on 3.x.
The arguments the fix passes exist in every PyArrow version 3.x supports (pyarrow>=10.0.0): access_key/secret_key since 0.17.0, session_token since 2.0.0, and connect_timeout/request_timeout since 10.0.0 (checked in python/pyarrow/_s3fs.pyx at the release tags).

TEST

Tested commit: e3caedc (live AWS and pyarrow 10.0.0 runs at 0133b15; the amend only adds the license header to the new test file)

• just lint: passed.
• tests/pyathena/arrow/test_result_set.py (offline): 4 passed; with the source change reverted, the credential_process and role_arn cases fail.
• The same test with Python 3.10 and pyarrow 10.0.0 (uv run --isolated --with pyarrow==10.0.0 --with "numpy<2"): 4 passed.
• The fake S3/STS endpoint from ArrowCursor's native S3 filesystem ignores the connection's profile and role credentials #1115, with Python 3.10 and pyarrow 10.0.0: for profile_name="static", credential_process, role_arn + source_profile, and role_arn=, the Arrow filesystem signs with the same access key ID as a boto3 S3 client of the connection's session.
• uv run --env-file .env pytest -n 4 tests/pyathena/arrow tests/pyathena/aio/arrow (live AWS, Python 3.13, static credentials): 110 passed.

Not run: IAM Identity Center profiles and real cross-account roles against AWS; the full Python matrix (the release gate runs it).

🤖 Generated with Claude Code

[laughingman7743]

Self-review round one (implementation behavior): FINDINGS (1, repaired)

Base 7fc7f07310f7f32af08afacb0d5eb8ea80a61746 (3.x), head 0133b152cec94ad188cffb823acf8c5bee7a1412 (initial pass), repaired head e3caedc9bcf214d51588c21b7dcc8f7458bd9b21.

Covered: the three changed files against #1116 (209549db) and 3.x's surrounding code.

• The method body is byte-identical to master's; only the 3.x name __s3_file_system differs, and its only caller is __init__ (pyathena/arrow/result_set.py:113). Any is already imported on 3.x.
• 3.x reads self._fs only in _read_csv() and _read_parquet() (:270, :304), reached through _as_arrow() from __init__ (:115), so the frozen snapshot covers every read, as on master. AioArrowCursor builds the result set in asyncio.to_thread (pyathena/aio/arrow/cursor.py:148).
• 3.x's Connection.__init__ has the same session / role_arn / serial_number branches as master (pyathena/connection.py:280-320), so the session holds the assumed-role or MFA credentials there too.
• PyArrow floor: every argument passed exists in pyarrow 10.0.0; the new test passed with Python 3.10 + pyarrow 10.0.0, and the fake S3/STS endpoint showed pyarrow 10.0.0 signing with the session's credentials in all four scenarios.
• Regression test: the credential_process and role_arn cases fail on 3.x with the source change reverted.
• Finding (repaired in e3caedc): the new test file lacked the license header that master's file has and that other files added to 3.x by backports carry (for example tests/pyathena/pandas/test_result_set.py from Return the whole result from as_pandas() when auto chunking applies #950). The amend adds master's header; commit message and PR body updated.

Validation after repair: just lint passed; the new test 4 passed. Live Arrow suite (110 passed) and the pyarrow 10.0.0 runs were at 0133b15; the amend changes only the header comment.

[laughingman7743]

Self-review round two (claims, compatibility, operations): CLEAN

Base 7fc7f07310f7f32af08afacb0d5eb8ea80a61746, head e3caedc9bcf214d51588c21b7dcc8f7458bd9b21.

Claims checked:

• Conflict causes in the commit message and PR body: the rename and docstring come from Replace name-mangled helpers with single-underscore methods #881 (f4cc848b, merged in a86a180e); the paragraph rewrite from Correct user guide claims that contradict the implementation #940 (8bd91260, merged in 23ff61f6); the PyArrow 10 note removal from Require pandas 3.0 and pyarrow 22.0 #885 (842f0f05). The cherry-pick source SHA is 209549db8ff551dfd66290bdbad19130e129aff3 (verified with git rev-parse; an earlier local draft of the message had a wrong SHA and was corrected before the first push).
• PyArrow version claims: session_token first appears in _s3fs.pyx at apache-arrow-2.0.0 (absent at 1.0.0); connect_timeout/request_timeout at 10.0.0 (absent at 9.0.0); access_key/secret_key at 0.17.0.
• Release-note items hold for 3.x's Connection (same branches as master).
• Docs: other role-assumption text (docs/usage.md:743-791) describes connection arguments and does not mention Arrow.

Deferred (pre-existing, not introduced): 3.x's sentence that the timeouts "apply to all S3 operations performed by the cursor, including HeadObject and GetObject" is inaccurate; master corrected it in #940, which is not part of this backport.

Operations: as on master, one fewer sts:AssumeRole per Arrow result set with role_arn.

[laughingman7743]

Independent review (relayed): CLEAN

Reviewer: Codex CLI 0.162.1 (codex exec -s read-only --ephemeral, model gpt-6.1-sol), session 01a1248a-68f6-7242-b828-12246ec46a7e. Static review only: no tests, builds, edits, or GitHub access. Snapshot: detached worktree at head e3caedc9bcf214d51588c21b7dcc8f7458bd9b21, base 7fc7f07310f7f32af08afacb0d5eb8ea80a61746, compared with upstream 209549db8ff551dfd66290bdbad19130e129aff3; HEAD and clean status remained unchanged. The prompt gave the intended behavior, backport policy, and SHAs only (no PR number, description, or self-review findings).

Covered (as reported): both diffs and the factory/test correspondence; the commit message's resolutions against #881, #940, and #885 history; connection credentials for default/static/process/SSO profiles, profile-based roles, role/MFA arguments, supplied sessions, and absent/error paths; ArrowCursor, AsyncArrowCursor, AioArrowCursor, the CSV/Parquet readers, timeout propagation, Python 3.10 syntax, and PyArrow 10 constructor arguments; changed docs and the regression test.

The backport faithfully applies the upstream delta with the documented 3.x adaptations. The conflict-resolution description is accurate. No actionable defects found. The test can detect the original defect: the process-profile case requires PROCESSKEY, while the role case requires ROLEKEY and an empty PyArrow role_arn.

Non-actionable (as reported): docs/arrow.md:290 retains the pre-existing overstatement that the timeouts cover all cursor S3 operations (the boto3 HeadObject and manifest GetObject requests use connection.config); preserving it follows the backport policy. This matches the deferral in self-review round two (corrected on master by #940).