Check the user guides and docstrings against the implementation #927

Activity

1. laughingman7743

   added

   documentation

   on Oct 3, 2026
2. laughingman7743 commented on Oct 3, 2026

   

   laughingman7743

   on Oct 3, 2026

   MemberAuthor

   More actions

   Audit results against master 9b74767 (after #919). Line numbers are from that commit.

   Docstring list above

   13 entries are confirmed as written, and 2 hold only in part. None were dropped.

   • DefaultParameterFormatter time: this is a documentation error. time was never implemented; the wording came in with the docstring commit 41ed5c4. The docstring will be fixed, and no bug issue is needed.
   • pandas/result_set.py arraysize / retry_config: both descriptions are wrong. arraysize is the fetchmany() default. retry_config covers GetQueryResults and the result set's own HeadObject/GetObject; file reads use the connection's retry configuration.
   • polars/converter.py: decimal holds through get_dtype() → pl.Decimal(precision, scale). The wrong part is "Preserving JSON": json values are parsed for fetched rows, and the converter functions do not apply to as_polars().
   • The result_set_type_hints "column names" wording is also in 16 execute() docstrings. The fetch wording from aio/result_set.py is also in aio/cursor.py and the sync WithFetch.

   Also found in the same classes:

   • pandas/cursor.py:53,64,200 and pandas/result_set.py:214: cursor.fetchall() # Returns pandas DataFrame. It returns tuples.
   • pandas/cursor.py:56-59 and pandas/result_set.py:216-218: for chunk_df in cursor yields rows, not DataFrames.
   • pandas/cursor.py:47: an Attribute chunksize that does not exist.
   • "Automatically optimizes chunk sizes": this happens only with auto_optimize_chunksize=True.
   • spark/cursor.py:52: cursor.fetchall(). Spark cursors have no fetch methods.
   • spark/cursor.py:37: description is the calculation's description, not the session's.
   • cursor.py:36: a tuple parameter with %s. pyformat needs a dict.
   • options.py:37-39: "reset a field with merge". merge ignores None.
   • filesystem/s3_async.py:56-63: sync wrappers on asynchronous=True.
   • formatter.py:142: "Must be a SELECT or WITH statement" contradicts the method's own Note.

   User guides

   Examples that fail as written

   • Wrong import module: pandas.md:768 (AsyncPandasCursor), arrow.md:462 (AsyncArrowCursor), and cursor.md:264-268 (AsyncDictCursor).
   • Missing import: usage.md:113 (import pyathena), polars.md:312 (polars as pl), and s3fs.md:133 (typing.Any).
   • arrow.md:163-165: convert is not indented into the class, so the class stays abstract and cannot be instantiated.
   • filesystem.md:51,142,167: connect(region_name=...) alone raises ProgrammingError (no staging dir or work group).
   • aio.md:293-301: the sync wrapper is used on AioS3FileSystem(asynchronous=True).
   • polars.md:338-378: as_polars() always returns one DataFrame, never a PolarsDataFrameIterator, so all three examples iterate columns or call a missing method. Only iter_chunks() returns the iterator.
   • sqlalchemy.md:847,863,996,1012: select().from_statement(text(...)) raises NotImplementedError on Core.
   • sqlalchemy.md:871,1020: json.loads() is applied to values the converter already decoded.
   • usage.md:434-455 and sqlalchemy.md:606-684: the cancellation examples always wait out the full sleep (ThreadPoolExecutor exit). The SQLAlchemy one also cancels nothing: a new cursor has no query_id, so cancel() raises ProgrammingError.

   Wrong descriptions

   • usage.md:146-149: AsyncCursor-based cursors do not take on_start_query_execution. It is ignored when passed in options; on AsyncPandasCursor, a keyword argument goes to read_csv.
   • usage.md:262: expiration_time should be cache_expiration_time.
   • usage.md:283-284: the cache_size match also requires a SUCCEEDED query and the same schema and catalog. It never matches with unload=True, because the location includes a fresh UUID.
   • aio.md:212-225: not every aio cursor downloads in execute(). AioCursor pages GetQueryResults, AioS3FSCursor streams, and a chunked as_pandas() reads S3 synchronously while iterating.
   • aio.md:265-267: file operations use asyncio.to_thread, which is threads.
   • arrow.md:293-294: connect_timeout/request_timeout apply to the pyarrow filesystem only. HeadObject and the UNLOAD manifest GetObject use boto3.
   • spark.md:308: AsyncSparkCursor does not inherit from AsyncCursor.
   • spark.md:403-415: get_std_out/get_std_error can return None.
   • s3fs.md:125-128: there is no binary mapping, timestamp with time zone is missing, and nested scalars stay strings.
   • introduction.md:63: "connection pooling" does not exist.
   • sqlalchemy.md:331,397,412: "≥ 0" should be "positive"; 0 is treated as unset.
   • sqlalchemy.md:559: the callback runs after StartQueryExecution, not before.
   • sqlalchemy.md:723-729: the aio* dialects also support the callback.
   • sqlalchemy.md:854,882,896,951: ROW scalars are shown as ints; they stay strings.
   • sqlalchemy.md:1165: INTEGER should be INT in the DDL.
   • sqlalchemy.md:827,977,1164: without Iceberg, the dialect emits CREATE EXTERNAL TABLE ... LOCATION.
   • Wrong names: convertes → converter (pandas.md:295, arrow.md:171), AthenaResultSetObject → AthenaResultSet (pandas.md:665, arrow.md:368, polars.md:453, s3fs.md:315), the self-referencing "As with AsyncXCursor" (pandas.md:738,752, arrow.md:432,446, polars.md:511,525), and AsyncDIctCursor/DictCursor in the AsyncDictCursor section (cursor.md:219-222).
   • Omissions: pandas.md does not say that to_sql and unload need pyarrow. arrow.md:197-199 does not give the UNLOAD scope (SELECT/WITH only, location, staging dir required).

   testing.md, null_handling.md, and api.md have no mismatches; all 119 autodoc targets exist.

   Service-dependent claims, checked against Athena

   • CAST('<text>' AS JSON) returns a JSON string scalar. sqlalchemy.types.JSON turns it into a dict only by decoding a second time. Real JSON values (json_parse, JSON columns, CAST(ROW/MAP AS JSON)) raise TypeError → SQLAlchemy JSON columns fail on JSON objects and arrays returned by Athena #934.
   • Top-level CAST('[1, 2, 3]' AS JSON) works, so the "will raise InvalidRequestException" text at sqlalchemy.md:1387,1403-1405 is wrong.
   • CAST(ROW('John', 30) AS JSON) returns {"":"John","":30}, which decodes to {'': 30}. The example at sqlalchemy.md:871 will use a named ROW type.

   Code defects filed instead of documented

   #934 (SQLAlchemy JSON), #935 (as_pandas()/as_polars() consume the iterator), #936 (version_id on open()/cat_file()), #937 (lowercase file_format compression), and #938 (enhancement: AsyncPandasCursor constructor options). They join the earlier #923 and #924. The guides keep describing the intended behavior for these.

   Not filed:

   • AsyncPandasCursor forwarding extra keyword arguments to read_csv is documented. The guide text is fixed instead.
   • AioS3FileSystem(asynchronous=True) creates its executor with no loop, but no documented path opens files in that mode.
   • The cache's database/catalog case handling is intentional (Fix/739 cache schema catalog #740).

   Split

   There are about 25 docstring fixes and 40 guide fixes, so they go in two PRs: docstrings under pyathena/ (Part of #927), then the docs/*.md guides (Closes #927).
3. laughingman7743

   added a commit that references this issue on Oct 3, 2026

   Correct docstrings that contradict the implementation

   ...

   3840d11
4. laughingman7743

   closed this as completedin #940on Oct 3, 2026