Repository navigation
Add temporal type tests for each synchronous cursor #1213
Description
Part of #1212.
Summary
Add integration tests that check each synchronous cursor's temporal values against the contract in #1212, using live Athena output.
Scope
- Patterns. The output depends on the read path, which is UNLOAD or CSV. The asynchronous cursors and managed results return the same values as the synchronous cursors (measured for Test the temporal types of each synchronous cursor #1226), so they are not repeated.
- Each of Cursor, S3FSCursor, PandasCursor, ArrowCursor, and PolarsCursor gets one temporal test.
- The test is parametrized by
unloadwhere the cursor supports it.
- One query per test. The formats to check are columns of one SELECT:
- 0 to 12 fractional digits.
- Years 1, 1600, 2300, and 9999.
- NULL in a mixed column and in an all-NULL column.
UTC, named zones, and offsets, fordate,time, and their time zone variants.
- Only output that Athena returns. No synthetic values, rewritten Parquet, or simulated time zone databases. Years outside 1 to 9999 are out of scope (Define and enforce the contract for Athena temporal types across cursors #1212).
- Expected values are written by hand from the contract. The current gaps are marked with the sub-issue that fixes them.
Activity
laughingman7743 commentedon Oct 16, 2026
The time zone cases are now decided in #1220 (comment). Expected results for the corpus, for fetch methods and PandasCursor (Arrow and Polars keep the text):
2020-11-01 05:30 UTCand06:30 UTCboth as2020-11-01 01:30:00.000 America/New_York. Generate both rows withat_timezone(TIMESTAMP '2020-11-01 05:30:00 UTC', 'America/New_York')and the 06:30 equivalent, so the live check also confirms that the two texts are identical. Expected:DataErrornaming the column, with nofold=0value. The PR for Define time zone conversion for values whose zone cannot be represented #1220 makes the shared converter raiseDataError(without the column name); Name the column in DataError for time zone values in every cursor #1217 adds the column name and unwraps PandasCursor's whole-readOperationalError, and Make PandasCursor's temporal dtypes and fetched types depend only on the Athena type #1219 makes the PandasCursor check vectorized.2020-03-08 02:30:00.000 America/New_York). Athena never writes one, so this is a synthetic corpus entry without a live counterpart. Expected:DataError(Define time zone conversion for values whose zone cannot be represented #1220 PR; column name in Name the column in DataError for time zone values in every cursor #1217).tzdata2024.2). Simulate it with a zone missing from the test's tz database instead of a fake name, if practical. Expected:DataErrornaming the column and the zone, not a naivedatetime(Define time zone conversion for values whose zone cannot be represented #1220 PR; column name in Name the column in DataError for time zone values in every cursor #1217).datetime, and PandasCursor returnsdatetime64[us, UTC](Make PandasCursor's temporal dtypes and fetched types depend only on the Athena type #1219).+14:00,-14:00, and+05:30, and the normalized keys:Etc/GMT+3is written as-03:00,UTC+3as+03:00,ZasUTC, andUS/Easternunchanged.timestamp(6) with time zonewritten as... UTCwith six fractional digits, which is how Iceberg columns read back.time with time zonealways carries an offset;TIME '... America/New_York'is rejected by Athena, so no named-zonetimeentry is needed.Tz rule differences between Athena and Python (for example
America/Asuncionfrom 2026) cannot be detected, so they are a documentation item in #1221, not a test case.[-]Add conformance tests for the temporal type contract[/-][+]Add temporal type tests for each synchronous cursor[/+]