Skip to content

Preserve fractional seconds in to_timestamp() - #338

Open
fallintoplace wants to merge 1 commit into
ClickHouse:mainfrom
fallintoplace:fix/to-timestamp-precision
Open

Preserve fractional seconds in to_timestamp()#338
fallintoplace wants to merge 1 commit into
ClickHouse:mainfrom
fallintoplace:fix/to-timestamp-precision

Conversation

@fallintoplace

Copy link
Copy Markdown
Contributor

Summary

to_timestamp(float8) was pushed down as fromUnixTimestamp(toInt64(...)), which truncated fractional Unix timestamps. Leave the built-in for PostgreSQL evaluation so subsecond precision is preserved.

The regression coverage checks that 2042323443.232 does not match 2042323443, and does match the fractional value.

Testing

  • The extension builds and installs with -Werror.
  • The to_timestamp section of make installcheck REGRESS=functions matches.
  • This ClickHouse 26.7 image still has unrelated existing base64/base64url, JSON, and timeout expectation differences.

@fallintoplace
fallintoplace marked this pull request as draft August 3, 2026 23:56
@fallintoplace
fallintoplace marked this pull request as ready for review August 3, 2026 23:58
@theory

theory commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

This is not the correct solution. Switch to toDateTime64() instead.

Postgres:

david=# select to_timestamp(1284352323.32300) ;
        to_timestamp        
----------------------------
 2010-09-13 04:32:03.323+00
(1 row)

ClickHouse:

:) set date_time_output_format='iso'

:) select toDateTime64(1284352323.32300, 6, 'UTC');

   ┌─toDateTime64(⋯23, 6, 'UTC')─┐
1. │ 2010-09-13T04:32:03.323000Z │
   └─────────────────────────────┘

@theory theory added pushdown Improvements to query pushdown data types Improve data type support functions Improve function pushdown bug Something isn't working labels Aug 6, 2026

@theory theory left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much better, thanks. Just needs a few tweaks.

Comment thread test/sql/functions.sql Outdated
Comment on lines +366 to +368
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM t6 WHERE to_timestamp(i64) = to_timestamp(0);
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM t6 WHERE to_timestamp(i64) IS NOT NULL;
SELECT * FROM t6 WHERE to_timestamp(i64) = to_timestamp(0);
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM t6 WHERE to_timestamp(f64) = to_timestamp(0);
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM t6 WHERE to_timestamp(f64) IS NOT NULL;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these not coming out the same? We want to make sure they're equivalent, not just non-NULL.

Comment thread CHANGELOG.md Outdated
that could corrupt memory and hang ([#296]).
* Fixed subsecond precision lost inserting timestamps over HTTP ([#300]).
* Fixed loss of subsecond precision in `to_timestamp(float8)` by mapping it
to ClickHouse's microsecond-precision `toDateTime64()` function.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link to this PR.

Comment thread CHANGELOG.md Outdated
Comment on lines +604 to +606
`fromUnixTimestamp(toInt64())` in ClickHouse.
`toDateTime64(value, 6, 'UTC')` in ClickHouse.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not change this

Comment thread test/sql/functions.sql
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM t6 WHERE to_timestamp(f64) = to_timestamp(0);
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM t6 WHERE to_timestamp(f64) IS NOT NULL;
SELECT * FROM t6 WHERE to_timestamp(f64) = to_timestamp(0);
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM t6 WHERE to_timestamp(i64) = to_timestamp(0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still need to be sure that epoch 0 is the same. Please restore.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed the remaining tweaks. I restored the epoch-0 coverage, changed the integer and float tests to verify equality, added the PR link to the changelog, and restored the historical changelog entry. I also rebased onto main and pushed commit 221d64d.

Please take another look when you have a chance.

Map to_timestamp(float8) to ClickHouse toDateTime64 with six-digit
precision and UTC so fractional Unix timestamps survive pushdown.

Cover integer and float inputs at epoch zero and fractional timestamps,
and document the ClickHouse mapping.
@fallintoplace
fallintoplace force-pushed the fix/to-timestamp-precision branch from 73fdceb to 221d64d Compare August 6, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working data types Improve data type support functions Improve function pushdown pushdown Improvements to query pushdown

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants