Skip to content

fix!: a page token is only good under the filter it was issued for - #15

Open
iamralch wants to merge 3 commits into
mainfrom
fix/page-token-request-checksum
Open

iamralch wants to merge 3 commits into
mainfrom
fix/page-token-request-checksum

Conversation

@iamralch

@iamralch iamralch commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Closes #13.

Problem

The page token carried only the order_by and the boundary values. A client could change its filter between pages and keep its page_token; the server then continued from that cursor under a different predicate and returned pages that matched neither filter, with no error. AIP-158 requires the filter to stay the same between pages.

Change

The cursor now carries a checksum of the filter, and the composer checks it alongside order_by.

  • Cursor::with_filter(impl Into<WhereClause>) adds a filter clause, once per push_where call, in the same order.
  • The token carries only a CRC32 checksum of the filter, never its SQL. A checksum can't be ANDed onto, so it's chained one clause at a time; empty clauses count for nothing, as in push_where.
  • compose() computes the same chain over its own push_where clauses. A changed, added or dropped filter returns Error::CursorMismatch, whose message and docs now cover both order_by and the filter.
  • The order_by keys stay in the token as before: the cursor needs them to build its comparison and to supply the ORDER BY, and compose() already compares them directly, so a checksum of them would add nothing.
  • encode()/parse(token) keep their signatures.
  • CURSOR_TOKEN_CHECKSUM_MASK is bumped (0x5eed_c0de → 0x5eed_c0df), so tokens issued before this change are rejected cleanly instead of misread.
  • README and module docs are updated, including a stale CursorOrderByMismatch reference corrected to Error::CursorMismatch.

The commits show how the design changed: a request digest first, then the full filter in the token, then the filter checksum. Squash on merge.

For review

  • Breaking: a cursor used together with push_where now has to carry the same filter (Cursor::new(order_by).with_filter(filter)), or compose() fails. Callers who don't attach it get CursorMismatch on page 2 instead of the old silent behaviour.
  • The filter is matched clause by clause: push_where(a.and(b)) doesn't match with_filter(a).with_filter(b). The docs ask for the same clauses in the same order.
  • CRC32 means a changed filter goes unnoticed with odds of about 1 in 4 billion. This catches client mistakes; it isn't a security check, since the token is already forgeable (an unkeyed CRC).
  • Scope bound into the base query with bind_value (e.g. tenant_id = $1) isn't part of the filter, so it isn't checked. Scope added through push_where is.
  • Both mismatches share Error::CursorMismatch. Splitting them into separate variants would be a small change if you want to tell clients which one changed.

Testing

  • make lint, make test and make doc-check pass locally, and cargo check --no-default-features passes for both crates.
  • New tests:
    • The filter checksum round-trips through the token, and only the same clauses in the same order match it.
    • The composer accepts matching clauses (including an empty push_where) and rejects a changed, added or dropped filter, and a missing tenant scope.
    • A SQLite end-to-end test pages a filtered query through Pager and a token, then shows a changed filter is refused.
  • The PostgreSQL and MySQL tests skipped locally because no database URLs are set.

The token's checksum covered only the cursor, so a client could change
its `filter` between pages and keep its `page_token`: the server paged
on from that position under a predicate the token was never cut for,
returning pages that matched neither filter, without an error. AIP-158
requires every field but `page_token` and `page_size` to stay the same
between pages, and `aip-go`'s `CalculateRequestChecksum` enforces it.

The token now carries a digest of the request's fields -- whatever the
caller passes, typically parent, filter and order_by -- next to the
cursor. `parse` recomputes it from the request in hand and returns
`CursorError::RequestMismatch` when they differ. The digest is kept in
its own field so a changed request is told apart from a corrupted token,
and the checksum covers it, so rewriting it is still
`TokenChecksumMismatch`. Fields are length-prefixed, so moving text
across a field boundary is a different request. The empty token is
still the first page for any request.

`CURSOR_TOKEN_CHECKSUM_MASK` is bumped, so tokens issued before this
are rejected cleanly rather than misread.

Closes #13

BREAKING CHANGE: `Cursor::encode` and `Cursor::parse` take the request's
fields, `&[&str]`, in the same order on both sides. Tokens issued by
earlier versions no longer parse.
@iamralch iamralch self-assigned this Oct 6, 2026
@iamralch iamralch added the bug Something isn't working label Oct 6, 2026
Replaces the request digest in the page token. The cursor now carries
the filter it was issued under, set with `Cursor::with_filter`, and
`compose()` checks the `push_where` clauses against it the same way it
checks an explicit `order_by`: a changed, added or dropped filter is
`Error::CursorMismatch`. Comparing the decoded `WhereClause` is a
stronger guarantee than a hash, and `encode`/`parse` keep their
signatures.

`with_filter` accumulates with `and`, as `push_where` does, so the same
clauses in the same order combine to the same clause on both sides.

`CURSOR_TOKEN_CHECKSUM_MASK` is bumped, so tokens issued before this
are rejected cleanly rather than misread.

Closes #13

BREAKING CHANGE: a cursor used with `push_where` must carry the same
filter: build the pager's cursor with `Cursor::new(order_by)
.with_filter(filter)`, or `compose()` returns `Error::CursorMismatch`.
Tokens issued by earlier versions no longer parse.
@iamralch iamralch changed the title fix!: a page token is only good for the request it was issued for fix!: check the filter against the cursor in the composer, like order_by Oct 6, 2026
The cursor never needs the filter itself, only to tell whether the next
page's is the same, so a checksum is enough: tokens no longer grow with
the filter, and its SQL no longer travels to the client.

A checksum can't be ANDed onto, so `with_filter` chains one CRC32 per
clause, and `compose()` folds the same chain over its `push_where`
clauses -- the same clauses in the same order match, empty ones counting
for nothing. `Cursor::filter()` goes, since there is no clause to return.

The order_by keys stay in the token as they were: the cursor needs them
to build its comparison and to supply the ORDER BY, and `compose()`
already compares them directly.
@iamralch iamralch changed the title fix!: check the filter against the cursor in the composer, like order_by fix!: a page token is only good under the filter it was issued for Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Page-token checksum covers only the cursor, so filter changes between pages go undetected

1 participant