Skip to content

Conversation

@wzhero1
Copy link

@wzhero1 wzhero1 commented Dec 30, 2025

Purpose

This PR adds support for reading _SEQUENCE_NUMBER field in audit_log and binlog system tables for primary key tables.

A new configuration option changelog-read.sequence-number.enabled (default: false) is introduced to control whether to include the _SEQUENCE_NUMBER field in the output schema of audit_log and binlog system tables. When enabled, users can access the internal sequence number that Paimon uses for ordering records within primary key tables.

Key changes:

  • Added new CoreOptions.CHANGELOG_READ_SEQUENCE_NUMBER_ENABLED configuration
  • Modified AuditLogTable and BinlogTable to optionally include _SEQUENCE_NUMBER field in their row types
  • Updated ValueContentRowDataRecordIterator to output sequence number when enabled
  • Added schema validation to ensure this option is only valid for primary key tables
  • Updated predicate conversion logic to handle the additional system field

Tests

  • AuditLogTableTest - basic auditlog table reading and sequence number with table option
  • BinlogTableTest - basic binlog table reading and sequence number with table option
  • BatchFileStoreITCase - Flink integration test for audit_log and binlog reading sequence number

API and Format

  • New configuration option: changelog-read.sequence-number.enabled (Boolean, default: false)
  • When enabled, audit_log and binlog system tables will have an additional _SEQUENCE_NUMBER column after rowkind
  • No storage format changes

Documentation

  • Configuration documentation updated in core_configuration.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant