Skip to content

[FLINK-40148][common] Fix TableIdRouter producing wrong sink table for multi-digit source suffix#4473

Open
sd4324530 wants to merge 3 commits into
apache:masterfrom
sd4324530:fix/table-id-router-find-vs-matches
Open

[FLINK-40148][common] Fix TableIdRouter producing wrong sink table for multi-digit source suffix#4473
sd4324530 wants to merge 3 commits into
apache:masterfrom
sd4324530:fix/table-id-router-find-vs-matches

Conversation

@sd4324530

@sd4324530 sd4324530 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of this pull request?

Fix TableIdRouter.resolveReplacement so it routes tables correctly when the source-table regex contains a multi-character capturing group (e.g. a multi-digit suffix like 13, 14, 16) regardless of whether the sink-table uses a $N back-reference.

Previously, with a rule such as:

route:
  - source-table: db_6.table_([1-9]|1[0-6])
    sink-table:   new_db_6.table_merged

db_6.table_13 was wrongly routed to new_db_6.table_merged3 (and similarly _14...merged4, _16...merged6). The bug was caused by Matcher.find() + Matcher.replaceAll in resolveReplacement: find() only matched the prefix db_6.table_1 (because the alternation [1-9] wins over 1[0-6] for input 13), and replaceAll left the trailing 3 appended to the configured sink-table. Tables 1-9 were unaffected because the regex consumed the entire single-digit suffix.

Brief change log

  • Replace Matcher.find() + Matcher.replaceAll(sinkTable) with Matcher.matches() + Matcher.appendReplacement / Matcher.appendTail in TableIdRouter.resolveReplacement. This guarantees the whole source table id is consumed before the substitution is applied, so trailing characters can no longer leak into the sink-table name.
  • Add testRouteWithoutBackReferenceWithMultiDigitSuffix and testRouteWithBackReferenceAndMultiDigitCapture to TableIdRouterTest as regression coverage for this bug.

Verifying this change

This change added tests and can be verified as follows:

  • Added unit tests in flink-cdc-runtime/src/test/java/org/apache/flink/cdc/common/route/TableIdRouterTest.java:
    • testRouteWithoutBackReferenceWithMultiDigitSuffix — reproduces the originally reported scenario (sink-table without $1, multi-digit source suffix); fails on master, passes with the fix.
    • testRouteWithBackReferenceAndMultiDigitCapture — guards the same root cause for the $1 back-reference case to prevent future regressions.
  • All 12 tests in TableIdRouterTest pass, together with the 7 tests in TableIdRouterMatchModeTest and the 8 tests in SchemaDerivatorTest (27 tests total, no regressions).
  • Manually verified by running the failing test against unmodified master: the new test reproduces wrong sink-table names such as new_db_6.table_merged0, ...merged1, ...merged3, ...merged4, ...merged5, ...merged6, which match the issue's report.

Was generative AI tooling used to co-author this PR?
  • Yes

Generated-by: cluade code with minimax m3

…r multi-digit source suffix

TableIdRouter.resolveReplacement used Matcher.find() followed by Matcher.replaceAll
to compute the sink table id. find() only matches a prefix of the source table id,
and replaceAll internally calls reset() + find(), so:

  - When the source-table regex has a multi-character capturing group (e.g.
    db_6.table_([1-9]|1[0-6])), find() matched the prefix db_6.table_1 and left
    the trailing digit (3, 4, 5, 6, ...) appended to the configured sink-table,
    producing wrong sinks like new_db_6.table_merged3 for source db_6.table_13.
  - When the sink-table uses a $1 back-reference, the captured value was only
    the first matched digit for the same reason.

Switch to Matcher.matches() + Matcher.appendReplacement / Matcher.appendTail so
the entire source table id is consumed before the substitution, eliminating the
trailing-characters-leak. Add regression tests in TableIdRouterTest.

Signed-off-by: Pei Yu <125331682@qq.com>
@sd4324530
sd4324530 force-pushed the fix/table-id-router-find-vs-matches branch from adb78b7 to 543bb57 Compare July 15, 2026 00:22
@lvyanquan
lvyanquan requested a review from Copilot July 20, 2026 06:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes TableIdRouter.resolveReplacement to avoid incorrect sink table names when routing rules use regex capturing groups that can match multi-character suffixes (e.g., 13, 14, 16). This aligns the replacement logic with the router’s full-string match semantics and adds regression tests to prevent recurrence.

Changes:

  • Switch replacement logic from Matcher.find() + replaceAll(...) to Matcher.matches() + appendReplacement/appendTail to ensure the full source table id is consumed before substitution.
  • Add regression tests covering both “no back-reference” and “$1 back-reference” sink-table configurations for multi-digit suffix captures.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
flink-cdc-common/src/main/java/org/apache/flink/cdc/common/route/TableIdRouter.java Fix replacement logic to avoid trailing-character leakage into sink table names for multi-digit regex captures.
flink-cdc-runtime/src/test/java/org/apache/flink/cdc/common/route/TableIdRouterTest.java Add regression tests reproducing the bug and validating correct routing with/without $1 back-references.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +180 to +183
// would only match a prefix of the source table id (when the regex alternation
// order matters, e.g. `([1-9]|1[0-6])` matches `1` from `13` and leaves `3` as a
// leftover that gets appended to the sink table name, routing
// `table_13` to `table_3`).
sd4324530 and others added 2 commits July 20, 2026 14:45
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Pei Yu <125331682@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants