Skip to content

PHOENIX-8032 Support SPLIT ON in CREATE CDC to pre-split the CDC index - #2643

Open
shubham-roy wants to merge 5 commits into
apache:masterfrom
shubham-roy:PHOENIX-8032
Open

shubham-roy wants to merge 5 commits into
apache:masterfrom
shubham-roy:PHOENIX-8032

Conversation

@shubham-roy

Copy link
Copy Markdown
Contributor

JIRA: https://issues.apache.org/jira/browse/PHOENIX-8032

What changes were proposed in this pull request?

  • CREATE CDC accepts an optional SPLIT ON (...) clause after its properties, in the same position as CREATE TABLE and CREATE INDEX.
  • The split points are compiled with the CREATE CDC statement through CreateTableCompiler.compileSplits, the existing CREATE TABLE split evaluation
    extracted into a static method, so constants and bind parameters both work.
  • MetaDataClient.createCDC passes them to the internal CREATE UNCOVERED INDEX statement as hex literals, so the existing CREATE INDEX split handling
    pre-splits the CDC index with exactly those bytes.

Why are the changes needed?

The CDC index is always created with a single region, so every CDC index write lands on that region until it splits by size. On large, write-heavy tables
that region becomes a hotspot as soon as CDC is enabled.

Does this PR introduce any user-facing change?

Yes, a new optional clause:

CREATE CDC my_cdc ON my_table SPLIT ON ('4', '8', 'c')
  • Before: SPLIT ON was a syntax error for CREATE CDC.
  • Split points apply to the CDC index row key, which starts with PARTITION_ID() (TENANT_ID first for multi-tenant tables).
  • For a CDC on a view the clause is accepted and has no effect, the same as CREATE INDEX ... SPLIT ON on a view.

How was this patch tested?

  • New QueryParserTest#testCreateCDCWithSplitPoints, plus a clause-order case in testCreateCDCWithErrors.
  • New CDCDefinitionIT#testCreateCDCWithSplitPoints: the CDC index regions match a mix of literal and raw byte bind split points.
  • JDK 8: full mvn clean install, then QueryParserTest, QueryCompilerTest, CDCDefinitionIT, SkipScanAfterManualSplitIT, SaltedTableIT and
    AggregateIT pass.
  • JDK 17: the same tests pass.
  • mvn spotless:check passes.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5.5)

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@virajjasani

Copy link
Copy Markdown
Contributor

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@shubham-roy

Copy link
Copy Markdown
Contributor Author

@virajjasani , I have synced for both PRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@virajjasani
virajjasani self-requested a review November 21, 2026 01:06

@virajjasani virajjasani left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one comment, looks good otherwise. We need fix for some generic check failures reported by github workflow

.execute("CREATE VIEW " + parentName + " AS SELECT * FROM " + tableName);
}
String cdcName = generateUniqueName();
byte[][] splitPoints = { Bytes.toBytes("4"), { 0x5a, (byte) 0xff }, Bytes.toBytes("c") };

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use some complex binaries here to test that MetaDataClient logic is able to parse and provide splits correctly?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ba27a74. The test now uses 7 split points:

  • zero and high bytes, including a zero byte in the middle of a key
  • a multi-byte UTF-8 string (8é)
  • a key longer than the 32-character PARTITION_ID() column

They are given as a hex literal in the SQL (X'0001'), plain string literals and 4 bind values (one string, three raw bytes). Each region start key is now compared exactly with the bytes Phoenix computes for it (SchemaUtil.processSplit), instead of a prefix check.

The same commit fixes the new checkstyle and spotbugs warnings from the precommit general check in CreateCDCStatement. The remaining unit-check failures (UCFWithDisabledIndexWithDDLValidationIT, HAConnectionWithMasterAndRPCRegistryIT, ParallelPhoenixConnectionFailureTest and a few timing ones) also fail on recent master branch builds and other PRs.

Shubham Roy and others added 2 commits November 22, 2026 09:13
…tbugs warnings

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shubham-roy

Copy link
Copy Markdown
Contributor Author

CI on 24e7d7e (run https://github.com/apache/phoenix/actions/runs/44982037673): the general check now passes, after the checkstyle and spotbugs fixes in ba27a74. CDCDefinitionIT (34/34) and QueryParserTest (94/94) pass.

The remaining failures are not from this change. Each also fails on other PRs or branch builds:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants