Repository navigation
PHOENIX-8032 Support SPLIT ON in CREATE CDC to pre-split the CDC index - #2643
shubham-roy wants to merge 5 commits into
Conversation
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Need to sync with latest master. |
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
@virajjasani , I have synced for both PRs. |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
virajjasani
left a comment
There was a problem hiding this comment.
Left one comment, looks good otherwise. We need fix for some generic check failures reported by github workflow
| .execute("CREATE VIEW " + parentName + " AS SELECT * FROM " + tableName); | ||
| } | ||
| String cdcName = generateUniqueName(); | ||
| byte[][] splitPoints = { Bytes.toBytes("4"), { 0x5a, (byte) 0xff }, Bytes.toBytes("c") }; |
There was a problem hiding this comment.
Can we use some complex binaries here to test that MetaDataClient logic is able to parse and provide splits correctly?
There was a problem hiding this comment.
Done in ba27a74. The test now uses 7 split points:
- zero and high bytes, including a zero byte in the middle of a key
- a multi-byte UTF-8 string (
8é) - a key longer than the 32-character
PARTITION_ID()column
They are given as a hex literal in the SQL (X'0001'), plain string literals and 4 bind values (one string, three raw bytes). Each region start key is now compared exactly with the bytes Phoenix computes for it (SchemaUtil.processSplit), instead of a prefix check.
The same commit fixes the new checkstyle and spotbugs warnings from the precommit general check in CreateCDCStatement. The remaining unit-check failures (UCFWithDisabledIndexWithDDLValidationIT, HAConnectionWithMasterAndRPCRegistryIT, ParallelPhoenixConnectionFailureTest and a few timing ones) also fail on recent master branch builds and other PRs.
…tbugs warnings Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
CI on 24e7d7e (run https://github.com/apache/phoenix/actions/runs/44982037673): the general check now passes, after the checkstyle and spotbugs fixes in ba27a74. The remaining failures are not from this change. Each also fails on other PRs or branch builds:
|
JIRA: https://issues.apache.org/jira/browse/PHOENIX-8032
What changes were proposed in this pull request?
CREATE CDCaccepts an optionalSPLIT ON (...)clause after its properties, in the same position asCREATE TABLEandCREATE INDEX.CREATE CDCstatement throughCreateTableCompiler.compileSplits, the existingCREATE TABLEsplit evaluationextracted into a static method, so constants and bind parameters both work.
MetaDataClient.createCDCpasses them to the internalCREATE UNCOVERED INDEXstatement as hex literals, so the existingCREATE INDEXsplit handlingpre-splits the CDC index with exactly those bytes.
Why are the changes needed?
The CDC index is always created with a single region, so every CDC index write lands on that region until it splits by size. On large, write-heavy tables
that region becomes a hotspot as soon as CDC is enabled.
Does this PR introduce any user-facing change?
Yes, a new optional clause:
SPLIT ONwas a syntax error forCREATE CDC.PARTITION_ID()(TENANT_IDfirst for multi-tenant tables).CREATE INDEX ... SPLIT ONon a view.How was this patch tested?
QueryParserTest#testCreateCDCWithSplitPoints, plus a clause-order case intestCreateCDCWithErrors.CDCDefinitionIT#testCreateCDCWithSplitPoints: the CDC index regions match a mix of literal and raw byte bind split points.mvn clean install, thenQueryParserTest,QueryCompilerTest,CDCDefinitionIT,SkipScanAfterManualSplitIT,SaltedTableITandAggregateITpass.mvn spotless:checkpasses.Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5)
🤖 Generated with Claude Code