Repository navigation
test(amber): share engine and service test fixtures - #9534
Merged
aglinxinyuan merged 1 commit intoNov 19, 2026
Merged
Conversation
Contributor
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9534 +/- ##
============================================
- Coverage 91.57% 91.55% -0.02%
+ Complexity 5157 5103 -54
============================================
Files 1300 1304 +4
Lines 58092 57400 -692
Branches 7615 7560 -55
============================================
- Hits 53195 52550 -645
+ Misses 2959 2919 -40
+ Partials 1938 1931 -7
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
aglinxinyuan
marked this pull request as draft
November 19, 2026 05:02
Contributor
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 422 | 0.258 | 23,387/34,970/34,970 us | 🔴 +38.8% / 🔴 +173.0% |
| 🔴 | bs=100 sw=10 sl=64 | 865 | 0.528 | 114,152/138,377/138,377 us | 🔴 +7.0% / 🔴 +69.2% |
| ⚪ | bs=1000 sw=10 sl=64 | 965 | 0.589 | 1,032,758/1,135,914/1,135,914 us | ⚪ within ±5% / 🔴 +48.7% |
Baseline details
Latest main 0aea3df from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 422 tuples/sec | 479 tuples/sec | 1,084 tuples/sec | -11.9% | -61.1% |
| bs=10 sw=10 sl=64 | MB/s | 0.258 MB/s | 0.292 MB/s | 0.662 MB/s | -11.6% | -61.0% |
| bs=10 sw=10 sl=64 | p50 | 23,387 us | 21,973 us | 9,548 us | +6.4% | +144.9% |
| bs=10 sw=10 sl=64 | p95 | 34,970 us | 25,189 us | 12,811 us | +38.8% | +173.0% |
| bs=10 sw=10 sl=64 | p99 | 34,970 us | 25,189 us | 15,432 us | +38.8% | +126.6% |
| bs=100 sw=10 sl=64 | throughput | 865 tuples/sec | 900 tuples/sec | 1,467 tuples/sec | -3.9% | -41.0% |
| bs=100 sw=10 sl=64 | MB/s | 0.528 MB/s | 0.549 MB/s | 0.895 MB/s | -3.8% | -41.0% |
| bs=100 sw=10 sl=64 | p50 | 114,152 us | 110,279 us | 74,441 us | +3.5% | +53.3% |
| bs=100 sw=10 sl=64 | p95 | 138,377 us | 129,364 us | 81,800 us | +7.0% | +69.2% |
| bs=100 sw=10 sl=64 | p99 | 138,377 us | 129,364 us | 103,712 us | +7.0% | +33.4% |
| bs=1000 sw=10 sl=64 | throughput | 965 tuples/sec | 977 tuples/sec | 1,517 tuples/sec | -1.2% | -36.4% |
| bs=1000 sw=10 sl=64 | MB/s | 0.589 MB/s | 0.596 MB/s | 0.926 MB/s | -1.2% | -36.4% |
| bs=1000 sw=10 sl=64 | p50 | 1,032,758 us | 1,022,947 us | 725,911 us | +1.0% | +42.3% |
| bs=1000 sw=10 sl=64 | p95 | 1,135,914 us | 1,106,339 us | 764,152 us | +2.7% | +48.7% |
| bs=1000 sw=10 sl=64 | p99 | 1,135,914 us | 1,106,339 us | 808,901 us | +2.7% | +40.4% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,473.72,200,128000,422,0.258,23386.99,34969.94,34969.94
1,100,10,64,20,2312.20,2000,1280000,865,0.528,114151.81,138376.64,138376.64
2,1000,10,64,20,20731.99,20000,12800000,965,0.589,1032758.11,1135914.06,1135914.06Consolidate repeated scheduling, coordinator handler, worker checkpoint, storage, replay, reconfiguration, and execution-service test scaffolding across 28 Amber test files (25 modified, 3 added; +1,416 / -2,209 lines) without altering production code or test assertions.
aglinxinyuan
force-pushed
the
refactor/repo-dedup-cleanup
branch
from
November 19, 2026 05:09
37193fb to
d399f72
Compare
aglinxinyuan
marked this pull request as ready for review
November 19, 2026 05:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
Consolidate repeated scheduling, coordinator handler, worker checkpoint, storage, replay, reconfiguration, and execution-service test scaffolding across
amber/src/test/scala/(+1,416 / −2,209 lines across 28 test files: 25 modified, 3 added) without modifying production code or test assertions:BarePhysicalPlanFixture: Extracts shared barePhysicalOp,Region,PhysicalLink, andGlobalPortbuilders acrossWorkflowExecutionSpec,JumpToOperatorRegionHandlerSpec,RegionPlanSpec,RegionSpec,ScheduleSpec, andSchedulingUtilsSpec.CheckpointHandlerSpecSupport: Shares PekkoTestKit/SingleNodeListenerlifecycle,liveWorkerand handler wiring, andPlainExecutorscaffolding acrossPrepareCheckpointHandlerSpecandFinalizeCheckpointHandlerSpec.ExecutionDatabaseTestSupport: Extracts shared database and workflow-execution seeding helpers acrossDefaultCostEstimatorSpec,AdminExecutionResourceSpec,ExecutionsMetadataPersistServiceSpec,ResultExportServiceSpec, andWorkflowExecutionServiceSpec.RecordStorageTestSupport: Shares temporary-folder lifecycle, record I/O helpers, and storage contract checks acrossSequentialRecordStorageSpec,HDFSRecordStorageSpec, andVFSRecordStorageSpec.CoordinatorMessageFixture&RecordingExecutionClient: AdoptsCoordinatorMessageFixtureinEmbeddedControlMessageHandlerSpecandRetrieveWorkflowStateHandlerSpec, and adds shared batch event/command helpers onRecordingExecutionClientforExecutionConsoleServiceSpecandExecutionStatsServiceSpec.DPThreadSpec,FriesReconfigurationAlgorithmSpec,ReplayLogGeneratorSpec,ExecutionReconfigurationServiceSpec): Consolidates repeated worker polling/lifecycle drivers, FRIES scenario registration, replay log generation with nested writer cleanup, and reconfiguration stub/dispatch assertions inside each spec.Any related issues, documentation, discussions?
Closes #9533.
How was this PR tested?
Verified on JDK 17 after final commit
d399f721b0(STORAGE_ICEBERG_CATALOG_TYPE=hadoop,AMBER_TEST_FILTER=skip-integration,UDF_PYTHON_PATHconfigured):Observed result: both checks exited
0.Observed result:
13suites completed,104tests succeeded (0failed,0canceled,0skipped).STORAGE_ICEBERG_CATALOG_TYPE=hadoop AMBER_TEST_FILTER=skip-integration sbt -batch \ "WorkflowExecutionService/testOnly org.apache.texera.amber.engine.common.FriesReconfigurationAlgorithmSpec org.apache.texera.amber.engine.common.storage.HDFSRecordStorageSpec org.apache.texera.amber.engine.common.storage.SequentialRecordStorageSpec org.apache.texera.amber.engine.common.storage.VFSRecordStorageSpec org.apache.texera.web.resource.dashboard.admin.execution.AdminExecutionResourceSpec org.apache.texera.web.service.ExecutionConsoleServiceSpec org.apache.texera.web.service.ExecutionReconfigurationServiceSpec org.apache.texera.web.service.ExecutionRuntimeServiceSpec org.apache.texera.web.service.ExecutionStatsServiceSpec org.apache.texera.web.service.ExecutionsMetadataPersistServiceSpec org.apache.texera.web.service.ResultExportServiceSpec org.apache.texera.web.service.WorkflowExecutionServiceSpec"Observed result:
12suites completed,194tests succeeded (0failed,0canceled,0skipped). JUnit XML across all25suites (24modified plusExecutionRuntimeServiceSpec):298tests,0failures,0errors,0skipped.Was this PR authored or co-authored using generative AI tooling?
Generated-by: Antigravity (Gemini 4 Neon)