Skip to content

test(amber): share engine and service test fixtures - #9534

Merged
aglinxinyuan merged 1 commit into
apache:mainfrom
aglinxinyuan:refactor/repo-dedup-cleanup
Nov 19, 2026
Merged

aglinxinyuan merged 1 commit into
apache:mainfrom
aglinxinyuan:refactor/repo-dedup-cleanup

Conversation

@aglinxinyuan

@aglinxinyuan aglinxinyuan commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Consolidate repeated scheduling, coordinator handler, worker checkpoint, storage, replay, reconfiguration, and execution-service test scaffolding across amber/src/test/scala/ (+1,416 / −2,209 lines across 28 test files: 25 modified, 3 added) without modifying production code or test assertions:

  • BarePhysicalPlanFixture: Extracts shared bare PhysicalOp, Region, PhysicalLink, and GlobalPort builders across WorkflowExecutionSpec, JumpToOperatorRegionHandlerSpec, RegionPlanSpec, RegionSpec, ScheduleSpec, and SchedulingUtilsSpec.
  • CheckpointHandlerSpecSupport: Shares Pekko TestKit / SingleNodeListener lifecycle, liveWorker and handler wiring, and PlainExecutor scaffolding across PrepareCheckpointHandlerSpec and FinalizeCheckpointHandlerSpec.
  • ExecutionDatabaseTestSupport: Extracts shared database and workflow-execution seeding helpers across DefaultCostEstimatorSpec, AdminExecutionResourceSpec, ExecutionsMetadataPersistServiceSpec, ResultExportServiceSpec, and WorkflowExecutionServiceSpec.
  • RecordStorageTestSupport: Shares temporary-folder lifecycle, record I/O helpers, and storage contract checks across SequentialRecordStorageSpec, HDFSRecordStorageSpec, and VFSRecordStorageSpec.
  • CoordinatorMessageFixture & RecordingExecutionClient: Adopts CoordinatorMessageFixture in EmbeddedControlMessageHandlerSpec and RetrieveWorkflowStateHandlerSpec, and adds shared batch event/command helpers on RecordingExecutionClient for ExecutionConsoleServiceSpec and ExecutionStatsServiceSpec.
  • Scoped suite helpers (DPThreadSpec, FriesReconfigurationAlgorithmSpec, ReplayLogGeneratorSpec, ExecutionReconfigurationServiceSpec): Consolidates repeated worker polling/lifecycle drivers, FRIES scenario registration, replay log generation with nested writer cleanup, and reconfiguration stub/dispatch assertions inside each spec.

Any related issues, documentation, discussions?

Closes #9533.

How was this PR tested?

Verified on JDK 17 after final commit d399f721b0 (STORAGE_ICEBERG_CATALOG_TYPE=hadoop, AMBER_TEST_FILTER=skip-integration, UDF_PYTHON_PATH configured):

sbt -batch "WorkflowExecutionService/scalafmtCheckAll" "WorkflowExecutionService/scalafixAll --check"

Observed result: both checks exited 0.

STORAGE_ICEBERG_CATALOG_TYPE=hadoop AMBER_TEST_FILTER=skip-integration sbt -batch \
  "WorkflowExecutionService/testOnly org.apache.texera.amber.engine.architecture.coordinator.execution.WorkflowExecutionSpec org.apache.texera.amber.engine.architecture.coordinator.promisehandlers.EmbeddedControlMessageHandlerSpec org.apache.texera.amber.engine.architecture.coordinator.promisehandlers.JumpToOperatorRegionHandlerSpec org.apache.texera.amber.engine.architecture.coordinator.promisehandlers.RetrieveWorkflowStateHandlerSpec org.apache.texera.amber.engine.architecture.logreplay.ReplayLogGeneratorSpec org.apache.texera.amber.engine.architecture.scheduling.DefaultCostEstimatorSpec org.apache.texera.amber.engine.architecture.scheduling.RegionPlanSpec org.apache.texera.amber.engine.architecture.scheduling.RegionSpec org.apache.texera.amber.engine.architecture.scheduling.ScheduleSpec org.apache.texera.amber.engine.architecture.scheduling.SchedulingUtilsSpec org.apache.texera.amber.engine.architecture.worker.DPThreadSpec org.apache.texera.amber.engine.architecture.worker.promisehandlers.FinalizeCheckpointHandlerSpec org.apache.texera.amber.engine.architecture.worker.promisehandlers.PrepareCheckpointHandlerSpec"

Observed result: 13 suites completed, 104 tests succeeded (0 failed, 0 canceled, 0 skipped).

STORAGE_ICEBERG_CATALOG_TYPE=hadoop AMBER_TEST_FILTER=skip-integration sbt -batch \
  "WorkflowExecutionService/testOnly org.apache.texera.amber.engine.common.FriesReconfigurationAlgorithmSpec org.apache.texera.amber.engine.common.storage.HDFSRecordStorageSpec org.apache.texera.amber.engine.common.storage.SequentialRecordStorageSpec org.apache.texera.amber.engine.common.storage.VFSRecordStorageSpec org.apache.texera.web.resource.dashboard.admin.execution.AdminExecutionResourceSpec org.apache.texera.web.service.ExecutionConsoleServiceSpec org.apache.texera.web.service.ExecutionReconfigurationServiceSpec org.apache.texera.web.service.ExecutionRuntimeServiceSpec org.apache.texera.web.service.ExecutionStatsServiceSpec org.apache.texera.web.service.ExecutionsMetadataPersistServiceSpec org.apache.texera.web.service.ResultExportServiceSpec org.apache.texera.web.service.WorkflowExecutionServiceSpec"

Observed result: 12 suites completed, 194 tests succeeded (0 failed, 0 canceled, 0 skipped). JUnit XML across all 25 suites (24 modified plus ExecutionRuntimeServiceSpec): 298 tests, 0 failures, 0 errors, 0 skipped.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Antigravity (Gemini 4 Neon)

Copilot AI balanced review requested due to automatic review settings November 19, 2026 04:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@github-actions github-actions Bot added engine refactor Refactor the code frontend Changes related to the frontend GUI common platform Non-amber Scala service paths labels Nov 19, 2026
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @Yicong-Huang, @Neilk1021, @anthonychengit
    You can notify them by mentioning @Yicong-Huang, @Neilk1021, @anthonychengit in a comment.

@aglinxinyuan
aglinxinyuan requested a review from mengw15 November 19, 2026 05:01
@codecov-commenter

codecov-commenter commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.55%. Comparing base (0aea3df) to head (37193fb).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #9534      +/-   ##
============================================
- Coverage     91.57%   91.55%   -0.02%     
+ Complexity     5157     5103      -54     
============================================
  Files          1300     1304       +4     
  Lines         58092    57400     -692     
  Branches       7615     7560      -55     
============================================
- Hits          53195    52550     -645     
+ Misses         2959     2919      -40     
+ Partials       1938     1931       -7     
Flag Coverage Δ *Carryforward flag
access-control-service 77.44% <ø> (ø)
agent-service 99.56% <ø> (ø) Carriedforward from 0aea3df
amber 85.30% <ø> (-0.13%) ⬇️ Carriedforward from 0aea3df
computing-unit-managing-service 63.88% <ø> (ø)
config-service 87.50% <ø> (ø)
file-service 82.16% <ø> (ø)
frontend 96.43% <ø> (-0.03%) ⬇️
notebook-migration-service 83.77% <ø> (ø)
pyamber 98.38% <ø> (ø) Carriedforward from 0aea3df
result-service 85.71% <ø> (ø)
workflow-compiling-service 78.30% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@aglinxinyuan
aglinxinyuan removed the request for review from mengw15 November 19, 2026 05:02
@aglinxinyuan
aglinxinyuan marked this pull request as draft November 19, 2026 05:02
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 7 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main 0aea3df benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 422 0.258 23,387/34,970/34,970 us 🔴 +38.8% / 🔴 +173.0%
🔴 bs=100 sw=10 sl=64 865 0.528 114,152/138,377/138,377 us 🔴 +7.0% / 🔴 +69.2%
⚪ bs=1000 sw=10 sl=64 965 0.589 1,032,758/1,135,914/1,135,914 us ⚪ within ±5% / 🔴 +48.7%
Baseline details

Latest main 0aea3df from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 422 tuples/sec 479 tuples/sec 1,084 tuples/sec -11.9% -61.1%
bs=10 sw=10 sl=64 MB/s 0.258 MB/s 0.292 MB/s 0.662 MB/s -11.6% -61.0%
bs=10 sw=10 sl=64 p50 23,387 us 21,973 us 9,548 us +6.4% +144.9%
bs=10 sw=10 sl=64 p95 34,970 us 25,189 us 12,811 us +38.8% +173.0%
bs=10 sw=10 sl=64 p99 34,970 us 25,189 us 15,432 us +38.8% +126.6%
bs=100 sw=10 sl=64 throughput 865 tuples/sec 900 tuples/sec 1,467 tuples/sec -3.9% -41.0%
bs=100 sw=10 sl=64 MB/s 0.528 MB/s 0.549 MB/s 0.895 MB/s -3.8% -41.0%
bs=100 sw=10 sl=64 p50 114,152 us 110,279 us 74,441 us +3.5% +53.3%
bs=100 sw=10 sl=64 p95 138,377 us 129,364 us 81,800 us +7.0% +69.2%
bs=100 sw=10 sl=64 p99 138,377 us 129,364 us 103,712 us +7.0% +33.4%
bs=1000 sw=10 sl=64 throughput 965 tuples/sec 977 tuples/sec 1,517 tuples/sec -1.2% -36.4%
bs=1000 sw=10 sl=64 MB/s 0.589 MB/s 0.596 MB/s 0.926 MB/s -1.2% -36.4%
bs=1000 sw=10 sl=64 p50 1,032,758 us 1,022,947 us 725,911 us +1.0% +42.3%
bs=1000 sw=10 sl=64 p95 1,135,914 us 1,106,339 us 764,152 us +2.7% +48.7%
bs=1000 sw=10 sl=64 p99 1,135,914 us 1,106,339 us 808,901 us +2.7% +40.4%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,473.72,200,128000,422,0.258,23386.99,34969.94,34969.94
1,100,10,64,20,2312.20,2000,1280000,865,0.528,114151.81,138376.64,138376.64
2,1000,10,64,20,20731.99,20000,12800000,965,0.589,1032758.11,1135914.06,1135914.06

Consolidate repeated scheduling, coordinator handler, worker checkpoint, storage, replay, reconfiguration, and execution-service test scaffolding across 28 Amber test files (25 modified, 3 added; +1,416 / -2,209 lines) without altering production code or test assertions.
@aglinxinyuan
aglinxinyuan force-pushed the refactor/repo-dedup-cleanup branch from 37193fb to d399f72 Compare November 19, 2026 05:09
@aglinxinyuan aglinxinyuan changed the title refactor(amber, workflow-operator, frontend): share code test(amber): share engine and service test fixtures Nov 19, 2026
@github-actions github-actions Bot removed frontend Changes related to the frontend GUI common platform Non-amber Scala service paths labels Nov 19, 2026
@aglinxinyuan
aglinxinyuan requested a review from mengw15 November 19, 2026 05:13
@aglinxinyuan
aglinxinyuan marked this pull request as ready for review November 19, 2026 05:13
@aglinxinyuan
aglinxinyuan requested a review from aicam November 19, 2026 05:13

@mengw15 mengw15 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aglinxinyuan
aglinxinyuan added this pull request to the merge queue Nov 19, 2026
Merged via the queue into apache:main with commit f020980 Nov 19, 2026
28 checks passed
@aglinxinyuan
aglinxinyuan deleted the refactor/repo-dedup-cleanup branch November 19, 2026 05:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engine refactor Refactor the code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Share Amber engine and execution service test fixtures

4 participants