Repository navigation
refactor(amber): use region links in Fries for cache reuse - #8734
Xiao-zhen-Liu wants to merge 1 commit into
Conversation
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8734 +/- ##
============================================
- Coverage 91.57% 91.57% -0.01%
+ Complexity 5159 5158 -1
============================================
Files 1300 1300
Lines 58092 58093 +1
Branches 7615 7615
============================================
- Hits 53198 53197 -1
- Misses 2958 2959 +1
- Partials 1936 1937 +1
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 520 | 0.318 | 19,108/23,165/23,165 us | 🔴 -6.8% / 🔴 +103.3% |
| 🔴 | bs=100 sw=10 sl=64 | 1,248 | 0.762 | 79,497/95,082/95,082 us | 🔴 +5.5% / 🔴 +18.7% |
| 🔴 | bs=1000 sw=10 sl=64 | 1,441 | 0.88 | 686,623/842,034/842,034 us | 🔴 +8.7% / 🔴 +12.4% |
Baseline details
Latest main 7f9bdf3 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 520 tuples/sec | 558 tuples/sec | 1,092 tuples/sec | -6.8% | -52.4% |
| bs=10 sw=10 sl=64 | MB/s | 0.318 MB/s | 0.341 MB/s | 0.666 MB/s | -6.7% | -52.3% |
| bs=10 sw=10 sl=64 | p50 | 19,108 us | 18,111 us | 9,398 us | +5.5% | +103.3% |
| bs=10 sw=10 sl=64 | p95 | 23,165 us | 23,124 us | 12,941 us | +0.2% | +79.0% |
| bs=10 sw=10 sl=64 | p99 | 23,165 us | 23,124 us | 15,496 us | +0.2% | +49.5% |
| bs=100 sw=10 sl=64 | throughput | 1,248 tuples/sec | 1,278 tuples/sec | 1,486 tuples/sec | -2.3% | -16.0% |
| bs=100 sw=10 sl=64 | MB/s | 0.762 MB/s | 0.78 MB/s | 0.907 MB/s | -2.3% | -16.0% |
| bs=100 sw=10 sl=64 | p50 | 79,497 us | 75,381 us | 72,544 us | +5.5% | +9.6% |
| bs=100 sw=10 sl=64 | p95 | 95,082 us | 99,041 us | 80,069 us | -4.0% | +18.7% |
| bs=100 sw=10 sl=64 | p99 | 95,082 us | 99,041 us | 101,531 us | -4.0% | -6.4% |
| bs=1000 sw=10 sl=64 | throughput | 1,441 tuples/sec | 1,470 tuples/sec | 1,533 tuples/sec | -2.0% | -6.0% |
| bs=1000 sw=10 sl=64 | MB/s | 0.88 MB/s | 0.897 MB/s | 0.936 MB/s | -1.9% | -6.0% |
| bs=1000 sw=10 sl=64 | p50 | 686,623 us | 674,334 us | 711,170 us | +1.8% | -3.5% |
| bs=1000 sw=10 sl=64 | p95 | 842,034 us | 774,648 us | 749,276 us | +8.7% | +12.4% |
| bs=1000 sw=10 sl=64 | p99 | 842,034 us | 774,648 us | 794,786 us | +8.7% | +5.9% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,384.40,200,128000,520,0.318,19107.67,23164.69,23164.69
1,100,10,64,20,1601.99,2000,1280000,1248,0.762,79497.40,95081.58,95081.58
2,1000,10,64,20,13878.01,20000,12800000,1441,0.880,686623.48,842034.37,842034.37|
@parshimers @sshiv012 could you review this one too? It is small and independent of the other three. Design discussion: #5880. Thanks! |
|
This pull request has been automatically marked as stale because it has If this PR is still relevant, please:
To exempt this PR from auto-close, a maintainer can add the |
|
This pull request has been automatically closed due to 37 days of inactivity. |
3ff6eb5 to
5c63d6f
Compare
What changes were proposed in this PR?
This PR is part of cache reuse (#5881), where an execution reads the saved results of an earlier run instead of computing them again. When
FriesReconfigurationAlgorithmcomputes a reconfiguration's scope, it filters candidate port links withregion.getLinks.contains(link)(Region.physicalLinks) instead of following every link listed on an operator's port objects.region.getLinksbehaviormain)R -> U1 -> U2,Split -> U1 / U2,#8733)SplitandU1still list theCacheReadlink onSplit:0R_1.getLinks, so reconfiguring{Split, U2}skipsSplit:0 -> U1and leavesU1untouched.Any related issues, documentation, discussions?
Part of #5881 (cache reuse). Advances #5884. Design discussion: #5880. Related: #8733, where
CostBasedScheduleGeneratorleavesCacheReadlinks out ofRegion.physicalLinkswhile operators keep their full link lists.How was this PR tested?
FriesReconfigurationAlgorithmSpecadds two cases where a link listed only on operator ports would otherwise pull an unrelated operator into a reconfiguration's scope, while keeping all existing reconfiguration cases green.Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude (Claude Code), Gemini (Antigravity)