fix(statistics): IntervalAccumulator boundary 不再系統性低估 kWh#146
Open
csp0924 wants to merge 2 commits into
Open
Conversation
IntervalAccumulator._finalize 之前直接以 _kwh_accumulated（INSTANTANEOUS）或
last - first（CUMULATIVE）結算當下 interval，未補齊 boundary 前最後一個 sample
到 boundary 之間的尾段；CUMULATIVE 也只取最後 sample 而非 boundary 對應值。
emergent 後果是每 interval 都低估，shortfall_ratio ≈ sample_period / interval_size：
- 60 s 取樣 / 15 min interval：低估 1/15 ≈ 6.67% (2.333 vs 2.5 kWh)
- 180 s 取樣 / 15 min interval：低估 20% (2.000 vs 2.5 kWh)
- 5 s 取樣 / 15 min interval：低估 0.56%
修復策略（boundary-aligned linear interpolation）：
feed() 偵測到 timestamp >= boundary 時，以 prev sample 與剛抵達的 sample 對
boundary 瞬間做線性內插：
v_b = prev_value + (value - prev_value) * (boundary - prev_ts) / (ts - prev_ts)
- INSTANTANEOUS：補上 trapezoid 尾段 (prev_value + v_b)/2 * (boundary - prev_ts).hours
- CUMULATIVE：kwh = v_b - first_value
- 下一個 interval 以 (v_b, boundary) 為合成種子 sample 開啟（不計入 sample_count），
再 accumulate 真實抵達的 sample → 跨 interval 連續、無 leakage
sandbox/statistics_tracker_boundary_truncation_demo.py 重跑七種 scenario
（60s/30s/5s/180s × CUMULATIVE/INSTANTANEOUS）全部 shortfall = 0.00%。
constant 信號完全精確；ramp 信號是兩 sample 內插的二階近似誤差。
公開 API 不變：IntervalRecord shape 不變、feed/sample_count/period_start 簽名不變。
依 patch 版本約束，所有上層呼叫者不需任何修改。
新測試：tests/statistics/test_tracker_boundary.py 8 案例覆蓋 60s/180s/5s
constant power + first-interval-partial + cross-interval continuity。
既有測試 4 個 case 因含 boundary cross 對應 kwh 預期值更新（記錄修復後的正確語義）。
There was a problem hiding this comment.
Pull request overview
This PR fixes IntervalAccumulator energy under-reporting at interval boundaries by interpolating the boundary value and carrying it forward as the next interval seed.
Changes:
- Adds boundary interpolation and tail-segment accounting in
IntervalAccumulator. - Updates existing cumulative/instantaneous statistics expectations.
- Adds regression tests for boundary truncation and first partial interval behavior.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
csp_lib/statistics/tracker.py |
Implements boundary interpolation and seeded interval continuation. |
tests/statistics/test_tracker.py |
Updates accumulator expectations for boundary-interpolated kWh. |
tests/statistics/test_tracker_boundary.py |
Adds regression coverage for constant-power boundary accuracy. |
tests/statistics/test_engine.py |
Updates engine boundary-crossing expectation. |
tests/statistics/test_manager.py |
Updates upload boundary-crossing expectation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+120
to
+123
| self._period_start = self._floor_timestamp(timestamp) | ||
| self._reset() | ||
| self._start_sample(value, timestamp) | ||
| self._seed_from_boundary(value_at_boundary, boundary) | ||
| self._accumulate(value, timestamp) |
當 feed() timestamp 跨越 >1 個 interval boundary（取樣斷線後恢復），舊版只 finalize 第一個跨越的 interval，然後把 _period_start 跳到 floor(timestamp)，但 seed 用的是 舊 boundary 的內插值。後續 _accumulate(value, timestamp) 會以 (prev=v_b@舊 boundary, prev_ts=舊 boundary) 為起點對 timestamp 算 trapezoid / 差值，把整個 gap 段的能量 全部灌進新 interval（silent corruption）。 實際數值：恆定 10 kW、15 min interval，12:00 餵一筆後沉默到 12:46 才再餵一筆。 13:01 完成 [12:45, 13:00] 時舊版回 7.5 kWh，正確應為 2.5 kWh（300% 灌水）。 修復：seed 新 interval 改用 floor(timestamp) 對應時刻的內插值（而非舊 boundary 的內插值）。新 interval 從 period_start 起算，trapezoid / 差值僅覆蓋 [period_start, timestamp]。中間被跳過的 interval 不 emit record（缺乏足夠資訊 重建 partial interval）。 回歸測試：tests/statistics/test_tracker_boundary.py::TestMultiIntervalSkip 三個案例 覆蓋 INSTANTANEOUS / CUMULATIVE / period_start floor 行為。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
IntervalAccumulator._finalize結算 interval 時，漏算_prev_timestamp → boundary的 trapezoid 尾段：kwh = last - first，last不在 boundary 上，跨 interval 不連續通式：每 interval 漏
sample_period / interval_size的能量。Closed-loop sandbox 量化（10 kW 定值 × 15 min interval）
每個部署都受影響，與失效模式無關（不是 edge case，是 default path）。月度能源報表系統性低估。
Fix（boundary linear interpolation）
feed(value, ts)偵測ts >= boundary時，用 prev sample + 剛抵達 sample 對 boundary 線性內插出v_b：(prev_value + v_b) / 2 * (boundary - prev_ts).hourskwh = v_b - first_value（取代last - first）(v_b, boundary)為合成種子（不計入 sample_count）— 跨 interval 連續選用內插非外推（ZOH）：到貨 sample 是真實量測非假設；CUMULATIVE/INSTANTANEOUS 語意對稱；constant 信號完全精確，ramp 比 ZOH 更準。
Tests
tests/statistics/test_tracker_boundary.py（新檔，8 case）— INSTANTANEOUS × 5s/60s/180s、CUMULATIVE × 60s/180s/跨 interval 連續、首 interval partial period caveat。注意：6 個既有 test 預期值修正（pre-existing test 把 bug 答案 lock-in 當期望值）：
test_tracker.py::TestCumulativeAccumulator::test_boundary_crossing_returns_record(10.0 → 18.333)test_tracker.py::TestCumulativeAccumulator::test_multiple_intervals(0.0/20.0 → 9.286/29.048)test_tracker.py::TestCumulativeAccumulator::test_zero_energy_when_single_sample(0.0 → 5.0)test_tracker.py::TestInstantaneousAccumulator::test_trapezoidal_integration/test_varying_power/test_multiple_samples（補尾段值）test_engine.py::test_process_read_boundary_crossing(10.0 → 18.333)test_manager.py::test_upload_on_boundary_crossing(10.0 → 18.333)這 6 個的舊預期值就是 bug 本身——譬如
boundary_crossing_returns_record期待 10 kWh 但理論值是 18.333 kWh。Public API
IntervalRecordshape、feed簽名、period_start/sample_count屬性、上層 caller 全部零改動 → patch 版本。Quality gates