fix(manager): AlarmPersistenceManager 修 stuck-active silent monitoring blackout

[csp0924]

Summary

• 修 AlarmPersistenceManager _create_alarm 在 is_new=False 時 silent skip → audit history 漏記
• Production scenario:Mongo replica 短暫掉線 → resolve fail → DB 殘留 ACTIVE → 該 device 從此停發任何 audit / alert,維運完全察覺不到 silent monitoring blackout
• Notification 維持 dedupe(保留 v0.8.x 合約,避免 spam),但 history 不去重(每 event 都記)

Why(觸發路徑)

t=0: disconnect → upsert(is_new=True)  → ACTIVE 記錄 + notify + history(triggered)
t=1: connect    → resolve()            → modify_count=0 (transient Mongo error) → False
                                       → DB 仍 ACTIVE,無 resolved notify、無 resolved history
t=2: disconnect → upsert(is_new=False) → ❌ SILENT(無 history、無 notify、無 log)
t=3+: 永久 silent,直到某次 resolve 巧合成功

upsert(is_new=False) 不代表 disconnect 沒發生,只代表 DB 已有 ACTIVE 記錄。is_new 是 record state,不是 event state。將兩者混淆是 root cause。

Invariant(修法後合約)

• is_new=True → INFO log + notify + history event=\"triggered\"
• is_new=False → WARNING log + ❌ 不 notify(維持 dedupe) + history event=\"duplicate_trigger\"
• audit history 必須每 event 一筆,不因 record-state dedupe 而漏記

Closed-loop sandbox 驗證

Scenario	resolve_fail_rate	disc	hist trig/dup	audit_blackout	silent_notify
A baseline	0.0	20	20/0	0 ✓	0
B transient	0.3	20	17/3	0 ✓	3(預期 dedupe)
C permanent	1.0	20	1/19	0 ✓	19(預期 dedupe)

修法前:C 場景只記 1 個 history record + 19 silent。修法後:20 個 record 完整覆蓋。

Test plan

• 4854 tests passed(本地全 regression)
• 新增 TestStuckActiveSilentBlackoutRegression 三個 test:
  • test_resolve_fail_then_subsequent_disconnect_still_writes_history — 兩 cycle 重現
  • test_continuous_stuck_blackout_quantified — N=5 cycle 量化驗證(1 triggered + 4 duplicate)
  • test_duplicate_trigger_does_not_notify — dedupe 合約保護
• 既有 test_existing_alarm_does_not_write_history 更新為 test_existing_alarm_writes_duplicate_trigger_history(合約改動)
• ruff + mypy + format 全通過

Compatibility(合約變更)

• buffered_uploader history collection 新增 event=\"duplicate_trigger\" 文檔類型
  • caller 若 query event=\"triggered\" 不會誤包(明確 query)
  • caller 若 query 全部 trigger event,需改 query 包含 event in [\"triggered\", \"duplicate_trigger\"]
• API signature 不變:_create_alarm 是 private,public AlarmPersistenceManager 介面零變動
• Notification 行為不變:仍按 is_new=True 才發,維持 dedupe spam 設計

bump:patch(fix:)

Background — closed-loop emergent bug

pytest 單測各 method 獨立,抓不到這條 multi-cycle stateful chain (trigger → resolve-fail → trigger-suppressed)。需用 plant model(FakeRepository with resolve_fail_rate)模擬 multi-cycle 才能 surface。第 4 個 csp_lib closed-loop emergent bug 案例(前 3:PowerCompensator、Droop chatter、Modbus queue dead branch、SetpointDriftReconciler audit spam)。

[Copilot]

Pull request overview

This PR fixes a silent “stuck ACTIVE” alarm scenario in AlarmPersistenceManager where upsert(is_new=False) previously caused trigger events to be silently dropped (no audit history / no notify), leading to monitoring blackout after transient resolve failures.

Changes:

• Update alarm creation flow to always write an audit history record for every trigger event, using event="triggered" for new alarms and event="duplicate_trigger" for duplicate triggers (while keeping notification dedupe behavior).
• Add/adjust tests to validate the new history contract and provide multi-cycle regression coverage for the stuck-ACTIVE blackout chain.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
csp_lib/manager/alarm/persistence.py	Writes history for duplicate triggers (duplicate_trigger) while keeping notifications deduped; adds warning log for duplicate trigger events.
tests/manager/test_persistence_history.py	Updates existing expectations and adds regression tests covering resolve-fail → subsequent disconnect still writes history and does not notify.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.