From 533e3b44eb6a781f407261b1f8df498f070ca45a Mon Sep 17 00:00:00 2001 From: abdulsaheel Date: Sun, 26 Jul 2026 18:08:57 +0530 Subject: [PATCH 1/5] fix: close data-loss, privacy and absent-vs-zero defects across the app MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An audit of the whole tree surfaced defects clustered in three seams the 886-test suite never reached: failure paths, absent-vs-zero coercion, and guards modelled with regexes and blocklists. Every fix below carries a regression test that fails against the old code. BLE — the safe-trim invariant held by ordering but not by errors DrainController.commit() cleared its buffer, then swallowed the durable commit's exception and returned void; the caller ACKed regardless, so the band trimmed flash for records that were never persisted. db.dart's own comment documents the OOM that makes this reachable (Crashlytics 0.9.13). commit() now reports failure, re-buffers at the front, and the ACK is gated on a TrimAckPolicy verdict. The drain path also had no session guard — the only two async paths lacking the check four siblings make — so a stale drainer could ACK an old token onto a new link. discardOpenChunk now poisons its burst, so a straggler terminal can't trim discarded records. Also: a failed connect no longer holds the band claim forever (starving the headless drainer), link-down tears the session down instead of leaking five timers and four subscriptions per drop, a corrupt far-future RTC is refused, and the Android boot wake goes through the headless gate. Persistence — three ways a good day could be destroyed - _markDaySkipped REPLACEd a real day_result with a skip marker on one 90 s isolate timeout, finalized, unrecoverable once raw was pruned. - Re-analyzing a day whose raw is gone derived an empty bundle with endSec 0, so it always looked finalizable and NULLed every scalar. - The per-sweep baseline snapshot was mutated by appendScalars, so each finished day polluted the window the next one read. None of the three can now overwrite a real result. Also: exportDaysDb passed onCreate with no version, so it threw ArgumentError on every call and had never worked; two unguarded ALTER TABLE ADD COLUMN collided with modernized CREATE helpers and bricked upgrades from old schemas permanently; the orphan guard missed the counter-PK eviction and decodedRrByCounterRange returned zero rows across a counter reset; deleteDays never cascaded workout_route; and local days were assumed to be exactly 86400 s. Privacy The coach SQL guard modelled table references with a regex that ignored comma cross-joins, and its blocklist was missing five real tables, so `SELECT r.lat, r.lng FROM v_sessions s, workout_route r` returned GPS coordinates into an LLM prompt. Replaced with a lexer + allow-list parser plus a structural gate that derives the permitted btree root pages by EXPLAINing the coach views, so a table added later is closed by default. Firebase collected by default: no collection flags existed in the plist or manifest, and the only gate ran after an async prefs read, so a fresh install auto-logged sessions and uploaded startup crashes before consent was read. Collection is now off at the platform level and enabled only after consent resolves. Absent is not zero "Wear stretches" read _n(segments) on a List and rendered 0 every day on every device. TrendBoard discarded the `has` flag the repository emits to mark absent days, so unworn days drew real bars — and the oxygen board printed a literal "0" for an unmeasured night. getDayWear collapsed "never measured" into worn_min 0, so an imported day asserted "not worn". Charts now take nullable values and render gaps as gaps. DeltaChip was also being fed an absolute delta and rendering it as a percentage, so sleeping 20 minutes more than last week displayed "20.0%". State Unpair could be silently undone by a late engine-state callback re-creating the pairing from a stale serial. The once-per-day notification guard was consumed before the notification was presented, so a recovery alert suppressed by quiet hours never fired and never retried. A denied notification permission was cached process-wide with no invalidation, and schedules added absolute Durations to TZDateTime and drifted across DST. kAlgoVersion 47 -> 48. The sibling analytics fixes from the same sweep are deliberately NOT claimed in that changelog: pubspec.yaml still pins the previous analytics SHA, and citing changes a pinned SHA does not contain is how v43 documented a readiness fix that stayed broken for three releases. The pin must be updated, and the version bumped again, once the analytics change merges. --- android/app/src/main/AndroidManifest.xml | 31 + .../openstrap_edge/OpenStrapWidgetProvider.kt | 5 +- .../WatchMetrics.swift | 6 +- ios/OpenStrapWidget/OpenStrapWidget.swift | 6 +- ios/Runner/Info.plist | 25 + lib/app.dart | 7 + lib/ble/ble_engine.dart | 601 +++++++++++---- lib/ble/ble_state.dart | 193 +++++ lib/cloud/cloud_import.dart | 24 +- lib/coach/coach_db.dart | 441 +++++++++-- lib/coach/coach_engine.dart | 94 ++- lib/coach/coach_prompt.dart | 13 +- lib/compute/crossday_pipeline.dart | 96 ++- lib/compute/derivation_engine.dart | 563 +++++++++++--- lib/compute/derive_prepare.dart | 84 +- lib/compute/onehz_pipeline.dart | 28 +- lib/compute/substrate.dart | 49 +- lib/data/day_label.dart | 46 ++ lib/data/db.dart | 724 ++++++++++++------ lib/data/local_repository_impl.dart | 103 ++- lib/gps/route_math.dart | 10 +- lib/gps/route_tracker.dart | 35 +- lib/import/noop_import.dart | 68 +- lib/import/whoop_import.dart | 150 +++- lib/main.dart | 12 +- lib/notify/notification_center.dart | 48 +- lib/notify/notification_event.dart | 18 +- lib/notify/notification_ids.dart | 176 +++++ lib/notify/notification_service.dart | 135 +++- lib/state/app_state.dart | 267 +++++-- lib/sync/headless_boot.dart | 49 +- lib/sync/sync_policy.dart | 15 + lib/telemetry/telemetry_service.dart | 60 ++ lib/ui/ai/ai_breakdown_screen.dart | 10 + lib/ui/ai/ai_settings_screen.dart | 24 +- lib/ui/coach/ai_coach_screen.dart | 14 +- lib/ui/coach/coach_chart.dart | 5 +- lib/ui/design/recap_card.dart | 13 +- lib/ui/kit/charts.dart | 196 +++-- lib/ui/kit/kit.dart | 15 +- lib/ui/screens/detail_cards.dart | 150 +++- lib/ui/screens/metric_screen.dart | 136 +++- lib/ui/screens/screens.dart | 16 +- lib/ui/screens/trend_screen.dart | 33 +- lib/ui/widgets/async_guards.dart | 51 ++ lib/widget/widget_service.dart | 7 +- pubspec.lock | 4 +- pubspec.yaml | 8 + test/absent_not_zero_test.dart | 548 +++++++++++++ test/ai_screen_async_guards_test.dart | 128 ++++ test/app_state_regressions_test.dart | 306 ++++++++ test/ble_engine_test.dart | 35 - test/ble_safe_trim_test.dart | 370 +++++++++ test/ble_transport_guards_test.dart | 217 ++++++ test/coach_provider_response_test.dart | 178 +++++ test/coach_sql_guard_adversarial_test.dart | 147 ++++ test/coach_sql_structural_test.dart | 109 +++ test/crossday_pipeline_test.dart | 121 +++ test/day_window_dst_test.dart | 198 +++++ test/db_migration_ladder_test.dart | 296 +++++++ test/db_p0_fixes_test.dart | 471 ++++++++++++ test/derive_day_window_test.dart | 234 ++++++ test/derive_isolate_lifecycle_test.dart | 237 ++++++ test/derive_result_protection_test.dart | 304 ++++++++ test/headless_gate_test.dart | 78 +- test/import_data_safety_test.dart | 320 ++++++++ test/local_repository_p0_test.dart | 135 ++++ test/metric_trend_redesign_test.dart | 7 +- test/notification_center_test.dart | 25 +- test/notification_day_guard_test.dart | 201 +++++ test/notification_ids_test.dart | 120 +++ test/notification_permission_test.dart | 127 +++ test/notification_schedule_dst_test.dart | 139 ++++ test/readiness_baseline_pollution_test.dart | 79 ++ test/route_math_test.dart | 51 ++ test/route_tracker_test.dart | 78 ++ test/telemetry_consent_default_test.dart | 149 ++++ test/widget_service_sentinels_test.dart | 70 ++ 78 files changed, 9382 insertions(+), 960 deletions(-) create mode 100644 lib/notify/notification_ids.dart create mode 100644 lib/ui/widgets/async_guards.dart create mode 100644 test/absent_not_zero_test.dart create mode 100644 test/ai_screen_async_guards_test.dart create mode 100644 test/app_state_regressions_test.dart create mode 100644 test/ble_safe_trim_test.dart create mode 100644 test/ble_transport_guards_test.dart create mode 100644 test/coach_provider_response_test.dart create mode 100644 test/coach_sql_guard_adversarial_test.dart create mode 100644 test/coach_sql_structural_test.dart create mode 100644 test/day_window_dst_test.dart create mode 100644 test/db_migration_ladder_test.dart create mode 100644 test/db_p0_fixes_test.dart create mode 100644 test/derive_day_window_test.dart create mode 100644 test/derive_isolate_lifecycle_test.dart create mode 100644 test/derive_result_protection_test.dart create mode 100644 test/import_data_safety_test.dart create mode 100644 test/local_repository_p0_test.dart create mode 100644 test/notification_day_guard_test.dart create mode 100644 test/notification_ids_test.dart create mode 100644 test/notification_permission_test.dart create mode 100644 test/notification_schedule_dst_test.dart create mode 100644 test/telemetry_consent_default_test.dart create mode 100644 test/widget_service_sentinels_test.dart diff --git a/android/app/src/main/AndroidManifest.xml b/android/app/src/main/AndroidManifest.xml index c0dd590c..22b93dbd 100644 --- a/android/app/src/main/AndroidManifest.xml +++ b/android/app/src/main/AndroidManifest.xml @@ -74,6 +74,37 @@ android:label="Edge" android:name=".EdgeApplication" android:icon="@mipmap/launcher_icon"> + + + + + + + 0, so an unknown need leaves it empty + // instead of filling against a fabricated 8h denominator. + val needMin = w.readInt(prefs, "sleep_need_min", -1) val hrv = w.readInt(prefs, "hrv", -1) val hrvBaseline = w.readInt(prefs, "hrv_baseline", -1) diff --git a/ios/OpenStrapWatch Watch App/WatchMetrics.swift b/ios/OpenStrapWatch Watch App/WatchMetrics.swift index 93f46414..60cccd63 100644 --- a/ios/OpenStrapWatch Watch App/WatchMetrics.swift +++ b/ios/OpenStrapWatch Watch App/WatchMetrics.swift @@ -19,7 +19,7 @@ struct WatchMetrics { var readiness: Int // 0–100, -1 = none var strain: Double // 0–21, -1 = none var sleepMin: Int // minutes asleep, -1 = none - var needMin: Int // sleep need (min) + var needMin: Int // sleep need (min); -1 = none — never fabricate 8h var hrv: Int // RMSSD ms, -1 = none var hrvBaseline: Int // baseline RMSSD ms, -1 = none var rhr: Int // bpm, -1 = none @@ -29,7 +29,7 @@ struct WatchMetrics { var themeDark: Bool // mirror the app's Ember-on-Paper (false) / Char (true) static let empty = WatchMetrics( - hasData: false, readiness: -1, strain: -1, sleepMin: -1, needMin: 480, + hasData: false, readiness: -1, strain: -1, sleepMin: -1, needMin: -1, hrv: -1, hrvBaseline: -1, rhr: -1, coachLine: "", battPct: -1, updatedAt: 0, themeDark: true) @@ -40,7 +40,7 @@ struct WatchMetrics { readiness: d?.object(forKey: "readiness") as? Int ?? -1, strain: d?.object(forKey: "strain") as? Double ?? -1, sleepMin: d?.object(forKey: "sleep_min") as? Int ?? -1, - needMin: d?.object(forKey: "sleep_need_min") as? Int ?? 480, + needMin: d?.object(forKey: "sleep_need_min") as? Int ?? -1, hrv: d?.object(forKey: "hrv") as? Int ?? -1, hrvBaseline: d?.object(forKey: "hrv_baseline") as? Int ?? -1, rhr: d?.object(forKey: "rhr") as? Int ?? -1, diff --git a/ios/OpenStrapWidget/OpenStrapWidget.swift b/ios/OpenStrapWidget/OpenStrapWidget.swift index b24bfa33..99b57963 100644 --- a/ios/OpenStrapWidget/OpenStrapWidget.swift +++ b/ios/OpenStrapWidget/OpenStrapWidget.swift @@ -58,7 +58,7 @@ struct OpenStrapEntry: TimelineEntry { let readiness: Int // -1 = none (composite 0..100) — the headline let strain: Double // -1 = none let sleepMin: Int // -1 = none - let needMin: Int + let needMin: Int // -1 = none (sleep need, min) — never fabricate 8h let hrv: Int // -1 = none (RMSSD, ms) let hrvBaseline: Int // -1 = none (personal RMSSD baseline, ms) let rhr: Int // -1 = none @@ -106,7 +106,7 @@ private enum Store { readiness: d?.object(forKey: "readiness") as? Int ?? -1, strain: d?.object(forKey: "strain") as? Double ?? -1, sleepMin: d?.object(forKey: "sleep_min") as? Int ?? -1, - needMin: (d?.object(forKey: "sleep_need_min") as? Int) ?? 480, + needMin: (d?.object(forKey: "sleep_need_min") as? Int) ?? -1, hrv: d?.object(forKey: "hrv") as? Int ?? -1, hrvBaseline: d?.object(forKey: "hrv_baseline") as? Int ?? -1, rhr: d?.object(forKey: "rhr") as? Int ?? -1, @@ -171,7 +171,7 @@ private enum TodayAPI { let strain = val(daily, "strain") ?? -1 let rhr = val(daily, "resting_hr").map { Int($0.rounded()) } ?? -1 let sleepMin = val(sleep, "duration_min").map { Int($0.rounded()) } ?? -1 - let needMin = val(sleep, "need_min").map { Int($0.rounded()) } ?? 480 + let needMin = val(sleep, "need_min").map { Int($0.rounded()) } ?? -1 let hrv = (hrvObj?["rmssd"] as? NSNumber).map { Int($0.doubleValue.rounded()) } ?? -1 let hrvBase = (hrvObj?["baseline"] as? NSNumber).map { Int($0.doubleValue.rounded()) } ?? -1 diff --git a/ios/Runner/Info.plist b/ios/Runner/Info.plist index e6ad6b44..3cfe2647 100644 --- a/ios/Runner/Info.plist +++ b/ios/Runner/Info.plist @@ -4,6 +4,31 @@ CADisableMinimumFrameDurationOnPhone + + FIREBASE_ANALYTICS_COLLECTION_ENABLED + + FirebaseCrashlyticsCollectionEnabled + + firebase_performance_collection_enabled + + + GOOGLE_ANALYTICS_DEFAULT_ALLOW_AD_PERSONALIZATION_SIGNALS + + GOOGLE_ANALYTICS_IDFV_COLLECTION_ENABLED + CFBundleDevelopmentRegion $(DEVELOPMENT_LANGUAGE) CFBundleDisplayName diff --git a/lib/app.dart b/lib/app.dart index f0b35985..45bba860 100644 --- a/lib/app.dart +++ b/lib/app.dart @@ -6,6 +6,7 @@ import 'package:provider/provider.dart'; import 'ai/briefing.dart'; import 'coach/coach_config.dart'; +import 'notify/notification_service.dart'; import 'notify/tap_router.dart'; import 'state/app_state.dart'; import 'state/prefs.dart'; @@ -67,6 +68,12 @@ class _OpenStrapAppState extends State with WidgetsBindingObserver void didChangeAppLifecycleState(AppLifecycleState state) { final app = context.read(); if (state == AppLifecycleState.resumed) { + // The user may have flipped our notification switch either way in OS + // Settings while we were backgrounded. Drop the cached authorization + // decision so the next present/schedule re-reads reality — a denial used + // to latch for the whole process, silencing every notification and + // scheduled reminder until a full app restart. + NotificationService.instance.invalidatePermissionCache(); app.maybeFinishFromLiveActivity(); unawaited(app.maybeStopBreathingFromLiveActivity()); app.refreshAppStatus(); // re-check OTA + admin banner on every foreground diff --git a/lib/ble/ble_engine.dart b/lib/ble/ble_engine.dart index 9659e7d2..ed063a64 100644 --- a/lib/ble/ble_engine.dart +++ b/lib/ble/ble_engine.dart @@ -129,17 +129,6 @@ int countBurstTrafficPackets({ unknownCount; } -@visibleForTesting -int nextBurstStablePollStreak({ - required bool queueEmpty, - required int currentCount, - required int previousCount, - required int stableStreak, -}) { - if (!queueEmpty) return 0; - return currentCount == previousCount ? (stableStreak + 1) : 0; -} - @visibleForTesting bool shouldPauseMaintenanceTraffic({required bool offloadActive}) => offloadActive; @@ -427,30 +416,64 @@ class BleEngine { /// harmless once we own the band). Future _claimBand() async { final other = _bandOwner; - if (other != null && !identical(other, this)) { - if (isBackgroundDrainer) { + final incumbentPresent = other != null && !identical(other, this); + final decision = BandClaimPolicy.decide( + incumbentPresent: incumbentPresent, + // LIVENESS, not just non-nullness: a claim held by an engine that has no + // session (its connect threw, or its link went down) is a STALE claim, + // and honouring it wedged every later background drain for the whole + // process lifetime ("strap not reachable this cycle", forever). + incumbentLive: incumbentPresent && other.holdsBandLink, + isBackgroundDrainer: isBackgroundDrainer, + ); + switch (decision) { + case BandClaimDecision.yieldToOwner: _log( - 'band already owned by the foreground session — background drain ' + 'band already owned by a live foreground session — background drain ' 'yielding (avoids duplicate ACKs on the same offload).', ); return false; - } - _log('preempting a background drain to take the foreground session.'); - try { - await other - .disconnect() - .timeout(const Duration(seconds: 10)); - } on TimeoutException { - _log('preempted engine teardown timed out after 10s — proceeding ' - 'with the foreground connect anyway.'); - } catch (e) { - _log('preempted engine teardown failed ($e) — proceeding.'); - } + case BandClaimDecision.preemptThenClaim: + _log('preempting a background drain to take the foreground session.'); + try { + await other!.disconnect().timeout(const Duration(seconds: 10)); + } on TimeoutException { + _log('preempted engine teardown timed out after 10s — proceeding ' + 'with the foreground connect anyway.'); + } catch (e) { + _log('preempted engine teardown failed ($e) — proceeding.'); + } + break; + case BandClaimDecision.claim: + if (incumbentPresent) { + _log('taking over a STALE band claim (the previous owner has no ' + 'live link).'); + } + break; } _bandOwner = this; return true; } + /// Whether this engine actually holds (or is actively bringing up) a BLE + /// link — the liveness test [BandClaimPolicy] uses on the incumbent owner. + /// A session object exists from the moment `_doConnect` starts, so a connect + /// still in flight correctly counts as live; every failure path nulls the + /// session and drops the phase to idle/error before returning. + bool get holdsBandLink => + _session != null && + _phase != BleConnState.idle && + _phase != BleConnState.error; + + /// Test-only view of the process-wide single-owner claim. + @visibleForTesting + static bool get bandClaimed => _bandOwner != null; + + /// Test-only reset of the process-wide claim (static state otherwise leaks + /// across test cases). + @visibleForTesting + static void resetBandClaimForTest() => _bandOwner = null; + void _releaseBand() { if (identical(_bandOwner, this)) _bandOwner = null; } @@ -567,7 +590,7 @@ class BleEngine { // Historical-offload bookkeeping. A controller is live for the whole connection // (we keep ACKing HISTORY_END markers as they arrive, even after the first // HISTORY_COMPLETE — a later strap-triggered offload reuses it). - _DrainController? _drain; + DrainController? _drain; bool _liveEnabled = false; // Background live downgrade: only the compact realtime-HR stream is armed // (no high-rate R10/R11 + IMU + optical flood). Set by [enableHrOnlyLive]. @@ -639,6 +662,9 @@ class BleEngine { // Lifetime count of GET_DATA_RANGE reads rejected by isCorruptFutureRtc — // see the range_oldest/range_newest handler below. int _corruptDataRangeCount = 0; + // Lifetime count of GET_CLOCK `clock_epoch` reads rejected by the same gate + // (ClockPolicy.acceptsClockRead) — see the clock_epoch handler below. + int _corruptClockReadCount = 0; DateTime? _bondTime; // when the handshake completed (bond confirmed) DateTime? _armTime; // when live (R10/R11) streams were last armed int _autoContinueCount = 0; // consecutive auto-continues this connection @@ -814,6 +840,10 @@ class BleEngine { // recovery already happens automatically at the DB layer. 'counter_regressions_total': _counterRegression.regressions, 'corrupt_data_ranges_total': _corruptDataRangeCount, + 'corrupt_clock_reads_total': _corruptClockReadCount, + // Bursts whose open chunk was discarded un-committed and whose HISTORY_END + // token was therefore refused (see BurstTrimGuard). + 'poisoned_bursts_total': _drain?.poisonedBursts ?? 0, 'history_requests': _historyRequests, 'history_completions': _historyCompletions, 'successful_bursts': _successfulBursts, @@ -922,9 +952,35 @@ class BleEngine { if (!await _claimBand()) return false; // Any prior session is dead to us now — tear it down before a new one. await _teardownSession(intentional: true); - return _doConnect(device); + try { + return await _doConnect(device); + } catch (e) { + // _doConnect guards its own known failure modes, but anything thrown + // OUTSIDE those guards (e.g. the connectionState subscription setup, which + // runs before the first try block) used to escape with the band claim + // still held and a half-built session left in `connecting` — which + // [BandClaimPolicy] would then read as a LIVE incumbent, permanently + // starving every later background drain. Route every escape through the + // one failure exit. + _log('connect failed (unhandled): $e'); + await _failConnect(); + return false; + } }); + /// Common exit for every failed-connect path: tear the half-built session + /// down, drop to `idle`, AND RELEASE THE BAND CLAIM. + /// + /// [_claimBand] runs BEFORE the link is up, so a connect that threw used to + /// leave `_bandOwner` pointing at an engine with no link — and only + /// `disconnect()` ever released it, which nothing calls on this path. Every + /// later background drain then saw a non-null owner and yielded forever. + Future _failConnect() async { + await _teardownSession(intentional: true); + _releaseBand(); + _setPhase(BleConnState.idle); + } + Future _doConnect(BluetoothDevice device) async { state.address = device.remoteId.str; _setPhase(BleConnState.connecting); @@ -958,8 +1014,7 @@ class BleEngine { ); } catch (e) { _log('connect failed: $e'); - await _teardownSession(intentional: true); - _setPhase(BleConnState.idle); + await _failConnect(); return false; } @@ -1026,6 +1081,7 @@ class BleEngine { if (!session.connected) { _log('connect: link dropped during setup.'); + await _failConnect(); return false; } @@ -1039,8 +1095,7 @@ class BleEngine { } if (svc == null) { _log('Harvard service not found on device.'); - await _teardownSession(intentional: true); - _setPhase(BleConnState.idle); + await _failConnect(); return false; } BluetoothCharacteristic? find(String prefix) { @@ -1059,8 +1114,7 @@ class BleEngine { events == null || data == null) { _log('Missing one or more Harvard characteristics.'); - await _teardownSession(intentional: true); - _setPhase(BleConnState.idle); + await _failConnect(); return false; } @@ -1144,7 +1198,7 @@ class BleEngine { // fire INIT — which triggers the historical flood. Historical + live records // then arrive on the same subscription; HISTORY_END markers are committed // (raw+samples+cursor, atomically) BEFORE we ACK, so the offload is resumable. - _drain = _DrainController( + _drain = DrainController( onRecord: _storeRecord, onRecordsBatch: onRecordsBatch == null ? null : _storeRecordsBatch, onCommit: onCommitBatch == null ? null : _commitBatch, @@ -1159,8 +1213,7 @@ class BleEngine { return true; } catch (e) { _log('connect setup failed: $e'); - await _teardownSession(intentional: true); - _setPhase(BleConnState.idle); + await _failConnect(); return false; } } @@ -1313,7 +1366,7 @@ class BleEngine { _lastRx = DateTime.now(); for (final frame in session.asm[role]!.feed(chunk)) { if (frame.valid) { - _onFrame(role, frame); + _onFrame(role, frame, session); } else { // Previously silent: a degrading radio corrupting frames looked // identical to a healthy one everywhere. Now counted (surfaced in @@ -1339,7 +1392,10 @@ class BleEngine { // ── link-down handling (drives reconnect via the caller's contract) ───────────── void _onLinkDown(_Session session) { - if (_session != session) return; // a stale session's stream + if (LinkDownPolicy.evaluate(sessionIsCurrent: _session == session) == + LinkDownAction.ignoreStaleSession) { + return; // a stale session's stream + } final wasIntentional = session.intentionalClose; session.connected = false; // A drain in flight must complete (with linkDown) immediately, not run out @@ -1359,6 +1415,26 @@ class BleEngine { // through the same single-flight connect, so there's still exactly one path. _setOffloadActive(false); _setPhase(BleConnState.idle); + // TEAR THE SESSION DOWN NOW. This used to happen ONLY on the next + // connect()/disconnect() — which never comes when BondRefusalGiveUp pauses + // auto-reconnect (`state.autoReconnectPaused`), so the dead session's five + // timers (heartbeat 10 s, keep-alive 30 s, periodic backfill 900 s, idle + // watchdog, historical retry) kept firing into a dead characteristic + // forever and its four onValueReceived subscriptions stayed registered — + // one more full set leaked on every drop. Deferred off this notification + // callback (we are inside one of the very subscriptions being cancelled) + // and non-intentional, so no redundant device.disconnect() is issued. + unawaited( + Future(() async { + if (_session != session) return; // a connect already replaced us + await _teardownSession(intentional: false); + // A claim held with no link is a stale claim (see [BandClaimPolicy]); + // releasing it here is what stops a failed foreground link from + // wedging every later background drain. The caller's reconnect loop + // re-claims through connect(), where foreground still preempts. + _releaseBand(); + }), + ); } /// Feed an UNINTENTIONAL disconnect to the cross-reconnect detectors. A timeout @@ -1416,7 +1492,12 @@ class BleEngine { static const Duration _serviceDiscoveryTimeout = Duration(seconds: 15); static const Duration _notifySetupTimeout = Duration(seconds: 15); - Future _write(Uint8List raw) { + /// [owner] pins the write to ONE session. Without it a write queued by a + /// long-parked drain (a big commit, then up to ~25 s of ACK retries) lands on + /// whatever session happens to be current when the write chain reaches it — + /// i.e. an OLD connection's batch-ACK, with a re-used sync seq, written onto + /// a BRAND NEW link. Every offload write passes its owning session. + Future _write(Uint8List raw, {_Session? owner}) { final session = _session; final completer = Completer(); _writeChain = _writeChain.then((_) async { @@ -1427,6 +1508,10 @@ class BleEngine { _log('write skipped: link not ready.'); return; } + if (owner != null && !identical(owner, session)) { + _log('write skipped: it belongs to a session that is no longer live.'); + return; + } // allowLongWrite: the rich SET_ALARM_TIME frame is 32B — the only write // that exceeds the 20B ATT limit of a default (23B) MTU. Without a long // write, flutter_blue_plus throws "value > mtu-3" if the negotiated MTU @@ -1454,16 +1539,17 @@ class BleEngine { /// false only after every attempt failed — the caller must then bounce the /// link (the chunk is already durably committed; the band re-delivers it next /// session and the decoded store dedups by REPLACE). - Future _writeAckVerified(Uint8List ack) async { + Future _writeAckVerified(Uint8List ack, _Session session) async { var failures = 0; while (true) { - if (await _write(ack)) return true; + if (_sessionIsStale(session)) return false; + if (await _write(ack, owner: session)) return true; failures++; if (!ackRetryPolicy.shouldRetry(failures)) return false; _log('[SYNC] batch-ACK write failed (attempt $failures/' '${ackRetryPolicy.maxAttempts}) — retrying.'); await Future.delayed(ackRetryPolicy.delayFor(failures)); - if (_session?.connected != true) return false; + if (_sessionIsStale(session)) return false; } } @@ -1563,11 +1649,22 @@ class BleEngine { } // ── frame handling ───────────────────────────────────────────────────────────── - void _onFrame(String role, Frame frame) { + void _onFrame(String role, Frame frame, _Session session) { final pt = frame.packetType; - if (role == 'data' && - (pt == PacketType.metadata || pt == PacketType.historicalData)) { - _enqueueOffloadFrame(frame); + // Metadata ALWAYS takes the serialized queue, whatever characteristic it + // was reassembled on. It used to take the queue only on the `data` role; + // metadata off `cmd_from`/`events` was fired unawaited on the immediate + // path — the one route that could run a HISTORY_END handler CONCURRENTLY + // with the queued drain, i.e. two handlers on the same DrainController, + // where one snapshots an empty buffer and writes its ACK before the + // other's commit is durable. See [FrameRoutePolicy]. + final route = FrameRoutePolicy.route( + isMetadata: pt == PacketType.metadata, + isHistorical: pt == PacketType.historicalData, + isDataRole: role == 'data', + ); + if (route == FrameRoute.serializedQueue) { + _enqueueOffloadFrame(frame, session); return; } _processImmediateFrame(frame); @@ -1575,10 +1672,8 @@ class BleEngine { void _processImmediateFrame(Frame frame) { final pt = frame.packetType; - if (pt == PacketType.metadata) { - unawaited(_handleSyncMarker(frame)); - return; - } + // NOTE: metadata never reaches here — [FrameRoutePolicy] routes every + // metadata frame to the serialized offload queue regardless of role. // LIVE streams: realtime HR/RR (0x28), realtime R10 (0x2B), IMU (0x33). // EPHEMERAL — these are the high-rate flood (~655 MB/day) and the daily // metrics need ONLY the 1 Hz historical substrate (0x2F / R24). We do NOT @@ -1639,17 +1734,28 @@ class BleEngine { _absorbState(decoded); } - void _enqueueOffloadFrame(Frame frame) { + void _enqueueOffloadFrame(Frame frame, _Session session) { + if (_session != session || !session.connected) return; // stale session _offloadFrames.add(frame); if (_offloadActive || frame.packetType == PacketType.historicalData) { _setOffloadActive(true); } if (_drainingOffloadFrames) return; _drainingOffloadFrames = true; - unawaited(_drainOffloadFrames()); - } - - Future _drainOffloadFrames() async { + unawaited(_drainOffloadFrames(session)); + } + + /// The ONE serialized offload-frame processor. [session] is the connection + /// that started this loop; it is re-checked around every await because a + /// drain can be parked for a long time (a multi-second large-batch commit, + /// then up to ~25 s of ACK retries) and `_teardownSession` clears + /// `_drainingOffloadFrames` underneath it. Without the guard a SECOND + /// drainer starts on the new session while this one is still alive — two + /// loops on the same DrainController, so one commit() snapshots an empty + /// buffer and its ACK can be written before the other's commit is durable — + /// and this stale loop would go on to write the OLD connection's token onto + /// the NEW link, tearing down a healthy session when that write failed. + Future _drainOffloadFrames(_Session session) async { // this used to have no try/finally, so if anything inside the loop threw // (a couple of the ledger writes in _handleSyncMarker weren't guarded), // _drainingOffloadFrames never got reset back to false and every future @@ -1657,6 +1763,7 @@ class BleEngine { // sync progress until a full disconnect/reconnect. try { while (_offloadFrames.isNotEmpty) { + if (_sessionIsStale(session)) return; final count = _offloadFrames.length > 64 ? 64 : _offloadFrames.length; final batch = _offloadFrames.sublist(0, count); _offloadFrames.removeRange(0, count); @@ -1665,8 +1772,9 @@ class BleEngine { // no Timer churn at flood rates. Markers re-arm it in _handleSyncMarker. _armIdleWatchdog(); for (final frame in batch) { + if (_sessionIsStale(session)) return; if (frame.packetType == PacketType.metadata) { - await _handleSyncMarker(frame); + await _handleSyncMarker(frame, session); } else { _ingestHistoricalFrame(frame); } @@ -1676,10 +1784,18 @@ class BleEngine { } } } finally { - _drainingOffloadFrames = false; + // Only the CURRENT session's loop may clear the flag. A stale loop + // unwinding after the new session's drainer already started would + // otherwise re-open the door to a second concurrent drainer. + if (_session == session) _drainingOffloadFrames = false; } } + /// True once [session] is no longer the engine's live session — the guard + /// every long-parked offload callback shares. + bool _sessionIsStale(_Session session) => + _session != session || !session.connected; + /// THE single historical-record processing path — used by BOTH the queued /// offload drain (real traffic) and the immediate fallback. Decode → gate /// (plausibility + frontier via [RecordGate]) → storage enqueue. Keeping one @@ -1846,29 +1962,48 @@ class BleEngine { if (f.containsKey('clock_epoch')) { final dev = f['clock_epoch'] as int; final wall = DateTime.now().millisecondsSinceEpoch ~/ 1000; - _clockRef = ClockRef(device: dev, wall: wall); - _log('Clock correlated: device=$dev wall=$wall (drift=${wall - dev}s).'); - // Re-issue SET_CLOCK if the strap RTC has drifted > 1 day or is unset — - // but BOUND the retries: setClock() reads the clock back, so an unbounded - // re-issue on a firmware that never latches either payload form would spin - // SET_CLOCK/GET_CLOCK forever. Historical records carry their own embedded - // unix time regardless, so giving up after a few tries is safe. - if (ClockPolicy.shouldSetClock(dev, wall)) { - if (_clockCorrectTries < 3) { - _clockCorrectTries++; - _log( - 'Clock drift over policy — re-issuing SET_CLOCK ' - '(attempt $_clockCorrectTries/3).', - ); - unawaited(setClock()); + // SANITY GATE, mirroring the one `range_newest` gets below. An + // implausibly far-future `clock_epoch` yields a large NEGATIVE driftSec, + // and setAlarm arms at `when - driftSec` — years out, where the alarm + // silently never fires — while the bounded SET_CLOCK retry budget is + // spent chasing a value that was never real. Reject the read: with no + // correlation the alarm falls back to the raw wall epoch. connect() + // already issues an unconditional SET_CLOCK, and the periodic re-verify + // re-reads, so a genuinely-wrong RTC still gets corrected. + if (!ClockPolicy.acceptsClockRead(dev, wall)) { + _corruptClockReadCount++; + _log( + '[SYNC] GET_CLOCK clock_epoch=$dev is implausibly far in the future ' + '— treating as a corrupt strap RTC read; NOT correlating the strap ' + 'clock (alarms fall back to the raw wall epoch) ' + '(corrupt_clock_reads_total=$_corruptClockReadCount).', + ); + } else { + _clockRef = ClockRef(device: dev, wall: wall); + _log('Clock correlated: device=$dev wall=$wall (drift=${wall - dev}s).'); + // Re-issue SET_CLOCK if the strap RTC has drifted > 1 day or is unset — + // but BOUND the retries: setClock() reads the clock back, so an + // unbounded re-issue on a firmware that never latches either payload + // form would spin SET_CLOCK/GET_CLOCK forever. Historical records carry + // their own embedded unix time regardless, so giving up after a few + // tries is safe. + if (ClockPolicy.shouldSetClock(dev, wall)) { + if (_clockCorrectTries < 3) { + _clockCorrectTries++; + _log( + 'Clock drift over policy — re-issuing SET_CLOCK ' + '(attempt $_clockCorrectTries/3).', + ); + unawaited(setClock()); + } else { + _log( + 'Clock still off after 3 SET_CLOCK attempts — giving up; ' + 'firmware may not accept our payload length.', + ); + } } else { - _log( - 'Clock still off after 3 SET_CLOCK attempts — giving up; ' - 'firmware may not accept our payload length.', - ); + _clockCorrectTries = 0; // latched — reset for the next drift episode } - } else { - _clockCorrectTries = 0; // latched — reset for the next drift episode } } if (f.containsKey('range_oldest') && f.containsKey('range_newest')) { @@ -2007,7 +2142,84 @@ class BleEngine { } } - Future _handleSyncMarker(Frame frame) async { + /// Refuse to echo a HISTORY_END token, for one of the [TrimAckVerdict] + /// blocked reasons. The band keeps the chunk and re-delivers it on the next + /// offload; re-delivery is dedup-safe (decoded rows REPLACE by rec_ts, raw + /// rows key on the record hex). + Future _refuseHistoryEndTrim( + TrimAckVerdict verdict, { + required DrainController d, + required _Session session, + required String tokenHex, + required int? batchId, + }) async { + switch (verdict) { + case TrimAckVerdict.send: + return; + case TrimAckVerdict.blockedStaleSession: + // Deliberately no ledger write and no teardown: we may be mid-teardown + // already, and every side effect here would land on a session that is + // not ours. + _log( + '[SYNC] HISTORY_END token=$tokenHex belongs to a session that is no ' + 'longer live — NOT ACKing. Writing it would put an old connection\'s ' + 'token (with a re-used sync seq) onto the new link. The band ' + 're-delivers this chunk next offload.', + ); + return; + case TrimAckVerdict.blockedDiscardedBurst: + // The idle watchdog abandoned this burst's buffered records. Persist + // anything that arrived since (dedup-safe) but WITHOUT the token, so + // the trim cursor never claims a chunk we threw away. + await d.commit(null); + _log( + '[SYNC] HISTORY_END token=$tokenHex terminates a DISCARDED burst ' + '(its open chunk was abandoned un-committed) — NOT ACKing, so the ' + 'band cannot trim the records we dropped. It re-delivers them next ' + 'offload.', + ); + await _bestEffortLedgerWrite(() => LocalDb.upsertSyncLedgerEntry( + chunkId: 'batch:$tokenHex', + kind: 'historical_batch', + status: 'trim_refused', + lastError: 'discarded_burst', + metaPatch: {'batch_id': batchId, 'records': d.records}, + )); + return; + case TrimAckVerdict.blockedCommitFailed: + // THE safe-trim invariant. The transaction rolled back, so the cursor + // did not advance and the rows are not durable — commit() re-buffered + // them rather than losing them. Never ACK here: that is exactly the + // path where records existed nowhere, permanently and silently. + _log( + '[SYNC] DURABLE COMMIT FAILED for token=$tokenHex — NOT ACKing (the ' + 'band must keep this chunk). Records were re-buffered; bouncing the ' + 'link so the next session retries the commit from a clean batch.', + ); + await _bestEffortLedgerWrite(() => LocalDb.upsertSyncLedgerEntry( + chunkId: 'batch:$tokenHex', + kind: 'historical_batch', + status: 'commit_failed', + lastError: 'durable_commit_failed', + metaPatch: {'batch_id': batchId, 'records': d.records}, + )); + // Bounce rather than retry in place: a commit that failed on a large + // batch (the observed production OOM inside commitSyncBatch) only gets + // bigger if we keep appending to the same buffer. A reconnect drops + // the buffer, and the band re-delivers from its un-advanced cursor. + if (!_sessionIsStale(session)) { + unawaited( + _teardownSession(intentional: false).then((_) { + _setPhase(BleConnState.idle); // caller's reconnect loop takes over + }), + ); + } + return; + } + } + + Future _handleSyncMarker(Frame frame, _Session session) async { + if (_sessionIsStale(session)) return; final m = parseMetadata(frame.inner); if (m == null) return; _armIdleWatchdog(); @@ -2033,6 +2245,9 @@ class BleEngine { if (m.sub == SyncMeta.historyEnd && m.token != null) { final d = _drain; if (d == null) return; + final tokenHex = m.token! + .map((b) => b.toRadixString(16).padLeft(2, '0')) + .join(); if (!_offloadActive) { _setHpsTerminal( _HpsTerminalKind.metadataWhileNotSyncing, @@ -2040,7 +2255,25 @@ class BleEngine { drain: d, ); } - await _awaitBurstTrafficSettle(d); + // PRE-COMMIT GATE. Refuse the trim before anything else touches the link + // or the durable cursor: a stale session must not be written to at all, + // and a poisoned burst's records are already gone — there is nothing + // this token may legitimately trim. + final preVerdict = TrimAckPolicy.evaluate( + sessionCurrent: !_sessionIsStale(session), + burstDiscarded: d.burstDiscarded, + commitDurable: true, + ); + if (preVerdict != TrimAckVerdict.send) { + await _refuseHistoryEndTrim( + preVerdict, + d: d, + session: session, + tokenHex: tokenHex, + batchId: m.batchId, + ); + return; + } final expected = m.expectedPacketCount; // Records the plausibility gate silently rejected THIS burst (stale/ // wandering-clock block — by design, "neither stored nor counted", @@ -2098,9 +2331,6 @@ class BleEngine { } _successfulBursts++; _mergeValidatedBurst(d); - final tokenHex = m.token! - .map((b) => b.toRadixString(16).padLeft(2, '0')) - .join(); final r = d.bufferedRecTsRange; _log( '[SYNC] HistoryEnd batch=${m.batchId} records=${d.records} ' @@ -2114,18 +2344,40 @@ class BleEngine { // once the ACK is link-layer confirmed, so a crash before the ACK // re-delivers the chunk. Echo the 8-byte slice the band acks verbatim — // a mangled echo is the "Groundhog Day" re-flood bug. - await d.commit(m.token); // raw + samples + strap_trim cursor, atomic + final durable = await d.commit(m.token); // raw+samples+cursor, atomic + // RE-GATE after the await. commit() can take seconds on a large batch — + // long enough for the link to die under us — and it now reports whether + // the transaction actually became durable instead of swallowing the + // exception. Either way the ACK is refused, which is the whole point: + // the ACK is what makes the band trim its flash. + final verdict = TrimAckPolicy.evaluate( + sessionCurrent: !_sessionIsStale(session), + burstDiscarded: d.burstDiscarded, + commitDurable: durable, + ); + if (verdict != TrimAckVerdict.send) { + await _refuseHistoryEndTrim( + verdict, + d: d, + session: session, + tokenHex: tokenHex, + batchId: m.batchId, + ); + return; + } final ack = buildHistoryResultOk(_seq.nextSync(), m.token!); _log( '[SYNC] ACK frame=' '${ack.map((b) => b.toRadixString(16).padLeft(2, '0')).join()}', ); - // VERIFIED ACK (retried): the cursor above is already durably committed, - // so a silently-failed ACK write would leave the band never trimming and - // re-flooding the same chunk forever. On persistent failure bounce the - // link — the committed data is safe, and the next session's re-delivery - // is dedup-safe (decoded rows REPLACE by rec_ts). - if (!await _writeAckVerified(ack)) { + // VERIFIED ACK (retried). We only reach here when [TrimAckPolicy] + // returned `send` — i.e. the commit above REPORTED durable (it no longer + // swallows its exception) and the session is still ours — so the cursor + // genuinely is committed. A silently-failed ACK write would leave the + // band never trimming and re-flooding the same chunk forever. On + // persistent failure bounce the link — the committed data is safe, and + // the next session's re-delivery is dedup-safe (rows REPLACE by rec_ts). + if (!await _writeAckVerified(ack, session)) { // Real per-chunk ledger row, keyed by the token itself — previously // every ledger write here collapsed onto one shared 'capture' row, // so a token that kept failing ACROSS reconnects (the "Groundhog @@ -2168,11 +2420,16 @@ class BleEngine { '${ackRetryPolicy.maxAttempts} attempts (token=$tokenHex, ' 'failures_for_this_token=$failCount) — bouncing the link; data ' 'is committed and the band will re-send.'); - unawaited( - _teardownSession(intentional: false).then((_) { - _setPhase(BleConnState.idle); // caller's reconnect loop takes over - }), - ); + // ONLY bounce a session that is still OURS. _writeAckVerified also + // returns false when the session died under it, and tearing down then + // would kill the healthy session that replaced it. + if (!_sessionIsStale(session)) { + unawaited( + _teardownSession(intentional: false).then((_) { + _setPhase(BleConnState.idle); // caller's reconnect loop takes over + }), + ); + } return; } _chunkFailures.recordSuccess(tokenHex); @@ -2215,7 +2472,24 @@ class BleEngine { // Backlog fully handed over (cursor is now at the live edge). Commit the tail // and KEEP LISTENING — live records continue on the same subscription. We do // NOT ACK a HISTORY_COMPLETE and we do NOT switch modes. - await d.commit(null); // tail (no new token) — persist anything buffered + final tailDurable = await d.commit(null); // tail — no new trim token + if (_sessionIsStale(session)) return; // link died under the tail commit + if (!tailDurable) { + // No ACK is written for a HISTORY_COMPLETE, so nothing was trimmed and + // no data is at risk — but the tail is NOT durable yet. commit() left + // the records buffered, so the next commit (the next burst's + // HISTORY_END, or the flush on teardown) re-attempts them. + _log( + '[SYNC] HistoryComplete tail commit FAILED — ${d.bufferedRecords} ' + 'records stay buffered for the next commit. Nothing was trimmed ' + '(HISTORY_COMPLETE is never ACKed), so no data is at risk.', + ); + await _bestEffortLedgerWrite(() => LocalDb.upsertSyncLedgerEntry( + status: 'tail_commit_failed', + lastError: 'durable_commit_failed', + metaPatch: {'records_seen': d.records}, + )); + } d.onComplete(); _historyCompletions++; _session?.idleWatchdog?.cancel(); @@ -2241,51 +2515,6 @@ class BleEngine { } } - Future _awaitBurstTrafficSettle(_DrainController d) async { - const poll = Duration(milliseconds: 60); - const budget = Duration(milliseconds: 720); - const requiredStablePolls = 3; - final deadline = DateTime.now().add(budget); - var previousCount = d.currentBurstPacketCount; - var waitedMs = 0; - var stablePolls = 0; - while (DateTime.now().isBefore(deadline)) { - if (_offloadFrames.isNotEmpty) { - await Future.delayed(poll); - waitedMs += poll.inMilliseconds; - previousCount = d.currentBurstPacketCount; - stablePolls = 0; - continue; - } - await Future.delayed(poll); - waitedMs += poll.inMilliseconds; - final currentCount = d.currentBurstPacketCount; - stablePolls = nextBurstStablePollStreak( - queueEmpty: _offloadFrames.isEmpty, - currentCount: currentCount, - previousCount: previousCount, - stableStreak: stablePolls, - ); - if (_offloadFrames.isEmpty && stablePolls >= requiredStablePolls) { - if (waitedMs > 0) { - _log( - '[SYNC] history-end settle: waited=${waitedMs}ms ' - 'traffic=$currentCount historical=${d.currentBurstHistoricalPacketCount} ' - 'stable_polls=$stablePolls', - ); - } - return; - } - previousCount = currentCount; - } - _log( - '[SYNC] history-end settle timed out at ${waitedMs}ms ' - 'traffic=${d.currentBurstPacketCount} ' - 'historical=${d.currentBurstHistoricalPacketCount} ' - 'stable_polls=$stablePolls', - ); - } - // ── post-offload policy: empty-sync, stuck-strap, auto-continue ────────────── Future _onOffloadFinished({required bool complete}) async { final d = _drain; @@ -2735,7 +2964,7 @@ class BleEngine { void _setHpsTerminal( _HpsTerminalKind kind, { String? reason, - _DrainController? drain, + DrainController? drain, }) { final d = drain ?? _drain; _lastHpsTerminal = _HpsTerminal( @@ -2748,7 +2977,7 @@ class BleEngine { ); } - void _mergeValidatedBurst(_DrainController d) { + void _mergeValidatedBurst(DrainController d) { final burstCounts = d.burstStats.dataPacketCountsByRevision; final mergedCounts = { ..._sessionPacketCounts.dataPacketCountsByRevision, @@ -2846,14 +3075,19 @@ class BleEngine { /// [awaitComplete] future that resolves when the band signals HISTORY_COMPLETE (or /// the link drops / a safety timeout elapses), so a caller can block until the /// backlog is fully handed over without disturbing the continuous listen. -class _DrainController { +/// +/// ENGINE-INTERNAL. Public only so the safe-trim invariant it enforces (a +/// commit that fails must re-buffer and must NOT let the caller ACK) can be +/// regression-tested directly without a real band. +@visibleForTesting +class DrainController { final SampleSink onRecord; final BatchSink? onRecordsBatch; final CommitSyncBatchSink? onCommit; final ArchiveSink? onArchive; final void Function(String) log; - _DrainController({ + DrainController({ required this.onRecord, required this.onRecordsBatch, required this.onCommit, @@ -2869,7 +3103,7 @@ class _DrainController { final List _archives = []; // Per-burst packet accounting (per-revision counts + sequence gap detection), // merged into the session totals when a burst validates. - final _BurstStats burstStats = _BurstStats(); + final BurstStats burstStats = BurstStats(); int records = 0; // total this connection int recordsThisOffload = 0; // since the last HISTORY_COMPLETE / rearm @@ -2908,6 +3142,17 @@ class _DrainController { bool lastTrimAdvanced = false; int consecutiveValidationFailures = 0; + // Poison latch for the burst currently open (see BurstTrimGuard): once its + // records have been discarded un-committed, its straggler HISTORY_END must + // never be echoed. + final BurstTrimGuard _trimGuard = BurstTrimGuard(); + + /// True while the open burst's terminal may NOT be ACKed. + bool get burstDiscarded => _trimGuard.discarded; + + /// Bursts poisoned this connection (diagnostics). + int get poisonedBursts => _trimGuard.poisonedBursts; + bool get _buffering => onCommit != null || onRecordsBatch != null; int get currentBurstPacketCount => burstStats.totalTrafficPacketCount; int get currentBurstTrafficCount => burstStats.totalTrafficPacketCount; @@ -2987,6 +3232,9 @@ class _DrainController { _linkDown = false; _lastProgressAt = DateTime.now(); burstStats.reset(); + // A fresh burst starts un-poisoned: whatever was discarded belonged to the + // burst that just ended, and the band will re-deliver it. + _trimGuard.beginBurst(); } void onLinkDown() => _linkDown = true; @@ -2997,10 +3245,19 @@ class _DrainController { /// Abandon the buffered-but-not-yet-committed chunk WITHOUT persisting (idle /// watchdog). These records were never ACKed, so the band re-delivers them on the /// next offload — dropping them here just avoids ACKing a partial. + /// + /// POISONS THE OPEN BURST. Discarding alone was not enough: the band had + /// already put that burst's HISTORY_END on the wire, and the terminal handler + /// went on to commit the (now empty) buffer and echo the token verbatim — + /// trimming exactly the records that were just dropped. The poison is + /// unconditional (even with an empty buffer, this burst was abandoned) and + /// is cleared only by [rearm] / a fresh HISTORY_START. void discardOpenChunk() { + _trimGuard.discardOpenChunk(); if (_raws.isEmpty && _archives.isEmpty) return; log('discarding ${_raws.length} un-ACKed buffered records + ' - '${_archives.length} archived (idle).'); + '${_archives.length} archived (idle). This burst\'s HISTORY_END token ' + 'is now un-ACKable — the band keeps the chunk.'); _raws.clear(); _samples.clear(); _archives.clear(); @@ -3010,10 +3267,24 @@ class _DrainController { /// ATOMICALLY (via onCommit) and return only once durable — the caller writes /// the ACK afterwards. Snapshots the buffer so records arriving during the await /// land in the next commit. Updates [lastTrimAdvanced]. - Future commit(List? token) async { + /// + /// RETURNS WHETHER THE CHUNK IS ACTUALLY DURABLE. This used to be + /// `Future` with the exception swallowed into a log line, while the + /// buffer had ALREADY been snapshotted and cleared — so a failed transaction + /// left the rows nowhere (buffer cleared, transaction rolled back, cursor + /// unadvanced) and the caller went straight on to echo the HISTORY_END trim + /// token, telling the band to delete them from its flash. Those records + /// existed nowhere, permanently and silently. On failure the buffer is now + /// RESTORED (at the front, so arrival order is preserved) and the trim + /// bookkeeping rolled back, and the caller MUST NOT ACK — the band keeps the + /// chunk and re-delivers it, which is dedup-safe (decoded rows REPLACE by + /// rec_ts). + Future commit(List? token) async { final tokenHex = token ?.map((b) => b.toRadixString(16).padLeft(2, '0')) .join(); + final previousAckedToken = _lastAckedToken; + final previousTrimAdvanced = lastTrimAdvanced; lastTrimAdvanced = tokenHex != null && tokenHex != _lastAckedToken; if (tokenHex != null) _lastAckedToken = tokenHex; final raws = List.from(_raws); @@ -3028,12 +3299,24 @@ class _DrainController { } else if (onRecordsBatch != null && raws.isNotEmpty) { await onRecordsBatch!(raws, samples); } + return true; } catch (e) { - log('offload commit error: $e'); + // Put the snapshot back at the FRONT: records that arrived during the + // await are already appended behind it, so this preserves arrival order. + _raws.insertAll(0, raws); + _samples.insertAll(0, samples); + _archives.insertAll(0, archives); + // Roll back the trim bookkeeping too — nothing advanced. + _lastAckedToken = previousAckedToken; + lastTrimAdvanced = previousTrimAdvanced; + log('offload commit FAILED ($e) — ${raws.length} records + ' + '${archives.length} archived re-buffered; the caller MUST NOT ACK ' + 'this chunk (the band still holds it).'); + return false; } } - Future flush() => commit(null); + Future flush() => commit(null); /// Resolve once the current offload reaches HISTORY_COMPLETE, the link drops, or /// [timeout] elapses. Pure waiting — NO abort is ever sent (cutting the offload @@ -3077,7 +3360,8 @@ class _DrainController { } } -class _BurstStats { +@visibleForTesting +class BurstStats { static const Set _ordinaryHistoricalRevisions = { 7, 9, @@ -3091,7 +3375,7 @@ class _BurstStats { }; final Map _dataPacketCountsByRevision = {}; - final Map _sequenceByRevision = {}; + final Map _sequenceByRevision = {}; int _eventCount = 0; int _consoleCount = 0; int _unknownCount = 0; @@ -3103,8 +3387,8 @@ class _BurstStats { Map get dataPacketCountsByRevision => Map.unmodifiable(_dataPacketCountsByRevision); - Map get sequenceByRevision => - Map.unmodifiable(_sequenceByRevision); + Map get sequenceByRevision => + Map.unmodifiable(_sequenceByRevision); int get eventCount => _eventCount; int get consoleCount => _consoleCount; int get unknownCount => _unknownCount; @@ -3194,7 +3478,7 @@ class _BurstStats { (_dataPacketCountsByRevision[revision] ?? 0) + 1; final seq = _sequenceByRevision.putIfAbsent( revision, - () => _SequenceState(firstSequence: counter), + () => SequenceState(firstSequence: counter), ); seq.observe(counter); return; @@ -3241,8 +3525,9 @@ class _BurstStats { } } -class _SequenceState { - _SequenceState({required this.firstSequence}); +@visibleForTesting +class SequenceState { + SequenceState({required this.firstSequence}); final int firstSequence; int? lastSequence; diff --git a/lib/ble/ble_state.dart b/lib/ble/ble_state.dart index b55660bf..039dc37c 100644 --- a/lib/ble/ble_state.dart +++ b/lib/ble/ble_state.dart @@ -297,6 +297,199 @@ class AckRetryPolicy { } } +/// Why an in-flight HISTORY_END token may (or may not) be echoed back to the +/// band. See [TrimAckPolicy]. +enum TrimAckVerdict { + /// Every precondition holds — echo the verbatim 8-byte token. + send, + + /// The session that received this HISTORY_END is no longer the engine's + /// current session (the link dropped / was replaced while the handler was + /// parked mid-await). Writing now would put an OLD connection's token, with + /// a re-used sync seq, onto a BRAND NEW link — and a failed write would tear + /// down the healthy new session. + blockedStaleSession, + + /// The burst this token terminates had its buffered records DISCARDED + /// without ever being committed (idle watchdog / abort-and-retry). Its + /// straggler terminal is still in flight; echoing it would trim flash we + /// deliberately threw away. + blockedDiscardedBurst, + + /// The durable commit did NOT complete. The records are not in the database, + /// so the band must keep them. + blockedCommitFailed, +} + +/// THE gate on the one irreversible act in the whole offload protocol: echoing +/// a HISTORY_END continuation token, which is what tells the band it may trim +/// that chunk out of its flash. +/// +/// The safe-trim invariant is "every row is durable BEFORE the caller echoes +/// the HISTORY_END trim token, or none is". Historically only the HAPPY path +/// honoured it: the commit's exception was swallowed and the caller ACKed +/// regardless, so a failed transaction (a real production OOM inside +/// `commitSyncBatch` — see `lib/data/db.dart`) meant the buffer had already +/// been cleared, the transaction had rolled back, the cursor had not advanced, +/// and the band was then told to trim — those records existed nowhere, +/// permanently and silently. +/// +/// Pure and total: the engine supplies three observations, this decides. The +/// engine must call it AGAIN after the commit await (the commit can take +/// seconds on a large batch — long enough for the session to die under it). +class TrimAckPolicy { + const TrimAckPolicy._(); + + /// [sessionCurrent] — the receiving session is still the engine's session + /// AND still connected. + /// [burstDiscarded] — this burst's open chunk was discarded un-committed. + /// [commitDurable] — the atomic commit completed (pass `true` when asking + /// the pre-commit question "should I even commit this + /// token?"). + static TrimAckVerdict evaluate({ + required bool sessionCurrent, + required bool burstDiscarded, + required bool commitDurable, + }) { + // Order is deliberate: a stale session must be refused before anything + // else touches the (new) link, and a poisoned burst must be refused before + // the commit result is even considered — its records are already gone. + if (!sessionCurrent) return TrimAckVerdict.blockedStaleSession; + if (burstDiscarded) return TrimAckVerdict.blockedDiscardedBurst; + if (!commitDurable) return TrimAckVerdict.blockedCommitFailed; + return TrimAckVerdict.send; + } +} + +/// Per-burst poison latch for [TrimAckVerdict.blockedDiscardedBurst]. +/// +/// The idle watchdog abandons the open chunk (N buffered, never-committed +/// records) on the contract "the band re-delivers them next offload", and the +/// abort path then sends ABORT_HISTORICAL. But that burst's HISTORY_END is +/// ALREADY in flight and arrives anyway — and nothing stopped the handler from +/// committing an empty buffer and echoing the token verbatim, which trims +/// exactly the records that were just thrown away. The token is unknown at +/// discard time (it only arrives with the terminal), so the poison is keyed to +/// the BURST, not the token: a discard poisons the open burst, and only a +/// fresh HISTORY_START / re-arm clears it. +class BurstTrimGuard { + bool _discarded = false; + + /// Bursts poisoned since construction (diagnostics — a rising count means + /// the drain keeps stalling mid-burst). + int poisonedBursts = 0; + + /// True while the open burst may NOT be trimmed. + bool get discarded => _discarded; + + /// A fresh burst begins (HISTORY_START / re-arm) — nothing lost yet. + void beginBurst() => _discarded = false; + + /// The open chunk was abandoned without a durable commit. + void discardOpenChunk() { + if (_discarded) return; + _discarded = true; + poisonedBursts++; + } +} + +/// What a link-down must do to the session that just died. +enum LinkDownAction { + /// Cancel every timer + subscription the session owns, then surface `idle`. + tearDownSession, + + /// The event belongs to a session we already replaced — ignore it entirely. + ignoreStaleSession, +} + +/// A dropped link used to only flip a flag and surface the phase; the actual +/// teardown happened solely on the NEXT connect()/disconnect(). When +/// [BondRefusalGiveUp] trips, the app pauses auto-reconnect and never calls +/// disconnect() — so the dead session's five timers (heartbeat, keep-alive, +/// periodic backfill, idle watchdog, historical retry) kept firing forever and +/// its four `onValueReceived` subscriptions stayed registered, one more full +/// set leaked per drop. +class LinkDownPolicy { + const LinkDownPolicy._(); + + static LinkDownAction evaluate({required bool sessionIsCurrent}) => + sessionIsCurrent + ? LinkDownAction.tearDownSession + : LinkDownAction.ignoreStaleSession; +} + +/// Outcome of a process-wide single-owner band claim. +enum BandClaimDecision { + /// Take the claim (nobody holds it, or the incumbent's claim is stale). + claim, + + /// A live foreground owner exists and we are the background drainer — + /// don't touch the band this cycle. + yieldToOwner, + + /// A live background owner exists and we are foreground — drop its link + /// (awaited) and then take the claim. + preemptThenClaim, +} + +/// Pure arbitration for the process-wide single-owner guard. +/// +/// The claim used to be tested for NON-NULLNESS only, and was taken BEFORE the +/// link was up and released only by an explicit `disconnect()`. So a connect +/// that threw left the claim pointing at an engine with no link, forever: a +/// later iOS restore wake spun up a background drainer, saw a non-null owner, +/// yielded, and reported "strap not reachable this cycle" for the rest of the +/// process lifetime. [incumbentLive] is the fix — a claim held by an engine +/// with no session is not a claim. +class BandClaimPolicy { + const BandClaimPolicy._(); + + static BandClaimDecision decide({ + required bool incumbentPresent, + required bool incumbentLive, + required bool isBackgroundDrainer, + }) { + if (!incumbentPresent) return BandClaimDecision.claim; + if (!incumbentLive) return BandClaimDecision.claim; + if (isBackgroundDrainer) return BandClaimDecision.yieldToOwner; + return BandClaimDecision.preemptThenClaim; + } +} + +/// Where an inbound reassembled frame is processed. +enum FrameRoute { + /// The single serialized offload queue (`_offloadFrames`), so history + /// records and their terminals are handled strictly in arrival order by ONE + /// loop. + serializedQueue, + + /// Handled inline (command responses, events, live high-rate frames). + immediate, +} + +/// Pure routing decision for [FrameRoute]. +/// +/// Metadata (HISTORY_START / HISTORY_END / HISTORY_COMPLETE) used to reach the +/// serialized queue only when it arrived on the `data` characteristic; +/// metadata reassembled on `cmd_from`/`events` was fired unawaited on the +/// immediate path instead — the one route that could run a HISTORY_END handler +/// CONCURRENTLY with the queued drain, i.e. two handlers racing on the same +/// drain controller (one snapshots an empty buffer and can ACK before the +/// other's commit is durable). Metadata now always takes the queue. +class FrameRoutePolicy { + const FrameRoutePolicy._(); + + static FrameRoute route({ + required bool isMetadata, + required bool isHistorical, + required bool isDataRole, + }) { + if (isMetadata) return FrameRoute.serializedQueue; + if (isHistorical && isDataRole) return FrameRoute.serializedQueue; + return FrameRoute.immediate; + } +} + /// Tracks ACK-write failures per historical-batch token ACROSS RECONNECTS — /// a chunk whose ACK keeps failing for the SAME token (the "Groundhog Day" /// re-flood signature: the band never trims, so it re-sends the identical diff --git a/lib/cloud/cloud_import.dart b/lib/cloud/cloud_import.dart index e0aa1eef..c6fd7ac6 100644 --- a/lib/cloud/cloud_import.dart +++ b/lib/cloud/cloud_import.dart @@ -17,6 +17,8 @@ import 'dart:convert'; +import 'package:flutter/foundation.dart' show visibleForTesting; + import '../compute/derivation_engine.dart' show kAlgoVersion; import '../data/db.dart'; import 'backend_client.dart'; @@ -79,8 +81,7 @@ class CloudImporter { var sessCount = 0; for (final w in sessions) { if (w is! Map) continue; - await _writeSession(w.cast()); - sessCount++; + if (await _writeSession(w.cast())) sessCount++; } return CloudImportResult(dayCount, sessCount, _mapProfile(profileRaw)); @@ -250,14 +251,26 @@ class CloudImporter { ); } - static Future _writeSession(Map w) async { + /// Test seam for [_writeSession] — the malformed-row skip is a data-integrity + /// invariant, and `run()` needs a whole authenticated BackendClient to reach. + @visibleForTesting + static Future debugWriteSession(Map w) => + _writeSession(w); + + /// Returns true when a session row was actually written. + static Future _writeSession(Map w) async { num? n(Object? v) => v is num ? v : null; final start = n(w['start_ts'])?.toInt(); final end = n(w['end_ts'])?.toInt(); + // A session with no usable start is not a workout — writing it as epoch 0 + // filed a phantom workout on 1970-01-01 that then showed up in every query + // keyed on start_ts. Skip the row (same contract as WhoopImporter's + // _writeWorkout). + if (start == null) return false; final zones = w['zones']; await LocalDb.putSession({ 'id': (w['id'] ?? 'cloud_$start').toString(), - 'start_ts': start ?? 0, + 'start_ts': start, 'end_ts': end, 'type': (w['type'] ?? w['detected_type'] ?? 'other').toString(), 'status': (w['status'] ?? 'done').toString(), @@ -266,10 +279,11 @@ class CloudImporter { 'strain': n(w['strain'])?.toDouble(), 'max_hr': n(w['max_hr'])?.toInt(), 'duration_min': - (start != null && end != null) ? ((end - start) / 60).round() : null, + end != null ? ((end - start) / 60).round() : null, 'zone_min_json': zones == null ? null : jsonEncode(zones), 'created_at': DateTime.now().millisecondsSinceEpoch, }); + return true; } static String _ymd(DateTime d) => '${d.year.toString().padLeft(4, '0')}-' diff --git a/lib/coach/coach_db.dart b/lib/coach/coach_db.dart index 941f96d1..1d999bfb 100644 --- a/lib/coach/coach_db.dart +++ b/lib/coach/coach_db.dart @@ -4,15 +4,38 @@ // handle, over DERIVED-only views (v_metric/v_daily/v_series/v_hypnogram/ // v_sessions/v_baselines/v_insights — created by LocalDb._ensureCoachViews). // -// Two layers of safety, both fail-closed: -// 1. guardAndPrepare() — a static validator: SELECT/WITH only, single -// statement, no comments, no DML/DDL/PRAGMA, every FROM/JOIN target must be -// an allow-listed view, no base/raw table name anywhere, auto-LIMIT. -// 2. A read-only Database handle (openDatabase readOnly:true) — even if the -// guard were bypassed, SQLite physically refuses writes/DDL, and raw byte -// tables are blocked by the guard's identifier allow-list. +// THREE layers of safety, all fail-closed: +// +// 1. guardAndPrepare() — a static validator built on a real (small) SQL +// lexer + table-reference parser, NOT a single regex. It is an +// ALLOW-LIST: SELECT/WITH only, single statement, no comments, no +// DML/DDL/PRAGMA, and EVERY table position (including the extra members +// of a comma-separated implicit cross-join, which the old FROM/JOIN +// regex never even looked at) must resolve to an allow-listed `v_*` view +// or a CTE declared in the same statement. Anything else — a base table, +// a schema-qualified name, sqlite_master/sqlite_schema, a table-valued +// function, a subquery in FROM — is rejected because it is not on the +// list, so a table added to the schema tomorrow is closed by default +// instead of open by default. +// +// 2. _assertAllowedBtrees() — a STRUCTURAL check that does not trust the +// parser at all. sqflite exposes no sqlite3_set_authorizer() hook and a +// read-only handle still sees every table in the file, so we ask SQLite +// itself which btrees a statement would open: `EXPLAIN ` lists the +// OpenRead/ReopenIdx opcodes with the ROOT PAGE of each btree, and +// sqlite_master maps root pages back to the owning table. The allowed +// root-page set is derived (once, at runtime) by EXPLAINing the allowed +// views themselves, so it is exactly "the base tables the coach views +// are made of" — every other btree in the database, including +// workout_route (raw GPS) and sqlite_master, is unreachable no matter +// what the text-level parser thinks. This is the authorizer we can't +// install, implemented with the compiler's own answer. +// +// 3. A read-only Database handle (openDatabase readOnly:true) — even if +// both layers above were bypassed, SQLite physically refuses writes/DDL. import 'dart:convert'; +import 'package:flutter/foundation.dart' show visibleForTesting; import 'package:path/path.dart' as p; import 'package:sqflite/sqflite.dart'; @@ -30,7 +53,9 @@ class SqlGuardError implements Exception { class CoachDb { CoachDb._(); - /// The only tables the coach may read — derived, never raw. + /// The ONLY relations the coach may name in a table position — derived, + /// never raw. This is the whole allow-list: a CTE declared inside the same + /// statement is the only other thing that may appear after FROM/JOIN. static const Set allowedViews = { 'v_metric', 'v_daily', @@ -47,20 +72,42 @@ class CoachDb { 'insert', 'update', 'delete', 'drop', 'alter', 'create', 'replace', 'truncate', 'attach', 'detach', 'pragma', 'vacuum', 'reindex', 'analyze', 'begin', 'commit', 'rollback', 'savepoint', 'grant', 'revoke', 'trigger', - 'into', 'load_extension', + 'into', 'load_extension', 'explain', + }; + + /// Real on-disk relations. This is NOT the security boundary any more (the + /// allow-list above is) — it is a redundant net whose only jobs are to give + /// the model a clearer rejection reason and to stop a CTE from SHADOWING a + /// real table name. Anything missing from it is still rejected by the + /// allow-list; nothing here is ever reachable. + static const Set reservedTableNames = { + // derived / user tables + 'day_result', 'metric_series', 'baselines', 'derived_day', 'sessions', + 'notifications', 'journal', 'cycle_log', 'cycle_symptom', 'notif_fired', + 'sleep_override', 'sleep_session_candidates', 'wake_day_features', + 'workout_suggestions', 'workout_route', 'live_coverage', + // raw / decoded substrate + 'raw_records', 'raw_archive', 'decoded_onehz', 'decoded_rr', 'samples', + 'events', 'band_events', 'band_battery', + // sync / compute bookkeeping + 'sync_ledger', 'sync_quarantine', 'sync_cursor', 'sync_ledger_legacy', + 'sync_quarantine_legacy', 'sync_cursor_legacy', 'compute_jobs', + 'compute_freshness', + // sqlite internals (the sqlite_* prefix is rejected wholesale as well) + 'sqlite_master', 'sqlite_schema', 'sqlite_temp_master', + 'sqlite_temp_schema', 'sqlite_sequence', 'sqlite_stat1', 'dbstat', }; - // Base/raw table names that must NOT appear anywhere (force the views). - static const Set _denyTables = { - 'raw_records', 'decoded_onehz', 'decoded_rr', 'samples', 'sqlite_master', - 'day_result', 'metric_series', 'baselines', 'sessions', 'notifications', - 'derived_day', 'events', 'band_events', 'band_battery', 'live_coverage', - 'sync_ledger', 'sync_quarantine', 'sync_cursor', 'compute_jobs', - 'compute_freshness', 'journal', 'cycle_log', 'cycle_symptom', - 'primitive_artifacts', 'wake_day_features', 'workout_suggestions', + /// Tokens that can follow a table reference without being an alias. + static const Set _clauseKeywords = { + 'where', 'group', 'order', 'limit', 'offset', 'having', 'window', 'union', + 'intersect', 'except', 'on', 'using', 'join', 'inner', 'left', 'right', + 'full', 'cross', 'natural', 'and', 'or', 'as', 'values', 'returning', + 'select', 'from', 'with', }; static Database? _ro; + static Set? _allowedRoots; /// Open (and cache) a read-only handle. Ensures the RW handle first so the /// views exist (a read-only handle can't CREATE VIEW). @@ -75,69 +122,344 @@ class CoachDb { static Future close() async { await _ro?.close(); _ro = null; + _allowedRoots = null; } + // ── layer 1: lexer + table-reference parser ──────────────────────────────── + /// Validate + normalize an LLM-supplied SELECT. Throws [SqlGuardError] on any /// rejection. Returns the (possibly LIMIT-appended) query to execute. static String guardAndPrepare(String raw, {int rowCap = 200}) { var s = raw.trim(); if (s.isEmpty) throw SqlGuardError('Empty query.'); + if (s.codeUnits.contains(0)) throw SqlGuardError('Illegal character.'); + + // (a) Strip one optional trailing ';' before anything else, so the + // single-statement check below sees only genuine inner separators. + var body = s.endsWith(';') ? s.substring(0, s.length - 1).trim() : s; + if (body.isEmpty) throw SqlGuardError('Empty query.'); + + // (b) Mask string literals to a neutral numeric token. Quoted IDENTIFIERS + // ("x", `x`, [x]) are rejected outright — the coach never needs them, + // and SQLite's double-quote fallback (a bare "foo" silently becomes a + // string when it doesn't resolve to a column) is a parser trap. + final masked = _maskStrings(body); - // (a) No comments (they can hide payloads). - if (s.contains('--') || s.contains('/*') || s.contains('*/')) { + // (c) No comments (they can hide payloads) — checked on the MASKED text so + // an inline '--' inside a legitimate string value isn't a false hit. + if (masked.contains('--') || + masked.contains('/*') || + masked.contains('*/')) { throw SqlGuardError('Comments are not allowed.'); } - // (b) Single statement — strip one optional trailing ';', reject any inner. - var body = s.endsWith(';') ? s.substring(0, s.length - 1).trim() : s; - if (body.contains(';')) { + // (d) Single statement. + if (masked.contains(';')) { throw SqlGuardError('Only one statement is allowed.'); } - // (c) Must start with SELECT or WITH. - final lower = body.toLowerCase(); + // (e) Must start with SELECT or WITH. + final lower = masked.toLowerCase(); if (!(lower.startsWith('select') || lower.startsWith('with'))) { throw SqlGuardError('Query must start with SELECT or WITH.'); } - // (d) Tokenize; reject any banned keyword + collect identifiers. - final tokens = RegExp(r'[A-Za-z_][A-Za-z0-9_]*') - .allMatches(lower) - .map((m) => m.group(0)!) - .toList(); - for (final t in tokens) { - if (_banned.contains(t)) throw SqlGuardError('Disallowed keyword: $t'); - if (_denyTables.contains(t)) throw SqlGuardError('Table not allowed: $t'); - } - // (e) CTE names declared via WITH … AS ( become valid FROM targets. - final cte = {}; - for (final m in RegExp(r'(?:with|,)\s+([A-Za-z_][A-Za-z0-9_]*)\s+as\s*\(', - caseSensitive: false) - .allMatches(lower)) { - cte.add(m.group(1)!); - } - // (f) Every FROM/JOIN target must be an allowed view (or a CTE name). - final refRe = RegExp(r'\b(?:from|join)\s+("?)([A-Za-z_][A-Za-z0-9_.]*)\1', - caseSensitive: false); - final refs = []; - for (final m in refRe.allMatches(lower)) { - final id = m.group(2)!; - if (id.contains('.')) { - throw SqlGuardError('Schema-qualified names are not allowed: $id'); + + final toks = _lex(lower); + + // (f) Banned keywords + reserved relation names, anywhere. + for (final t in toks) { + if (!t.ident) continue; + if (_banned.contains(t.text)) { + throw SqlGuardError('Disallowed keyword: ${t.text}'); + } + if (t.text.startsWith('sqlite_') || t.text.startsWith('pragma_')) { + throw SqlGuardError('SQLite internals are not queryable: ${t.text}'); + } + if (reservedTableNames.contains(t.text)) { + throw SqlGuardError('Table not allowed: ${t.text}. ' + 'Allowed views: ${allowedViews.join(', ')}.'); } - refs.add(id); } - if (refs.isEmpty) throw SqlGuardError('No table reference found.'); - for (final r in refs) { - if (!allowedViews.contains(r) && !cte.contains(r)) { - throw SqlGuardError( - 'Table "$r" is not queryable. Allowed views: ${allowedViews.join(', ')}.'); + + // (g) CTE names declared via ` AS (` become valid table targets. + // A CTE may not shadow an allowed view (that would silently redefine + // the coach's data surface) — shadowing a REAL table is already + // impossible because (f) rejects every reserved name outright. + final cte = _cteNames(toks); + for (final c in cte) { + if (allowedViews.contains(c)) { + throw SqlGuardError('A CTE may not shadow the view "$c".'); } } - // (g) Auto-append a LIMIT to protect context. + + // (h) THE allow-list gate: parse every table position and require it to be + // an allowed view or a declared CTE. Comma-separated cross-joins, + // subqueries in FROM, table-valued functions and schema-qualified + // names are all handled here (the old single-regex version only ever + // saw the identifier directly after FROM/JOIN). + final refs = _tableRefs(toks, allowed: {...allowedViews, ...cte}); + if (refs.isEmpty) throw SqlGuardError('No table reference found.'); + + // (i) Auto-append a LIMIT to protect context. if (!RegExp(r'\blimit\b', caseSensitive: false).hasMatch(lower)) { body = '$body LIMIT $rowCap'; } return body; } + /// Replace every single-quoted string literal with the neutral token `0`. + /// Rejects unterminated literals and every quoted-identifier form. + static String _maskStrings(String src) { + final out = StringBuffer(); + var i = 0; + while (i < src.length) { + final c = src[i]; + if (c == "'") { + i++; + var closed = false; + while (i < src.length) { + if (src[i] == "'") { + if (i + 1 < src.length && src[i + 1] == "'") { + i += 2; + continue; + } + i++; + closed = true; + break; + } + i++; + } + if (!closed) throw SqlGuardError('Unterminated string literal.'); + out.write('0'); + continue; + } + if (c == '"' || c == '`' || c == '[' || c == ']') { + throw SqlGuardError( + 'Quoted identifiers are not allowed — use bare view names.'); + } + out.write(c); + i++; + } + return out.toString(); + } + + static List<_Tok> _lex(String s) { + final out = <_Tok>[]; + var i = 0; + bool identStart(String c) => + (c.codeUnitAt(0) >= 97 && c.codeUnitAt(0) <= 122) || c == '_'; + bool identPart(String c) => + identStart(c) || (c.codeUnitAt(0) >= 48 && c.codeUnitAt(0) <= 57); + bool digit(String c) => c.codeUnitAt(0) >= 48 && c.codeUnitAt(0) <= 57; + while (i < s.length) { + final c = s[i]; + if (c.trim().isEmpty) { + i++; + continue; + } + if (identStart(c)) { + final st = i; + while (i < s.length && identPart(s[i])) { + i++; + } + out.add(_Tok(s.substring(st, i), true)); + continue; + } + if (digit(c)) { + final st = i; + while (i < s.length && (digit(s[i]) || s[i] == '.')) { + i++; + } + out.add(_Tok(s.substring(st, i), false)); + continue; + } + out.add(_Tok(c, false)); + i++; + } + return out; + } + + /// CTE names: an identifier preceded by `with`/`recursive`/`,` and followed + /// (past an optional parenthesised column list) by `as (`. + static Set _cteNames(List<_Tok> toks) { + final names = {}; + for (var i = 1; i < toks.length; i++) { + final prev = toks[i - 1]; + final t = toks[i]; + if (!t.ident) continue; + final opensCte = (prev.ident && (prev.text == 'with' || prev.text == 'recursive')) || + (!prev.ident && prev.text == ','); + if (!opensCte) continue; + var j = i + 1; + if (j < toks.length && !toks[j].ident && toks[j].text == '(') { + j = _skipParens(toks, j); + } + if (j < toks.length && toks[j].ident && toks[j].text == 'as') { + var k = j + 1; + // AS [NOT] MATERIALIZED ( + while (k < toks.length && + toks[k].ident && + (toks[k].text == 'not' || toks[k].text == 'materialized')) { + k++; + } + if (k < toks.length && !toks[k].ident && toks[k].text == '(') { + names.add(t.text); + } + } + } + return names; + } + + /// Index just past the balanced ')' that matches the '(' at [open]. + static int _skipParens(List<_Tok> toks, int open) { + var depth = 0; + for (var i = open; i < toks.length; i++) { + if (toks[i].ident) continue; + if (toks[i].text == '(') depth++; + if (toks[i].text == ')') { + depth--; + if (depth == 0) return i + 1; + } + } + return toks.length; + } + + /// Every table position in the statement, validated against [allowed]. + /// Walks the FULL table-reference list after each FROM/JOIN — including the + /// comma-separated members of an implicit cross-join. + static List _tableRefs(List<_Tok> toks, {required Set allowed}) { + final refs = []; + for (var i = 0; i < toks.length; i++) { + final t = toks[i]; + if (!t.ident || (t.text != 'from' && t.text != 'join')) continue; + var j = i + 1; + while (true) { + if (j >= toks.length) { + throw SqlGuardError('Malformed FROM clause.'); + } + final n = toks[j]; + if (!n.ident) { + if (n.text == '(') { + throw SqlGuardError( + 'Subqueries in FROM are not allowed — query a view directly.'); + } + throw SqlGuardError('Expected a view name after FROM/JOIN.'); + } + final name = n.text; + j++; + // Schema-qualified (main.x / temp.x) or a table-valued function. + if (j < toks.length && !toks[j].ident && toks[j].text == '.') { + throw SqlGuardError('Schema-qualified names are not allowed: $name'); + } + if (j < toks.length && !toks[j].ident && toks[j].text == '(') { + throw SqlGuardError('Functions are not allowed in FROM: $name'); + } + if (!allowed.contains(name)) { + throw SqlGuardError( + 'Table "$name" is not queryable. Allowed views: ' + '${allowedViews.join(', ')}.'); + } + refs.add(name); + // Optional alias: `AS x` or a bare `x`. + if (j < toks.length && toks[j].ident && toks[j].text == 'as') { + j++; + if (j >= toks.length || !toks[j].ident) { + throw SqlGuardError('Malformed alias after "$name".'); + } + j++; + } else if (j < toks.length && + toks[j].ident && + !_clauseKeywords.contains(toks[j].text)) { + j++; + } + // Another member of the same (comma-separated) table list? + if (j < toks.length && !toks[j].ident && toks[j].text == ',') { + j++; + continue; + } + break; + } + i = j - 1; + } + return refs; + } + + // ── layer 2: structural btree gate (SQLite's own answer) ─────────────────── + + /// Root pages of every btree the ALLOWED VIEWS themselves read. Computed + /// once per read-only handle by EXPLAINing `SELECT * FROM `; SQLite + /// expands the view and reports exactly which base tables/indexes it opens. + static Future> _allowedRootPages(Database db) async { + final cached = _allowedRoots; + if (cached != null) return cached; + final roots = {}; + for (final v in allowedViews) { + try { + roots.addAll(await _btreeRoots(db, 'SELECT * FROM $v')); + } catch (_) {/* a view that can't compile contributes nothing */} + } + // Indexes on an allowed base table are allowed too (the planner may pick + // one for a WHERE the plain scan above never used). + final schema = await db.rawQuery( + 'SELECT type, name, tbl_name, rootpage FROM sqlite_master'); + final tableOfRoot = {}; + for (final r in schema) { + final rp = (r['rootpage'] as num?)?.toInt(); + final tbl = r['tbl_name']?.toString(); + if (rp != null && rp > 0 && tbl != null) tableOfRoot[rp] = tbl; + } + final allowedTables = { + for (final rp in roots) + if (tableOfRoot[rp] != null) tableOfRoot[rp]!, + }; + for (final e in tableOfRoot.entries) { + if (allowedTables.contains(e.value)) roots.add(e.key); + } + _allowedRoots = roots; + return roots; + } + + /// Root pages of every btree a prepared [sql] opens, per SQLite's own + /// bytecode. Only cursor-opening opcodes carry a root page in P2 — + /// OpenEphemeral/SorterOpen/OpenPseudo reuse P2 for a column count and are + /// deliberately ignored. + static Future> _btreeRoots(Database db, String sql) async { + const opening = {'OpenRead', 'OpenWrite', 'ReopenIdx'}; + final rows = await db.rawQuery('EXPLAIN $sql'); + final out = {}; + for (final r in rows) { + final op = r['opcode']?.toString(); + if (op == null || !opening.contains(op)) continue; + final p2 = (r['p2'] as num?)?.toInt(); + if (p2 != null && p2 > 0) out.add(p2); + } + return out; + } + + /// Fail-closed structural gate: reject unless every btree the statement + /// opens belongs to a base table one of the allowed views is built from. + static Future _assertAllowedBtrees(Database db, String sql) async { + final allowed = await _allowedRootPages(db); + if (allowed.isEmpty) { + throw SqlGuardError('Query surface unavailable.'); + } + final Set used; + try { + used = await _btreeRoots(db, sql); + } catch (e) { + throw SqlGuardError('Query could not be compiled: $e'); + } + for (final r in used) { + if (!allowed.contains(r)) { + throw SqlGuardError('Query reaches storage outside the coach views. ' + 'Allowed views: ${allowedViews.join(', ')}.'); + } + } + } + + /// Test seam for layer 2 on its own (layer 1 normally rejects these first). + @visibleForTesting + static Future debugAssertAllowedBtrees(String sql) async => + _assertAllowedBtrees(await _readonly(), sql); + /// Run an LLM SELECT and return compact JSON for the tool result. On a guard /// rejection, returns the reason (so the model fixes its query) — never throws. static Future runCoachSql(String llmSql, {int rowCap = 200}) async { @@ -149,6 +471,7 @@ class CoachDb { } try { final db = await _readonly(); + await _assertAllowedBtrees(db, sql); final rows = await db.rawQuery(sql); final shown = rows.take(rowCap).toList(); final out = { @@ -164,8 +487,16 @@ class CoachDb { var s = jsonEncode(out); if (s.length > 16000) s = '${s.substring(0, 16000)}…(truncated)'; return s; + } on SqlGuardError catch (e) { + return jsonEncode({'error': e.reason}); } catch (e) { return jsonEncode({'error': 'Query failed: $e'}); } } } + +class _Tok { + final String text; + final bool ident; + const _Tok(this.text, this.ident); +} diff --git a/lib/coach/coach_engine.dart b/lib/coach/coach_engine.dart index 3cfade04..d60fe120 100644 --- a/lib/coach/coach_engine.dart +++ b/lib/coach/coach_engine.dart @@ -191,6 +191,51 @@ class CoachEngine { CoachEngine({required this.config, required this.api, this.storageKey = 'anon'}); + // ── prompt size ceilings ──────────────────────────────────────────────────── + // + // The coach's tools read the on-device health database and every result is + // resent, verbatim, on EVERY subsequent turn. Without a ceiling a model that + // keeps widening its queries would eventually serialize the whole database + // into a request bound for a third-party endpoint. Three bounds, all + // independent of the provider's own context limit: + // • per tool result — one query can't dominate the window; + // • rolling history — the resent conversation is bounded in BYTES, not + // just in message count (60 × 16 KB was ~1 MB); + // • per request — a hard fail-closed ceiling in [postChat]. + + /// Max characters of any single tool result kept in the resent history. + static const int kMaxToolResultChars = 16000; + + /// Max characters of running history resent on each turn. + static const int kMaxHistoryChars = 120000; + + /// Hard ceiling on one serialized provider request body. + static const int kMaxRequestBytes = 400 * 1024; + + static String _clipToolResult(String s) => s.length <= kMaxToolResultChars + ? s + : '${s.substring(0, kMaxToolResultChars)}…(truncated — narrow the query)'; + + int _historyChars() { + var n = 0; + for (final m in _history) { + n += jsonEncode(m).length; + } + return n; + } + + /// Bound the resent history in bytes, dropping WHOLE turns from the oldest + /// end so a `tool` message never outlives the assistant turn whose + /// `tool_calls` it answers (providers 400 on an orphaned tool message). + void _trimHistory() { + while (_historyChars() > kMaxHistoryChars && _history.length > 1) { + _history.removeAt(0); + while (_history.length > 1 && _history.first['role'] != 'user') { + _history.removeAt(0); + } + } + } + void reset() { _history.clear(); transcript.clear(); } bool get hasHistory => _history.isNotEmpty; @@ -392,6 +437,7 @@ class CoachEngine { const maxIters = 10; for (var i = 0; i < maxIters; i++) { onStatus(_shenanigans[_rand.nextInt(_shenanigans.length)]); + _trimHistory(); final messages = >[ {'role': 'system', 'content': '$kCoachSystemPrompt\n\nToday is ${_today()} (device-local date; all day-keyed data uses these local dates).'}, ..._history, @@ -429,7 +475,12 @@ class CoachEngine { onStatus(_statusFor(name, args)); final result = await _runTool(name, args, onItem: emit, confirm: confirm); - _history.add({'role': 'tool', 'tool_call_id': id, 'name': name, 'content': result}); + _history.add({ + 'role': 'tool', + 'tool_call_id': id, + 'name': name, + 'content': _clipToolResult(result), + }); } } emit(CoachItem.assistant('I dug through several steps but couldn’t wrap that up — try narrowing the question.')); @@ -533,6 +584,17 @@ class CoachEngine { ..remove('top_p') ..remove('top_k'); } + // FAIL-CLOSED size ceiling. Nothing leaves the device until this passes — + // the request is never truncated and silently sent, it is refused, so a + // runaway tool loop cannot ship the health database to a third party. + final payload = jsonEncode(body); + if (payload.length > kMaxRequestBytes) { + throw CoachException( + 'That request grew to ${payload.length ~/ 1024} KB, over the ' + '${kMaxRequestBytes ~/ 1024} KB safety limit for data leaving this ' + 'device. Start a new chat or ask a narrower question (aggregate with ' + 'AVG/MIN/MAX/COUNT instead of selecting every row).'); + } try { final resp = await c .post( @@ -541,17 +603,41 @@ class CoachEngine { 'Authorization': 'Bearer ${config.apiKey}', 'content-type': 'application/json', }, - body: jsonEncode(body), + body: payload, ) .timeout(const Duration(seconds: 120)); if (resp.statusCode != 200) { throw CoachException( 'Provider error (${resp.statusCode}): ${_briefErr(resp.body)}'); } - final j = jsonDecode(utf8.decode(resp.bodyBytes)); + final Object? j; + try { + j = jsonDecode(utf8.decode(resp.bodyBytes)); + } catch (_) { + throw CoachException( + 'Provider returned a non-JSON response. Check the API base URL — ' + 'it must point at an OpenAI-compatible /chat/completions endpoint.'); + } + if (j is! Map) throw CoachException('Unexpected response from provider.'); final choices = (j['choices'] as List?) ?? const []; if (choices.isEmpty) throw CoachException('Empty response from provider.'); - return (choices.first as Map)['message'] as Map; + // Every shape below is a REAL thing OpenAI-compatible proxies return: + // a streaming chunk (`delta` instead of `message`), the legacy + // completions shape (`text`), or a bare string. Reaching for + // `choices.first['message'] as Map` blind surfaced a raw + // TypeError ("type 'Null' is not a subtype of type 'Map'") instead of the documented CoachException, so the UI showed + // a Dart type name to the user rather than an actionable message. + final first = choices.first; + if (first is! Map) throw CoachException('Unexpected response from provider.'); + final msg = first['message'] ?? first['delta']; + if (msg is Map) return msg.cast(); + final text = first['text']; + if (text is String) return {'content': text}; + throw CoachException( + 'Provider returned an unsupported response shape (no message/delta). ' + 'Streaming-only endpoints are not supported — use a standard ' + 'OpenAI-compatible /chat/completions endpoint.'); } finally { if (client == null) c.close(); } diff --git a/lib/coach/coach_prompt.dart b/lib/coach/coach_prompt.dart index 01940317..148413d7 100644 --- a/lib/coach/coach_prompt.dart +++ b/lib/coach/coach_prompt.dart @@ -57,10 +57,15 @@ the user's DERIVED data (raw signals are intentionally unavailable). Tables are - v_baselines(key, value, mean, z, delta, ratio, n, updated_at) — rolling personal baselines. - v_insights(id, kind, title, body, date, created_at, read) — the local insight/alert feed. -Rules: SELECT only, derived views only (no raw/base tables), dates are 'YYYY-MM-DD', timestamps -are epoch SECONDS, irregular_flag/irregular_rhythm_flag are 1/0. Prefer aggregates -(AVG/MIN/MAX/COUNT, GROUP BY) over selecting many rows; results cap at 200 rows. If a query is -rejected, read the error and fix it. Examples: +Rules: SELECT (or WITH … SELECT) only, ONE statement, no comments. Every table position — after +FROM, after JOIN, and every extra member of a comma-separated list — must be one of the v_* views +above or a CTE you declared in the same statement; nothing else is queryable, including base +tables, sqlite_master, schema-qualified names ("main.v_daily"), quoted identifiers, table +functions, and subqueries in FROM (put the subquery in a WITH instead). Subqueries in +SELECT/WHERE over the views are fine. Dates are 'YYYY-MM-DD', timestamps are epoch SECONDS, +irregular_flag/irregular_rhythm_flag are 1/0. Prefer aggregates (AVG/MIN/MAX/COUNT, GROUP BY) +over selecting many rows; results cap at 200 rows. If a query is rejected, read the error and fix +it. Examples: SELECT date, resting_hr, hrv, readiness FROM v_daily ORDER BY date DESC LIMIT 30 SELECT AVG(strain) FROM v_daily WHERE date >= '2026-06-01' SELECT t, v FROM v_series WHERE date='2026-06-29' AND series='hr_curve' ORDER BY t diff --git a/lib/compute/crossday_pipeline.dart b/lib/compute/crossday_pipeline.dart index ba4e036a..55a94bad 100644 --- a/lib/compute/crossday_pipeline.dart +++ b/lib/compute/crossday_pipeline.dart @@ -14,6 +14,8 @@ // which is a deterministic function of the input string. Safe for Isolate.run // and directly unit-testable. +import 'dart:math' as math; + import 'package:openstrap_analytics/onehz.dart' as ana; /// Build the cross-day analytics bundle from a time-ordered (OLDEST FIRST) list @@ -64,14 +66,18 @@ Map buildCrossDayBundle( final anomaly = ana.multivariateAnomaly(dates, feats); // ── CTL/ATL/TSB training load from the daily-TRIMP series ────────────────── - // Only days that actually carry a TRIMP (time-ordered) contribute; a missing - // TRIMP is NOT a 0-load impulse here — we simply omit it (the user may not - // have worn / trained that day, and the EWMA in the package treats the gaps - // it does see as decay over the series it receives). - final dailyTrimp = [ - for (final d in days) - if (d['trimp'] != null) (_numOrNull(d['trimp']) ?? 0.0), - ]; + // `ctlAtlTsb` is an EWMA that treats its input as ONE SAMPLE PER CALENDAR DAY + // (λ = 1 − e^(−1/τ) per element) and documents "a missing day contributes a + // 0-load impulse (rest day) — the EWMA decays". Filtering to only the days + // that carry a TRIMP handed it a COMPRESSED calendar instead: a user who + // trains sporadically (say 10 loaded days across 90) got 10 consecutive + // "days" of load with no decay between them, so fitness/fatigue never fell + // across the gaps and TSB was systematically wrong. Build a DENSE per-day + // series over the observed date span instead — a day with no TRIMP, and a + // calendar day with no derived row at all, are both 0-load impulses. + final dailyTrimp = _denseDailyTrimp(dates, [ + for (final d in days) _numOrNull(d['trimp']), + ]); final load = ana.ctlAtlTsb(dailyTrimp); // ── skin-temp illness flag (Smarr, cycle-aware) ──────────────────────────── @@ -275,11 +281,19 @@ Map buildCrossDayBundle( final latestAnomaly = anomaly.isEmpty ? null : anomaly.last; final latestTemp = tempIllness.isEmpty ? null : tempIllness.last; - // per-day flags (for notifications / trends): asleep/illness/anomaly/temp. + // per-day flags (for notifications / trends): asleep/illness/anomaly/temp, + // plus the nightly RESTING HR the "your resting HR trend shifted" CUSUM + // notification reads back off these rows. That consumer + // (`DerivationEngine._runNotifications`) needed `r['rhr']` and this builder + // never emitted it, so its series was always empty, its `length >= 10` gate + // never passed, and the notification was dead code. The value is already + // computed right here (`rhrList`, parallel to `dates`) — emit it rather than + // delete a wanted feature. Null stays null (the consumer filters on `is num`). final recent = >[]; for (var i = 0; i < n; i++) { recent.add({ 'date': dates[i], + 'rhr': rhrList[i], 'illness': i < illness.length && illness[i].state == ana.IllnessState.red, 'anomaly': i < anomaly.length && anomaly[i].flagged, 'temp': @@ -364,6 +378,52 @@ bool _isFreeDay(String? date) { return dt.weekday == DateTime.saturday || dt.weekday == DateTime.sunday; } +/// Guard on how far [_denseDailyTrimp] will densify. Past this the input is not +/// a plausible recent-history window (a corrupt/absurd date), so we fall back to +/// the raw per-row series rather than allocating an unbounded list. +const int _maxDenseTrimpDays = 400; + +/// A DENSE per-calendar-day TRIMP series (oldest→newest) over the span covered +/// by [dates], with every unloaded or unobserved day contributing a 0 impulse. +/// +/// [dates] and [trimps] are parallel (one entry per derived day, oldest first); +/// [dates] may skip calendar days entirely (an underived day has no row). +/// PURE: `DateTime.parse` / `DateTime(y, m, d + 1)` are deterministic functions +/// of the input strings, and the day-step normalizes DST-length days correctly. +List _denseDailyTrimp(List dates, List trimps) { + final byDate = {}; + String? first, last; + for (var i = 0; i < dates.length && i < trimps.length; i++) { + final date = dates[i]; + if (date.isEmpty || DateTime.tryParse(date) == null) continue; + byDate[date] = trimps[i] ?? 0.0; + first ??= date; + if (last == null || date.compareTo(last) > 0) last = date; + if (date.compareTo(first) < 0) first = date; + } + if (first == null || last == null) return const []; + final start = DateTime.parse(first); + final end = DateTime.parse(last); + final out = []; + var cursor = DateTime(start.year, start.month, start.day); + final stop = DateTime(end.year, end.month, end.day); + while (!cursor.isAfter(stop)) { + if (out.length >= _maxDenseTrimpDays) { + // Implausible span — don't densify; the honest per-row series is better + // than an arbitrarily long zero-padded one. + return [for (final v in trimps) v ?? 0.0]; + } + out.add(byDate[_dateKey(cursor)] ?? 0.0); + cursor = DateTime(cursor.year, cursor.month, cursor.day + 1); + } + return out; +} + +String _dateKey(DateTime d) { + String two(int x) => x.toString().padLeft(2, '0'); + return '${d.year.toString().padLeft(4, '0')}-${two(d.month)}-${two(d.day)}'; +} + /// Absolute value of each present element (used to orient skin-temp by |z|). List _absList(List xs) => [for (final v in xs) v?.abs()]; @@ -436,11 +496,21 @@ ana.Metric _crossDaySri(List> days) { // line ~100) already exists to avoid. final startMin = _localTodMin(start.round()).floor(); // end is exclusive of the segment's trailing edge; cover [start,end). - final endMin = _localTodMin(end.round()).ceil(); + final endMinRaw = _localTodMin(end.round()).ceil(); + // WRAP, don't clamp. Both bounds are clock-minutes-of-day in [0,1440), + // so a segment crossing local midnight reads e.g. start=1430, + // end=20 — and the old `for (m = startMin; m < endMin; m++)` simply + // never executed, silently DROPPING that segment (the comment claimed + // it "clamps into grid"; it didn't). Every night has exactly one such + // segment, so sleep-regularity was always computed with a hole at the + // boundary. Unwrap the end past 1440 and write back modulo the grid. + // `endMinRaw == startMin` stays a genuinely empty (sub-minute) segment. + final endMin = + endMinRaw < startMin ? endMinRaw + epochsPerDay : endMinRaw; + final span = math.min(endMin - startMin, epochsPerDay); final asleepSeg = stage != null && stage != 'wake'; - for (var m = startMin; m < endMin; m++) { - // Guard wrap: a segment that crosses midnight just clamps into grid. - if (m < 0 || m >= epochsPerDay) continue; + for (var k = 0; k < span; k++) { + final m = (startMin + k) % epochsPerDay; cov[m] = true; if (asleepSeg) asleep[m] = true; } diff --git a/lib/compute/derivation_engine.dart b/lib/compute/derivation_engine.dart index f994a9e5..7f59d3f3 100644 --- a/lib/compute/derivation_engine.dart +++ b/lib/compute/derivation_engine.dart @@ -264,7 +264,38 @@ import 'substrate.dart'; // the real Baevsky SI was absent, violating the never-impute rule; the UI // already correctly renders "—" when `score` is null. Bump so affected days // re-derive without a same-day, no-sleep readiness/stress score. -const int kAlgoVersion = 47; +// v48: audit sweep across compute + the sibling analytics package. +// +// EDGE-LOCAL changes, which ship the moment this constant lands: +// - The per-sweep baseline snapshot is genuinely frozen. `appendScalars` is +// gone; the history is dated and loaded once, and `valuesBefore(key, date)` +// excludes the target day from its own baseline. A re-derive sweep could +// previously append each finished day back into the shared window and evict +// a real old day, collapsing median/MAD toward duplicated recent values — +// the same pollution shape edge#108 fixed on the load path. +// - A day is no longer allowed to sit inside its own readiness baseline, and +// lnRMSSD no longer double-appends today. +// - `nocturnalRhr` is now fed the positionally-dense day series instead of a +// compacted one, so its 30-minute window is real wall-clock again. +// - Profile imputation (age 30 / 70 kg / sex m / RHR 60) no longer persists +// strain, calories and zones as if they were measured. +// - SRI no longer drops the one hypnogram segment per night that crosses +// local midnight; CTL/ATL/TSB now sees a calendar-dense series so fitness +// and fatigue decay across rest days. +// - Historical days resolve their timezone offset at their own timestamp +// rather than through today's offset. +// +// ⚠️ SIBLING-PACKAGE changes are NOT in this bump yet. The analytics fixes from +// the same sweep (sleep no longer reporting a no-data window as light sleep, +// the Lipponen-Tarvainen threshold on the signed dRR series, abstention on +// degenerate dispersion, the reconciled TRIMP stack, circular social jetlag) +// live in an analytics commit that `pubspec.yaml` does NOT yet point at — the +// pin is still the previous SHA. Do NOT extend this changelog to claim them +// until that pin is updated to a merged analytics commit and verified with +// `git show :`. Citing an analytics change that the pinned SHA never +// contained is exactly how the v43 changelog documented a readiness fix that +// stayed broken for three releases; when the pin moves, bump again. +const int kAlgoVersion = 48; /// Raw is kept this many days past derivation, then pruned (derived stays). const int rawRetentionDays = 3; @@ -326,6 +357,21 @@ const int _headlineFreezeMarginSec = 60 * 60; Future> debugBaselineWindow(String key) async => (await _BaselineHistoryCache.load()).values(key); +/// Test seam: the EXACT per-day baseline windows one derivation sweep would feed +/// the readiness pass, in dispatch order (`orderedDays` is newest-first). +/// +/// Pins the two properties the sweep path must have — the snapshot is loaded +/// ONCE and never mutated as days complete, and each day's window self-excludes +/// that day's own date. Both were violated by the old `appendScalars` sweep. +@visibleForTesting +Future>> debugSweepBaselineWindows( + String key, + List orderedDays, +) async { + final history = await _BaselineHistoryCache.load(); + return [for (final day in orderedDays) history.valuesBefore(key, day)]; +} + @visibleForTesting ({List days, String reason}) selectLightDeriveDays({ required Set rawDays, @@ -350,10 +396,37 @@ class _DeriveScope { }); } +/// One (date, value) sample of a baseline series. +typedef _DatedValue = ({String date, double value}); + class _BaselineHistoryCache { _BaselineHistoryCache(this._series); - final Map> _series; + /// The baseline series this cache carries, keyed by `metric_series.key`. + static const List keys = [ + 'ln_rmssd', + 'rmssd', + 'rhr', + 'resp_rate', + 'skin_temp_adc', + 'readiness', + ]; + + /// DATED baseline samples, ascending by date, one entry per day (metric_series + /// is keyed `(date, key)` with REPLACE, so it is structurally de-duplicated). + /// + /// IMMUTABLE for the lifetime of one derivation sweep. There is deliberately + /// no mutator: the previous `appendScalars` mutated this shared snapshot as + /// each day of a sweep finished, which re-introduced exactly the duplicate-day + /// pollution the load path was rewritten to prevent — `load()` had ALREADY + /// read the persisted values of the days about to be re-derived, so appending + /// each finished day again (and evicting a real old day to stay at 28) left + /// later days in the sweep reading a window with up to 21 duplicated recent + /// values in descending date order. Median/MAD then collapsed toward the + /// repeated value and readiness went blank/wrong — the load-path bug, moved + /// into the sweep path. A sweep now reads ONE frozen snapshot, and each day + /// derives its own window from it by date. + final Map> _series; /// Load the rolling baseline window that feeds the readiness/illness /// computations. This ALWAYS rebuilds from `metric_series` — the canonical @@ -361,57 +434,73 @@ class _BaselineHistoryCache { /// value per day. /// /// We deliberately do NOT trust the persisted `rolling_artifact` for history. - /// That artifact is written from an in-memory cache that [appendScalars] only - /// appends to (no day identity), so repeated same-day re-derives could stack - /// duplicate copies of today into the window; once enough slots matched, the - /// readiness composite's robust z-score hit MAD=0 and went absent — the blank - /// readiness ring. A polluted artifact is still valid JSON, so trusting it on - /// read would let that pollution reach the computation on the first - /// post-upgrade derive (and, when every day is finalized and `run()` does no - /// work, forever). Rebuilding from the de-duplicated store on every load makes - /// the read path immune and self-heals any already-polluted install. + /// That artifact was written from an in-memory cache with no day identity, so + /// repeated same-day re-derives could stack duplicate copies of today into the + /// window; once enough slots matched, the readiness composite's robust z-score + /// hit MAD=0 and went absent — the blank readiness ring. A polluted artifact + /// is still valid JSON, so trusting it on read would let that pollution reach + /// the computation on the first post-upgrade derive (and, when every day is + /// finalized and `run()` does no work, forever). Rebuilding from the + /// de-duplicated store on every load makes the read path immune and self-heals + /// any already-polluted install. + /// + /// NOTE ON THE QUERY: `LocalDb.metricSeries(key)` with NO `limit` is the whole + /// series, `date ASC` — the dates are what make a per-day `date < target` + /// window possible at all. It must NOT be given a `limit` (that is `date ASC + /// LIMIT n`, i.e. the OLDEST n — the opposite of a trailing window); the + /// trailing window is taken here, in Dart, per target day. static Future<_BaselineHistoryCache> load() async { - Future> hist(String key) => - LocalDb.trailingSeriesValues(key, _baselineWindowDays); - - final loaded = await Future.wait([ - hist('ln_rmssd'), - hist('rmssd'), - hist('rhr'), - hist('resp_rate'), - hist('skin_temp_adc'), - hist('readiness'), - ]); + Future> hist(String key) async { + final rows = await LocalDb.metricSeries(key); + final out = <_DatedValue>[]; + for (final row in rows) { + final date = row['date']; + final value = row['value']; + if (date is! String || date.isEmpty || value is! num) continue; + out.add((date: date, value: value.toDouble())); + } + return out; + } + + final loaded = await Future.wait([for (final k in keys) hist(k)]); return _BaselineHistoryCache({ - 'ln_rmssd': loaded[0], - 'rmssd': loaded[1], - 'rhr': loaded[2], - 'resp_rate': loaded[3], - 'skin_temp_adc': loaded[4], - 'readiness': loaded[5], + for (var i = 0; i < keys.length; i++) keys[i]: loaded[i], }); } - List values(String key) => - List.from(_series[key] ?? const []); - - void appendScalars(Map scalars) { - void add(String seriesKey, String scalarKey) { - final v = (scalars[scalarKey] as num?)?.toDouble(); - if (v == null) return; - final list = _series.putIfAbsent(seriesKey, () => []); - list.add(v); - while (list.length > _baselineWindowDays) { - list.removeAt(0); - } - } - - add('ln_rmssd', 'ln_rmssd'); - add('rmssd', 'rmssd'); - add('rhr', 'rhr'); - add('resp_rate', 'resp_rate'); - add('skin_temp_adc', 'skin_temp_adc'); - add('readiness', 'readiness'); + /// The trailing [_baselineWindowDays] values for [key], oldest→newest. + /// + /// This is the WHOLE window including the newest day; it backs the persisted + /// rolling artifact + the rescan signature, which describe "the baseline as it + /// currently stands". Per-day derivation must use [valuesBefore] instead. + List values(String key) => _trailing(_series[key] ?? const []); + + /// The trailing [_baselineWindowDays] values for [key] STRICTLY BEFORE + /// [beforeDate] (`date < ?`), oldest→newest — the baseline for deriving the + /// day labelled [beforeDate]. + /// + /// SELF-EXCLUSION IS THE POINT. The previous derive of the same day has + /// already written its own row to `metric_series`, so an unfiltered trailing + /// window contained TODAY: every light pass after the first z-scored today's + /// RHR/HRV/temp against a baseline that already contained today (pulling the + /// baseline toward the value under test and understating a genuinely + /// off day), and the lnRMSSD stack — which is contractually handed + /// `[...history, today]` and takes all-but-last as its baseline — counted it + /// a second time. v38 fixed precisely this self-inclusion inside analytics; + /// this is the same defect at the edge layer that feeds it. Dates strictly + /// AFTER the target are excluded too: a baseline is prior days, and a backfill + /// sweep must not let later days leak into an older day's baseline (which + /// would also make the result depend on sweep order). + List valuesBefore(String key, String beforeDate) => _trailing([ + for (final s in _series[key] ?? const <_DatedValue>[]) + if (s.date.compareTo(beforeDate) < 0) s, + ]); + + static List _trailing(List<_DatedValue> samples) { + final from = samples.length <= _baselineWindowDays + ? 0 + : samples.length - _baselineWindowDays; + return [for (var i = from; i < samples.length; i++) samples[i].value]; } Map toArtifactJson() { @@ -935,7 +1024,12 @@ class DerivationEngine { // The worker isolate dies after `Isolate.run`, so the recording flag can't // leak into the next day's derivation — no try/finally reset needed. final profileJson = await _loadSleepUserProfileJson(); - final (candidateJson, updatedProfileJson) = await Isolate.run(() { + // Cancellable + TIMED OUT. This site previously used a bare `Isolate.run` + // with no timeout at all, so a hung staging pass never completed its future + // — `_running` stayed true and `DeriveScheduler._drain` never returned, i.e. + // all derivation was dead until app restart. + final (candidateJson, updatedProfileJson) = + await _runIsolateCancellable(() { try { ana.cardioUserProfile = profileJson == null ? null @@ -972,7 +1066,7 @@ class DerivationEngine { } } return (jsonEncode(candidate.toJson()), foldedJson); - }); + }, _perDayTimeout, label: 'sleep-staging $dayId'); final candidate = SleepSessionCandidate.fromJson( (jsonDecode(candidateJson) as Map).cast()); if (override == null) { @@ -1015,13 +1109,35 @@ class DerivationEngine { }) async { if (toRecTs < fromRecTs) return Substrate.empty; final port = ReceivePort(); - final isolate = await Isolate.spawn(derivationPrepareWorker, port.sendPort); + // onError/onExit are LOAD-BEARING. Without them, an uncaught throw inside + // the worker (a malformed SQLite row reaching one of the numeric reads in + // its 'page' handler — the worker only ever reported errors from its + // 'finish' branch) killed the isolate silently, and this side awaited + // `result.future` FOREVER with `_running == true`: DeriveScheduler._drain + // never returned and ALL derivation was dead until app restart. Now a + // worker death fails the future, and the timeout below bounds the wait + // even if no signal arrives at all. + final isolate = await Isolate.spawn( + derivationPrepareWorker, + port.sendPort, + onError: port.sendPort, + onExit: port.sendPort, + ); final ready = Completer(); final result = Completer(); + // A failure completes BOTH completers, but we may bail out via `ready` and + // never await `result` — register a listener so that error is never an + // unobserved async error. (The real error still propagates via `ready`.) + unawaited(result.future.catchError((_) => Substrate.empty)); late final StreamSubscription sub; - sub = port.listen((message) async { + void fail(Object error) { + if (!ready.isCompleted) ready.completeError(error); + if (!result.isCompleted) result.completeError(error); + } + + sub = port.listen((message) { if (message is SendPort) { - ready.complete(message); + if (!ready.isCompleted) ready.complete(message); return; } if (message is Map && message['type'] == 'result') { @@ -1029,23 +1145,28 @@ class DerivationEngine { if (kind == 'substrate') { final payload = ((message['payload'] as Map?) ?? const {}) .cast(); - await sub.cancel(); - port.close(); - isolate.kill(priority: Isolate.immediate); - result.complete(Substrate.fromJson(payload)); + if (!result.isCompleted) result.complete(Substrate.fromJson(payload)); } return; } if (message is Map && message['type'] == 'error') { - await sub.cancel(); - port.close(); - isolate.kill(priority: Isolate.immediate); - result.completeError(Exception(message['error'])); + fail(Exception('prepare worker error: ${message['error']}')); + return; + } + if (message is List) { + // `onError` wire format ([error, stackTrace]) — an uncaught throw. + fail(Exception('prepare worker crashed: ' + '${message.isNotEmpty ? message.first : "no detail"}')); + return; + } + if (message == null) { + // `onExit` — the isolate ended without ever sending a result. + fail(StateError('prepare worker exited without a result')); } }); - final worker = await ready.future; - worker.send(const {'type': 'config', 'mode': 'substrate'}); try { + final worker = await ready.future; + worker.send(const {'type': 'config', 'mode': 'substrate'}); int? afterRecTs; int? afterCursor; var rangePages = 0; @@ -1091,12 +1212,22 @@ class DerivationEngine { break; } worker.send(const {'type': 'finish'}); - return result.future; - } catch (_) { + // BOUNDED. `result.future` had no timeout at all, so any path that left + // the worker unable to answer hung this call — and with it the whole + // engine — permanently. + return await result.future.timeout( + _perDayTimeout, + onTimeout: () => throw TimeoutException( + 'substrate prepare for $dayId timed out after $_perDayTimeout', + ), + ); + } finally { + // ALWAYS tear down: on success, on error, and on timeout. The isolate is + // killed rather than abandoned so a wedged worker can never outlive the + // call that spawned it. await sub.cancel(); port.close(); isolate.kill(priority: Isolate.immediate); - rethrow; } } @@ -1467,9 +1598,13 @@ class DerivationEngine { ); final withHistory = _attachHistory(input, history); - final bundle = await Isolate.run( + // Cancellable: on timeout the isolate is KILLED, not merely abandoned to + // keep burning a core behind the worker pool's back. + final bundle = await _runIsolateCancellable( () => deriveDayBundle(withHistory), - ).timeout(_perDayTimeout); + _perDayTimeout, + label: 'day-bundle ${day.date}', + ); _logSpo2Diagnostics(day, input, bundle); // Readiness came back absent for TODAY specifically (not a historical // backfill day, which would just be noise) — log why. This ran inside @@ -1547,6 +1682,33 @@ class DerivationEngine { final scMap = (bundle['scalars'] as Map?)?.cast(); + // ── NEVER WRITE NOTHING OVER SOMETHING ─────────────────────────────────── + // Raw retention is 3 days, but derived history is forever — so a day older + // than retention has a good `day_result` and NO raw. Re-deriving it (which + // "Advanced data → Select all → Re-analyze" does for EVERY listed day, via + // runDays(force: true) → _prepareTargetDay, whose empty substrate yields an + // all-absent bundle) used to overwrite that good row: `putDayResult` is + // ConflictAlgorithm.replace on BOTH `day_result` and `metric_series`, so + // every scalar for the date was NULLed — and, because an empty bundle's + // endSec was 0, the blank was written FINALIZED and could never re-derive. + // Only `run()` had a pruned-raw guard, and only for user-override days. + // + // Detect it BEFORE the offloaded second half so its own writes + // (wake_day_features) can't clobber the early-read path either. With no day + // substrate and no sleep substrate the second half has nothing to add — its + // scalars are all derived from those two. + final producedNothing = daySub.isEmpty && + sleepSub.isEmpty && + (scMap == null || !scMap.values.any((v) => v != null)); + if (producedNothing) { + final existing = await LocalDb.dayResult(day.date); + if (_isRealDayResult(existing)) { + _log('derive ${day.date}: no substrate (raw pruned) — kept the ' + 'existing result rather than blanking it'); + return; + } + } + // ── SECOND HALF — OFFLOADED to a background isolate ────────────────────── // Everything that turns the isolate-1 bundle into the full day result (wake // features, hybrid steps + TDEE, all-day HRV/RSA/skin-temp Timeline lines, @@ -1659,7 +1821,12 @@ class DerivationEngine { // workouts/HRR/wear/curves would never get a chance to be filled in by a // later retry. final ageFinalized = (day.endSec + _finalizationSec) < dataNowSec; - final finalized = forceFinalize || (ageFinalized && secondHalfOk); + // A result with NOTHING in it is never finalized — not even by + // forceFinalize. Locking an all-absent row is what made the destructive + // re-analyze permanent; leaving it unlocked means a later pass (or a + // restored/backfilled substrate) can still fill the day in. + final finalized = + !producedNothing && (forceFinalize || (ageFinalized && secondHalfOk)); final scalars = (bundle['scalars'] as Map?)?.cast() ?? const {}; @@ -1722,7 +1889,10 @@ class DerivationEngine { 'hrr_bpm': sc('hrr_bpm'), }, ); - history.appendScalars(scalars); + // NOTE: the sweep's `history` snapshot is deliberately NOT updated here. + // See _BaselineHistoryCache — mutating the shared snapshot mid-sweep is the + // duplicate-day pollution bug, and each day already derives its own + // date-bounded window from the frozen snapshot. _log( 'derived ${day.date} v$kAlgoVersion ' '(sleep=${day.sleepOffsetSec > day.sleepOnsetSec}, final=$finalized)', @@ -1844,7 +2014,48 @@ class DerivationEngine { _log('[spo2-detect] ${jsonEncode(payload)}'); } + /// Skip reasons that describe a TRANSIENT failure of this particular pass + /// rather than a permanently pathological day. These must never finalize: + /// finalizing locks the day out of every future pass at this algo version. + static const Set _transientSkipReasons = {'timeout', 'error'}; + + /// Whether [row] is a REAL derived day result worth protecting — i.e. not a + /// skip marker and not an all-absent shell. + static bool _isRealDayResult(Map? row) { + if (row == null) return false; + if ((row['skipped'] as num?)?.toInt() == 1) return false; + final payload = _decodeBundle(row['payload_json']); + if (payload == null) return false; + if (payload['skipped'] == true) return false; + final scalars = payload['scalars']; + if (scalars is Map && scalars.values.any((v) => v != null)) return true; + return row['rhr'] != null || row['rmssd'] != null || row['readiness'] != null; + } + + /// Test seam for [_markDaySkipped] — the "a skip marker must never destroy a + /// real result" guarantee is the whole point of the method, so it is pinned + /// directly rather than through a full derive pass. + @visibleForTesting + Future debugMarkDaySkipped( + String dayId, + int dayEndSec, + int dataNowSec, { + required String reason, + }) => + _markDaySkipped(dayId, dayEndSec, dataNowSec, reason: reason); + /// Persist a minimal skip marker so a pathological day isn't retried forever. + /// + /// A SKIP MARKER MUST NEVER OVERWRITE A REAL RESULT. `putDayResult` is + /// ConflictAlgorithm.replace on both `day_result` AND `metric_series`, so this + /// used to blank a good day's every scalar on a single [_perDayTimeout] + /// overrun — and, once the day sat >48 h behind the data edge, wrote the blank + /// FINALIZED, making it permanent (raw is pruned 3 days later, so there is + /// nothing left to re-derive from). It hit TODAY too: a good 08:00 result + /// replaced by a skip marker after one transient 09:00 timeout on a loaded + /// phone. `rescanRecent` explicitly refuses to do this for exactly this + /// reason; `run()` did it anyway. Now: write the marker only when there is no + /// good row to lose, and never lock a transient failure. Future _markDaySkipped( String dayId, int dayEndSec, @@ -1852,12 +2063,23 @@ class DerivationEngine { required String reason, }) async { try { + final existing = await LocalDb.dayResult(dayId); + if (_isRealDayResult(existing)) { + _log('derive $dayId $reason — existing result kept (not overwritten ' + 'with a skip marker)'); + return; + } await LocalDb.putDayResult( dayId: dayId, algoVersion: kAlgoVersion, payloadJson: jsonEncode({'skipped': true, 'reason': reason}), windowJson: '{}', - finalized: (dayEndSec + _finalizationSec) < dataNowSec, + // Structural failures (a day that can never be prepared / blows the + // prepare budget) still finalize once aged out, so they aren't retried + // forever. A timeout or a one-off error does not — that day gets + // another chance while it still has raw. + finalized: !_transientSkipReasons.contains(reason) && + (dayEndSec + _finalizationSec) < dataNowSec, skipped: true, ); } catch (_) { @@ -1865,23 +2087,30 @@ class DerivationEngine { } } - /// Attach trailing personal history (from metric_series) for the readiness pass. + /// Attach trailing personal history (from metric_series) for the readiness + /// pass — the trailing window of days STRICTLY BEFORE the day being derived. + /// + /// The self-exclusion (`date < input.date`) is load-bearing, not cosmetic: see + /// [_BaselineHistoryCache.valuesBefore]. Every one of these series is a + /// BASELINE the day's own value is scored against, so the day's own row (which + /// a previous derive of the same day already persisted) must not be in it. Map _attachHistory( DayBundleInput input, _BaselineHistoryCache history, ) { final m = input.toJson(); - m['ln_rmssd_history'] = history.values('ln_rmssd'); - m['rhr_history'] = history.values('rhr'); - m['resp_history'] = history.values('resp_rate'); + final date = input.date; + m['ln_rmssd_history'] = history.valuesBefore('ln_rmssd', date); + m['rhr_history'] = history.valuesBefore('rhr', date); + m['resp_history'] = history.valuesBefore('resp_rate', date); // Robust nocturnal RMSSD history (the `rmssd` series) — feeds the EWMA hrv // baseline so its center matches today's headline RMSSD (same metric). - m['rmssd_history'] = history.values('rmssd'); + m['rmssd_history'] = history.valuesBefore('rmssd', date); // BASELINE for skin_temp_z is the RAW nightly ADC-mean series (`skin_temp_adc`), // NOT the z-score series. Feeding z-scores back as the baseline was a unit // mismatch that left z permanently null. The raw mean is stored every day so // this series fills and z starts computing once ≥3 days exist. - m['skin_temp_adc_history'] = history.values('skin_temp_adc'); + m['skin_temp_adc_history'] = history.valuesBefore('skin_temp_adc', date); return m; } @@ -1904,9 +2133,11 @@ class DerivationEngine { // main isolate after Isolate.run returned it. Returning the already- // encoded string avoids both the main-isolate encode cost AND transfers // a flat string across the isolate boundary instead of a large nested Map. - final bundleJson = await Isolate.run( + final bundleJson = await _runIsolateCancellable( () => jsonEncode(buildCrossDayBundle(days, profileMap)), - ).timeout(_crossDayTimeout); + _crossDayTimeout, + label: 'crossday', + ); await LocalDb.putBaseline('crossday', bundleJson); _log('crossday: stored over ${days.length} day(s)'); } catch (e) { @@ -1945,7 +2176,7 @@ class DerivationEngine { // unconditionally on every heavy pass. _decodeBundle/_crossDayRecord are // both static, so this whole transform+encode step is isolate-safe. final rows = await LocalDb.recentDayResults(_crossDayWindow); - final (days, json) = await Isolate.run(() { + final (days, json) = await _runIsolateCancellable(() { final days = >[]; for (final row in rows.reversed) { final payload = _decodeBundle(row['payload_json']); @@ -1955,7 +2186,7 @@ class DerivationEngine { if (rec != null) days.add(rec); } return (days, jsonEncode({'algo_version': kAlgoVersion, 'days': days})); - }); + }, _crossDayTimeout, label: 'crossday-input'); await LocalDb.putBaseline('crossday_input', json); return days; } @@ -2406,19 +2637,29 @@ class DerivationEngine { for (final h in daySub.hr) if (h > 0) h.toDouble(), ]; - final age = profile.ageYears?.toDouble() ?? 30.0; // fallback age - final weightKg = profile.weightKg ?? 70.0; // fallback weight - final sex = profile.sex?.toLowerCase() ?? 'm'; // fallback sex - final hrMax = 208 - 0.7 * age; - // Fallback to 60.0 so new users (no baseline yet, no manual RHR) still get Strain - final rhrForTrimp = restingHr ?? profile.restingHrManual?.toDouble() ?? 60.0; + // NEVER IMPUTE A PROFILE. These used to default to age 30 / 70 kg / sex 'm' + // / RHR 60 "so new users still get Strain" — and the results were then + // persisted as REAL scalars (strain, calories, calories_total, steps) into + // day_result AND metric_series for someone who never entered a profile. + // That is a fabricated number wearing a real number's clothes, and it + // contradicts the never-impute contract the rest of this layer (and + // `Profile`'s own doc, and the pure `onehz_pipeline` which already gates on + // exactly these fields) enforces. A missing input now makes the DEPENDENT + // metric absent — the UI already renders "—" correctly. + final age = profile.ageYears?.toDouble(); + final weightKg = profile.weightKg; + final sex = profile.sex?.toLowerCase(); + final hrMax = profile.hrMaxTanaka; // null when age is unknown + final rhrForTrimp = restingHr ?? profile.restingHrManual?.toDouble(); double? strain; double? calories; double? steps; double? caloriesTotal; Map zones = const {}; - if (perMin.isNotEmpty) { - if (dayHrValid.isNotEmpty) { + if (perMin.isNotEmpty && hrMax != null) { + // TRIMP needs a real resting HR (nightly or user-supplied) and a real sex + // constant — both are in the Banister formula itself. + if (dayHrValid.isNotEmpty && rhrForTrimp != null && sex != null) { final trimp = ana.banisterTrimp( perMin, restingHr: rhrForTrimp, @@ -2430,19 +2671,17 @@ class DerivationEngine { if (score.present) strain = score.value; } } + // Zones are pure %HRmax bands — real as soon as HRmax is real. zones = _wakeZoneMinutes(daySub, sleepOnsetSec, sleepOffsetSec, hrMax); - // age/weightKg always have a value by this point (defaulted above), so - // this used to be a dead "if (age != null && weightKg != null && ...)" - // that flutter analyze flagged - there was never actually a gate here. - calories = _keytelCaloriesWake( - perMin, - age, - weightKg, - hrMax, - sex == 'f', - ); + // Keytel takes age, weight and sex directly. + if (age != null && weightKg != null && sex != null) { + calories = _keytelCaloriesWake(perMin, age, weightKg, hrMax, sex == 'f'); + } } if (motion.isNotEmpty) { + // Steps do NOT need a profile: `dailyStepEstimate` falls back to the day's + // own 10th-percentile HR when `restingHr` is null, which is data-derived, + // not imputed. Pass the real value or nothing — never the old 60.0. final stepMetric = ana.dailyStepEstimate( motion, hrPerMin: hrPerMinAll, @@ -2451,9 +2690,11 @@ class DerivationEngine { if (stepMetric.present && stepMetric.value != null) { steps = stepMetric.value!.steps.toDouble(); } - // same story - age/weightKg can't be null here, heightCm is the only - // field that actually still needs a null check. - if (profile.heightCm != null) { + // TDEE needs the full anthropometric set (Mifflin BMR + Keytel surplus). + if (age != null && + weightKg != null && + sex != null && + profile.heightCm != null) { final energy = ana.Calories.dailyEnergy( hrPerMinAll, profile: ana.WorkoutUserProfile( @@ -3068,6 +3309,85 @@ class DerivationEngine { } } + /// Run [compute] in an explicitly spawned isolate and enforce [timeout] ON THE + /// ISOLATE — the general-purpose form of [_runDayBlocksCancellable]. + /// + /// `Isolate.run(...).timeout(...)` only stops the CALLER awaiting; the spawned + /// isolate keeps burning CPU to completion in the background. With a bounded + /// per-day worker pool that silently blows the concurrency budget during a + /// backlog sweep — which is exactly why [_runDayBlocksCancellable] exists, and + /// it had been applied to only one of the file's isolate sites. Worse, some + /// sites (the sleep-staging pass) had NO timeout at all, so a hung isolate + /// wedged the engine with `_running == true` forever. + /// + /// Also wires `onError`/`onExit` so an uncaught throw or a silent death + /// FAILS the future instead of hanging it. + static Future _runIsolateCancellable( + FutureOr Function() compute, + Duration timeout, { + required String label, + }) async { + final port = ReceivePort(); + final (SendPort, FutureOr Function()) message = + (port.sendPort, compute); + final isolate = await Isolate.spawn( + _cancellableIsolateEntry, + message, + onError: port.sendPort, + onExit: port.sendPort, + ); + final completer = Completer(); + late final StreamSubscription sub; + sub = port.listen((msg) { + if (completer.isCompleted) return; + if (msg is _IsolateValue) { + completer.complete(msg.value as R); + } else if (msg is List) { + // Our caught-exception report or the `onError` port's uncaught-error + // format — both 2-element lists of strings. + completer.completeError( + StateError( + msg.isNotEmpty + ? '$label isolate failed: ${msg.first}' + : '$label isolate failed with no error detail', + ), + ); + } else if (msg == null) { + // `onExit` — the isolate ended without ever sending a result. + completer.completeError( + StateError('$label isolate exited without a result'), + ); + } + }); + try { + return await completer.future.timeout( + timeout, + onTimeout: () { + isolate.kill(priority: Isolate.immediate); + throw TimeoutException('$label timed out after $timeout'); + }, + ); + } finally { + await sub.cancel(); + port.close(); + // No-op if it already exited; guarantees a hung isolate never outlives + // this call. + isolate.kill(priority: Isolate.immediate); + } + } + + /// `Isolate.spawn` entry point for [_runIsolateCancellable]. + static Future _cancellableIsolateEntry( + (SendPort, FutureOr Function()) args, + ) async { + final (sendPort, compute) = args; + try { + sendPort.send(_IsolateValue(await compute())); + } catch (e, st) { + sendPort.send([e.toString(), st.toString()]); + } + } + /// `Isolate.spawn` entry point for [_runDayBlocksCancellable]. Must be a /// static/top-level function taking exactly one (sendable) argument. static void _dayBlocksIsolateEntry((SendPort, _DayBlocksInput) args) { @@ -3451,18 +3771,55 @@ class DerivationEngine { return 'error'; } + /// The substrate range to LOAD so [calendarDays] can actually run its + /// documented nocturnal search for [dayId]. + /// + /// `calendarDays` searches from the previous local NOON + /// (`dayStart − kNocturnalSearchLookbackSec`), and its comment records that + /// widening from the old prev-18:00 window as deliberate — "the old + /// prev-18:00 → noon window missed late wakes and forced the detector to act + /// like there was only one candidate sleep". But this loader only fetched + /// `dayStart − 6 h` (= 18:00), and `searchStart = math.max(dataStart, …)` + /// clipped the search right back to the slice start, so the widening was a + /// no-op and any sleep onset before 18:00 was truncated. Load the whole + /// window the day model asks for, from the one shared constant. (int, int) _targetDayWindow(String dayId) { final startSec = _localDayLabelToSec(dayId); final endSec = _localNextDayLabelToSec(dayId); - return (math.max(0, startSec - 6 * 3600), endSec - 1); + return (math.max(0, startSec - kNocturnalSearchLookbackSec), endSec - 1); } + /// Test seam for [_targetDayWindow] — the bug was that this loader and + /// [calendarDays]' search window silently disagreed, so the agreement is + /// pinned directly. + @visibleForTesting + (int, int) debugTargetDayWindow(String dayId) => _targetDayWindow(dayId); + void _log(String m) { if (kDebugMode) debugPrint('[derive] $m'); log?.call('[derive] $m'); } } +/// Wrapper for a cancellable-isolate result, so a computation whose OWN result +/// happens to be a `List` (the uncaught-error wire format) or `null` (the +/// `onExit` signal) can never be misread as a failure. +class _IsolateValue { + final Object? value; + const _IsolateValue(this.value); +} + +/// Test seam for [DerivationEngine._runIsolateCancellable] — the isolate +/// lifecycle guarantees (value / error / killed-on-timeout) are what the engine +/// depends on to never hang, so they're pinned directly. +@visibleForTesting +Future runCancellableIsolate( + FutureOr Function() compute, + Duration timeout, { + String label = 'test', +}) => + DerivationEngine._runIsolateCancellable(compute, timeout, label: label); + /// Sendable input for [DerivationEngine._computeDayBlocks] — crosses the /// `Isolate.run` boundary, so every field is plain data (Substrate is int/double /// lists; Profile/StepCalibration are primitive data classes). DB reads that the diff --git a/lib/compute/derive_prepare.dart b/lib/compute/derive_prepare.dart index 9923af24..d172128a 100644 --- a/lib/compute/derive_prepare.dart +++ b/lib/compute/derive_prepare.dart @@ -163,7 +163,16 @@ class SleepSessionCandidate { required Substrate sleepSub, }) => PreparedDerivationDay( date: dayId, - endSec: daySub.lastTs == null ? 0 : daySub.lastTs! + 1, + // `endSec` is what the engine anchors FINALIZATION on + // (`endSec + 48 h < dataNowSec` ⇒ lock). An empty substrate used to yield + // endSec = 0, which makes that comparison unconditionally true — so a day + // whose raw has been pruned (retention is 3 days) derived an all-absent + // bundle and wrote it FINALIZED over the good historical row, permanently. + // With no data, fall back to the day's real calendar end so an empty + // result is aged exactly like a real one. + endSec: daySub.lastTs != null + ? daySub.lastTs! + 1 + : localNextMidnightSecForDayLabel(dayId), confidence: confidence, flags: flags, sleepJson: sleepJson, @@ -176,13 +185,25 @@ class SleepSessionCandidate { ); } +/// Local midnight at the START OF THE NEXT day for a `YYYY-MM-DD` day label. +/// +/// `DateTime(y, m, d + 1)` normalizes the overflow itself and +/// `millisecondsSinceEpoch` respects local DST rules, so this is correct on the +/// two 23 h/25 h days a year — unlike `startOfDay + 86400`. +int localNextMidnightSecForDayLabel(String dayId) { + final d = DateTime.tryParse(dayId); + if (d == null) return 0; + return DateTime(d.year, d.month, d.day + 1).millisecondsSinceEpoch ~/ 1000; +} + void derivationPrepareWorker(SendPort mainSendPort) { final port = ReceivePort(); final state = _PrepareAccumulator(); String? targetDay; var mode = 'prepared_day'; mainSendPort.send(port.sendPort); - port.listen((message) { + + void handle(Object? message) { if (message is! Map) return; final type = message['type']; if (type == 'page') { @@ -230,12 +251,26 @@ void derivationPrepareWorker(SendPort mainSendPort) { 'payload': payload.toJson(), }); } - } catch (e, st) { - mainSendPort.send({'type': 'error', 'error': '$e\n$st'}); } finally { port.close(); } } + } + + // EVERY branch is guarded, not just 'finish'. A throw inside the 'page' + // handler (a malformed SQLite row reaching one of the numeric reads) used to + // kill this worker silently: the only error report lived in the 'finish' + // branch, so the main side awaited a Completer that could never complete — + // `_running` stayed true and DerivationEngine's scheduler never drained + // again, i.e. ALL derivation was dead until app restart. Now any failure is + // reported back and the port is closed so the isolate exits. + port.listen((message) { + try { + handle(message); + } catch (e, st) { + mainSendPort.send({'type': 'error', 'error': '$e\n$st'}); + port.close(); + } }); } @@ -325,6 +360,14 @@ class _PrepareAccumulator { final List spo2Red = []; final List spo2Ir = []; final List skinTemp = []; + final List skinContact = []; + + /// Defensive numeric read. The decoded-page rows come straight out of SQLite, + /// where a column's storage class is per-VALUE, not per-column — a row written + /// by an older/importing path can hand back a String or null where an INTEGER + /// is declared. A bare `row['hr'] as num?` throws on that, and a throw in this + /// worker used to hang the whole engine (see [derivationPrepareWorker]). + static num? _num(Object? v) => v is num ? v : null; void addRawPage(List hexes) { if (hexes.isEmpty) return; @@ -340,6 +383,7 @@ class _PrepareAccumulator { spo2Red.addAll(sub.spo2Red); spo2Ir.addAll(sub.spo2Ir); skinTemp.addAll(sub.skinTemp); + skinContact.addAll(sub.skinContact); } void addDecodedPage( @@ -349,29 +393,35 @@ class _PrepareAccumulator { if (frames.isEmpty) return; final rrByCounter = >>{}; for (final row in rrRows) { - final counter = (row['counter'] as num?)?.toInt(); + final counter = _num(row['counter'])?.toInt(); if (counter == null) continue; rrByCounter.putIfAbsent(counter, () => >[]).add(row); } for (final row in frames) { - final recTs = (row['rec_ts'] as num?)?.toInt(); + final recTs = _num(row['rec_ts'])?.toInt(); if (recTs == null || recTs <= 0) continue; tsSec.add(recTs); - hr.add((row['hr'] as num?)?.toInt() ?? 0); - ax.add((row['ax'] as num?)?.toDouble() ?? 0); - ay.add((row['ay'] as num?)?.toDouble() ?? 0); - az.add((row['az'] as num?)?.toDouble() ?? 0); - spo2Red.add((row['spo2_red_raw'] as num?)?.toInt() ?? 0); - spo2Ir.add((row['spo2_ir_raw'] as num?)?.toInt() ?? 0); - skinTemp.add((row['skin_temp_raw'] as num?)?.toInt() ?? 0); - final counter = (row['counter'] as num?)?.toInt(); + hr.add(_num(row['hr'])?.toInt() ?? 0); + ax.add(_num(row['ax'])?.toDouble() ?? 0); + ay.add(_num(row['ay'])?.toDouble() ?? 0); + az.add(_num(row['az'])?.toDouble() ?? 0); + spo2Red.add(_num(row['spo2_red_raw'])?.toInt() ?? 0); + spo2Ir.add(_num(row['spo2_ir_raw'])?.toInt() ?? 0); + skinTemp.add(_num(row['skin_temp_raw'])?.toInt() ?? 0); + // `decoded_onehz` has no skin-contact column, so the live decoded path + // genuinely has no contact signal to offer; a page that DOES carry one + // (the raw-decode fallback) is honoured. Keeping the array 1:1 with + // tsSec is what lets `Substrate.fromJson` tell "absent" (empty ⇒ + // zero-filled) from "present but zero". + skinContact.add(_num(row['skin_contact'])?.toInt() ?? 0); + final counter = _num(row['counter'])?.toInt(); if (counter == null) continue; final beats = rrByCounter[counter]; if (beats == null) continue; for (final beat in beats) { - final rr = (beat['rr_ms'] as num?)?.toDouble(); + final rr = _num(beat['rr_ms'])?.toDouble(); if (rr == null || rr <= 0) continue; - rrTsMs.add((beat['rr_ts_ms'] as num?)?.toDouble() ?? recTs * 1000.0); + rrTsMs.add(_num(beat['rr_ts_ms'])?.toDouble() ?? recTs * 1000.0); rrMs.add(rr); } } @@ -388,6 +438,6 @@ class _PrepareAccumulator { spo2Red: spo2Red, spo2Ir: spo2Ir, skinTemp: skinTemp, - skinContact: const [], + skinContact: skinContact, ); } diff --git a/lib/compute/onehz_pipeline.dart b/lib/compute/onehz_pipeline.dart index 5ce9240d..5c1afb51 100644 --- a/lib/compute/onehz_pipeline.dart +++ b/lib/compute/onehz_pipeline.dart @@ -294,8 +294,19 @@ Map deriveDayBundle(Map inputJson) { tier: Tier.high, inputs_used: ['rr_cleaned'], ); - // Nocturnal RHR over the SLEEP HR (fallback to day-valid only if no sleep HR). - final rhr = nocturnalRhr(sleepHr.isNotEmpty ? sleepHr : dayHrValid); + // Nocturnal RHR over the SLEEP HR (fallback to the DAY series only if there + // is no sleep HR at all). + // + // Both arguments must be POSITIONALLY DENSE 1 Hz series where 0 means + // off-skin — `nocturnalRhr` slides its 30-minute window over wall-clock + // POSITIONS and enforces a minimum on-skin coverage per window. Passing the + // compacted `dayHrValid` here defeated that: with gaps squeezed out, 1800 + // consecutive entries could span many hours, so "lowest 30-minute mean" + // silently became "lowest mean over whatever 1800 samples happened to + // survive". `dayHr` keeps its zeros, so a day too sparse to contain a real + // contiguous window now abstains instead of reporting a stitched-together + // trough. + final rhr = nocturnalRhr(sleepHr.isNotEmpty ? sleepHr : dayHr); // HR dip: day-side = waking HR outside the sleep window; night-side = sleep HR. final dayOnly = _dayHrOutsideSleep(d); final dip = hrDip(dayOnly, sleepHr); @@ -384,7 +395,7 @@ Map deriveDayBundle(Map inputJson) { ? math.log(sleepSessionRmssd) : null; // Readiness's RHR input must come from an ACTUAL detected sleep session. - // `rhr` above intentionally falls back to daytime HR (`dayHrValid`) for the + // `rhr` above intentionally falls back to daytime HR (`dayHr`) for the // general-purpose "resting HR" display card, but feeding that fallback into // readiness let a handful of minutes of live daytime HR masquerade as an // overnight resting rate — the sole reason a same-day score of 100 could @@ -444,7 +455,16 @@ Map deriveDayBundle(Map inputJson) { 'note': composite.note, }; } - // Plews lnRMSSD readiness over the trailing history INCLUDING today. + // Plews lnRMSSD readiness. `readinessLnRmssd` is contractually handed the + // trailing history with TONIGHT AS THE LAST ELEMENT and takes strictly the + // prior elements as its baseline (analytics v38). So appending `lnToday` + // exactly once is right — PROVIDED `d.lnRmssdHistory` holds only days BEFORE + // this one. It didn't: the engine filled it from an unfiltered trailing + // `metric_series` window that already contained the row a previous derive of + // THIS day wrote, so today was in its own baseline AND counted a second time + // by this append. The engine now self-excludes the target date + // (`_attachHistory` → `_BaselineHistoryCache.valuesBefore`), which is what + // makes this single append the correct, non-duplicating one. final lnHist = [...d.lnRmssdHistory, ?lnToday]; final lnReadiness = lnHist.length >= 4 ? readinessLnRmssd(lnHist) diff --git a/lib/compute/substrate.dart b/lib/compute/substrate.dart index c497bd80..40df5050 100644 --- a/lib/compute/substrate.dart +++ b/lib/compute/substrate.dart @@ -388,6 +388,17 @@ class PhysioDay { bool get hasSleep => sleep.present; } +/// How far BEFORE a calendar day's local midnight [calendarDays] searches for +/// the main sleep that ends in that day — the previous local NOON. +/// +/// Exported because the coordinator has to LOAD at least this much substrate +/// before the day start, or `searchStart = math.max(dataStart, …)` silently +/// clips the window back to wherever the loaded slice happens to begin and any +/// sleep onset before that instant is truncated (see +/// `DerivationEngine._targetDayWindow`). One constant, two call sites — they +/// cannot drift apart again. +const int kNocturnalSearchLookbackSec = 12 * 3600; + /// A user-asserted sleep window for one day — manual entry (Approach 1) or a /// confirmation of the HR-led fallback (Approach 2). Passed into [calendarDays] /// so it overrides auto detection for the matching [dayId]. @@ -426,7 +437,18 @@ String localDateLabel(int epochSec) { /// A sleep that crosses midnight is attributed to the day it ENDS; its window /// indices (sleepLoIdx/Hi) point into the full substrate, so the coordinator /// still slices the whole window for HRV/RHR/recovery regardless of the boundary. -List calendarDays(Substrate sub, {SleepWindowOverride? override}) { +/// +/// [tzOffsetAt] resolves the local UTC offset in effect at an epoch second; +/// it defaults to [tzOffsetSecondsAt] and exists so tests can pin that the +/// habitual-midsleep prior is resolved AT THE DAY BEING SEGMENTED rather than +/// at "now" (a zone-independent way to test the DST/travel fix, since the +/// machine running the test may sit in a zone that never changes offset). +List calendarDays( + Substrate sub, { + SleepWindowOverride? override, + int Function(int epochSec)? tzOffsetAt, +}) { + final tzOffset = tzOffsetAt ?? tzOffsetSecondsAt; if (sub.isEmpty) return const []; final accel = sub.accelSamples(); final hr = sub.hr1hz(); @@ -452,7 +474,7 @@ List calendarDays(Substrate sub, {SleepWindowOverride? override}) { // prev-18:00 → noon window missed late wakes and forced the detector to act // like there was only one candidate sleep. The richer selector needs the // full set of sessions that can legitimately end today. - final searchStart = math.max(dataStart, dayStart - 12 * 3600); + final searchStart = math.max(dataStart, dayStart - kNocturnalSearchLookbackSec); final searchEnd = math.min(dataEnd, dayEnd); final loS = _lowerBound(sub.tsSec, searchStart); final hiS = _lowerBound(sub.tsSec, searchEnd); @@ -465,9 +487,17 @@ List calendarDays(Substrate sub, {SleepWindowOverride? override}) { final ov = (override != null && override.dayId == dayLabel) ? override : null; if (hiS - loS >= 600 || ov != null) { + // The habitual-midsleep prior converts each HISTORICAL sleep block's epoch + // seconds to a local time-of-day. Using `DateTime.now()`'s offset applied + // the CURRENT UTC offset to those historical instants, so re-deriving days + // from the other side of a DST transition (or a trip) shifted every + // historical midsleep by an hour — which can change which candidate sleep + // the selector's alignment bonus picks. Resolve the offset AT THE DAY + // BEING SEGMENTED instead of "whenever this code happens to run", so a + // re-derive of an old day is reproducible regardless of today's zone. final habitualMidsleepSec = ana.habitualMidsleepSecFromHistory( sleepHistory, - tzOffsetSeconds: DateTime.now().timeZoneOffset.inSeconds, + tzOffsetSeconds: tzOffset(dayStart), ); // Daytime HR baseline = valid HR before the nocturnal search window. final base = [for (var i = 0; i < loS; i++) if (hr[i] > 0) hr[i]]; @@ -581,6 +611,19 @@ List calendarDays(Substrate sub, {SleepWindowOverride? override}) { return days; } +/// The LOCAL UTC offset in effect AT [epochSec] — not the offset in effect now. +/// +/// `DateTime.now().timeZoneOffset` answers "what is the offset today", which is +/// the wrong question for any historical instant: a day derived from the other +/// side of a DST transition (or from before a trip) sits at a different offset, +/// and using today's would shift that day's local clock-times by up to an hour. +/// `DateTime.fromMillisecondsSinceEpoch(..., isUtc: false).timeZoneOffset` asks +/// the platform zone database for the offset that actually applied then. +int tzOffsetSecondsAt(int epochSec) => + DateTime.fromMillisecondsSinceEpoch(epochSec * 1000, isUtc: false) + .timeZoneOffset + .inSeconds; + /// Local midnight (epoch sec) at/before [epochSec]. int _localMidnight(int epochSec) { final d = DateTime.fromMillisecondsSinceEpoch(epochSec * 1000, isUtc: false); diff --git a/lib/data/day_label.dart b/lib/data/day_label.dart index 115156cc..5695c2b5 100644 --- a/lib/data/day_label.dart +++ b/lib/data/day_label.dart @@ -22,3 +22,49 @@ String dayLabelOf(DateTime dt) { /// Today's LOCAL day label — the key `LocalDb`/the derivation engine file days /// under. [now] is injectable for tests (defaults to the real clock). String todayLabel([DateTime? now]) => dayLabelOf(now ?? DateTime.now()); + +/// Parse a 'YYYY-MM-DD' label into its (year, month, day) parts, or null if it +/// isn't one. Deliberately strict — a malformed label must not silently become +/// epoch 0 and delete/export the wrong window. +List? _partsOf(String dayId) { + final p = dayId.split('-'); + if (p.length != 3) return null; + final y = int.tryParse(p[0]); + final m = int.tryParse(p[1]); + final d = int.tryParse(p[2]); + if (y == null || m == null || d == null) return null; + return [y, m, d]; +} + +/// Epoch SECONDS of the LOCAL midnight that STARTS day [dayId] ('YYYY-MM-DD'). +/// Returns null for a malformed label. +int? localDayStartSec(String dayId) { + final p = _partsOf(dayId); + if (p == null) return null; + return DateTime(p[0], p[1], p[2]).millisecondsSinceEpoch ~/ 1000; +} + +/// Epoch SECONDS of the LOCAL midnight that ENDS day [dayId] — i.e. the start +/// of the NEXT local calendar day. Returns null for a malformed label. +/// +/// This is deliberately NOT `localDayStartSec(dayId) + 86400`. A local calendar +/// day is only 86 400 s when there is no DST transition inside it: a +/// spring-forward day is 23 h and a fall-back day is 25 h. Using `+86400` made +/// day-window deletes/exports overrun into the NEXT day's first hour on +/// spring-forward, and leave the last hour behind on fall-back. `DateTime`'s +/// local constructor normalizes day/month/year rollover AND the wall-clock +/// offset, so "midnight of the next calendar date" is the only correct end. +int? localDayEndSec(String dayId) { + final p = _partsOf(dayId); + if (p == null) return null; + return DateTime(p[0], p[1], p[2] + 1).millisecondsSinceEpoch ~/ 1000; +} + +/// True local length of day [dayId] in seconds (86400 / 82800 / 90000 …), or +/// null for a malformed label. +int? localDayLengthSec(String dayId) { + final lo = localDayStartSec(dayId); + final hi = localDayEndSec(dayId); + if (lo == null || hi == null) return null; + return hi - lo; +} diff --git a/lib/data/db.dart b/lib/data/db.dart index 0c4d71b5..0e657739 100644 --- a/lib/data/db.dart +++ b/lib/data/db.dart @@ -85,7 +85,28 @@ class LocalDb { return null; } - static const int _daySec = 86400; + /// The live schema version — the ONE place it is declared. Every + /// `openDatabase` this class performs (the app DB and the day-export DB) must + /// pass it: sqflite throws `ArgumentError('onCreate must be null if no + /// version is specified')` BEFORE opening anything when `onCreate` is given + /// without `version` (sqflite_common database_mixin.dart). + static const int schemaVersion = 26; + + /// SQLite caps host parameters per statement (`SQLITE_MAX_VARIABLE_NUMBER` — + /// only 999 on the builds shipped with older Android/iOS). Any `IN (?, ?, …)` + /// built from row data MUST be chunked below this; a single day of + /// `decoded_onehz` is 86 400 counters. Same reason `commitSyncBatch` chunks. + static const int _maxSqlVars = 500; + + /// Split [items] into `_maxSqlVars`-sized chunks for `IN (…)` binding. + static Iterable> _sqlVarChunks(List items) sync* { + for (var i = 0; i < items.length; i += _maxSqlVars) { + yield items.sublist( + i, + i + _maxSqlVars > items.length ? items.length : i + _maxSqlVars, + ); + } + } static Future _open() async { final dir = await getDatabasesPath(); @@ -178,6 +199,12 @@ class LocalDb { // rec_ts (not captured_at), so a multi-day flash backfill received in one // sync no longer collapses into a single "today" bucket. Additive + safe // on a populated DB. + // GUARDED add: an oldV <= 2 DB already had raw_records DROPped and + // re-created by the step-3 block above using the CURRENT `_createRaw` + // DDL — which already carries rec_ts. A bare ALTER … ADD COLUMN then + // threw "duplicate column name: rec_ts", and because onUpgrade runs + // inside ONE exclusive transaction the whole ladder rolled back and + // openDatabase rethrew → app stuck on the loading screen, forever. await _addRecTsColumn(db); await _backfillRecTs(db); await db.execute( @@ -244,7 +271,14 @@ class LocalDb { await _backfillDecodedStore(db); // Live workout steps (Tier-A pedometer over the session's 100 Hz // R10 accel). Additive nullable column — old rows read null. - await db.execute('ALTER TABLE sessions ADD COLUMN steps INTEGER'); + // + // GUARDED add: an oldV <= 6 DB gets `sessions` from the step-7 + // `_createUserTables` block above, which uses the CURRENT DDL — and + // that already declares `steps`. A bare ALTER … ADD COLUMN then threw + // "duplicate column name: steps", rolling back the whole (single, + // exclusive) onUpgrade transaction so openDatabase rethrew → app + // permanently stuck on the loading screen. + await _addColumnIfMissing(db, 'sessions', 'steps', 'INTEGER'); } if (oldV < 12) { // PURGE the old 1 Hz step ESTIMATE. 1 Hz can't count steps (Nyquist), @@ -416,34 +450,56 @@ class LocalDb { await _dropRawStore(db); } - static Future _ensureDayResultSkippedColumn(Database db) async { - final info = await db.rawQuery('PRAGMA table_info(day_result)'); - final has = info.any((c) => c['name'] == 'skipped'); - if (!has) { - try { - await db.execute( - 'ALTER TABLE day_result ADD COLUMN skipped INTEGER NOT NULL DEFAULT 0', - ); - } catch (_) { - /* another opener won the race — column now exists */ - } - } + /// The column names [table] currently has (empty if the table is absent). + static Future> _columnsOf(Database db, String table) async { + final info = await db.rawQuery('PRAGMA table_info($table)'); + return { + for (final c in info) + if (c['name'] is String) c['name'] as String, + }; } - static Future _ensureDayResultPartialColumn(Database db) async { - final info = await db.rawQuery('PRAGMA table_info(day_result)'); - final has = info.any((c) => c['name'] == 'partial'); - if (!has) { - try { - await db.execute( - 'ALTER TABLE day_result ADD COLUMN partial INTEGER NOT NULL DEFAULT 0', - ); - } catch (_) { - /* another opener won the race — column now exists */ - } + /// THE ONLY sanctioned way to add a column in a migration step. + /// + /// A bare `ALTER TABLE … ADD COLUMN` in the ladder is a latent brick: the + /// `_create*` helpers are MODERNIZED IN PLACE (they always emit the current + /// DDL), so any DB old enough to have a table created by a LATER-numbered + /// step already has the column an EARLIER-numbered ALTER tries to add. That + /// throws "duplicate column name", and since `onUpgrade` runs inside ONE + /// exclusive transaction the entire ladder rolls back and `openDatabase` + /// rethrows — the app is stuck on the loading screen on EVERY launch, with no + /// way out. Check first; swallow a lost race (SQLite does statement-level + /// rollback, so a caught failure never poisons the surrounding transaction). + static Future _addColumnIfMissing( + Database db, + String table, + String column, + String ddlType, + ) async { + if ((await _columnsOf(db, table)).contains(column)) return; + try { + await db.execute('ALTER TABLE $table ADD COLUMN $column $ddlType'); + } catch (_) { + /* another opener won the race — the column exists now */ } } + static Future _ensureDayResultSkippedColumn(Database db) => + _addColumnIfMissing( + db, + 'day_result', + 'skipped', + 'INTEGER NOT NULL DEFAULT 0', + ); + + static Future _ensureDayResultPartialColumn(Database db) => + _addColumnIfMissing( + db, + 'day_result', + 'partial', + 'INTEGER NOT NULL DEFAULT 0', + ); + // ── MENSTRUAL SYMPTOM LOG ────────────────────────────────────────────────── static Future _createCycleSymptom(Database db) async { await db.execute(''' @@ -491,7 +547,7 @@ class LocalDb { // The "safe-trim invariant" is: persist decoded+raw → persist this cursor → // ACK with-response. The band only trims its flash once the ACK is link-layer // confirmed, so a crash anywhere before the ACK re-delivers the batch. - static Future _createSyncCursor(Database db) async { + static Future _createSyncCursor(DatabaseExecutor db) async { await db.execute(''' CREATE TABLE IF NOT EXISTS sync_cursor ( name TEXT PRIMARY KEY, @@ -1066,7 +1122,7 @@ class LocalDb { '''); } - static Future _createSyncState(Database db) async { + static Future _createSyncState(DatabaseExecutor db) async { await db.execute(''' CREATE TABLE IF NOT EXISTS sync_ledger ( chunk_id TEXT PRIMARY KEY, @@ -1156,17 +1212,8 @@ class LocalDb { } static Future _ensureSessionSchema(Database db) async { - final cols = await db.rawQuery("PRAGMA table_info(sessions)"); - final names = { - for (final c in cols) - if (c['name'] is String) c['name'] as String, - }; - if (!names.contains('steps')) { - await db.execute('ALTER TABLE sessions ADD COLUMN steps INTEGER'); - } - if (!names.contains('hrr_bpm')) { - await db.execute('ALTER TABLE sessions ADD COLUMN hrr_bpm REAL'); - } + await _addColumnIfMissing(db, 'sessions', 'steps', 'INTEGER'); + await _addColumnIfMissing(db, 'sessions', 'hrr_bpm', 'REAL'); } // ── WORKOUT SUGGESTIONS (opt-in auto-detect) ─────────────────────────────── @@ -1346,125 +1393,171 @@ class LocalDb { '''); } - static Future _ensureSyncCursorSchema(Database db) async { - final tables = await db.rawQuery( - "SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'sync_cursor'", - ); - if (tables.isEmpty) { - await _createSyncCursor(db); - return; - } - final cols = await db.rawQuery("PRAGMA table_info(sync_cursor)"); - final names = { - for (final c in cols) - if (c['name'] is String) c['name'] as String, - }; - if (names.contains('name') && - names.contains('value') && - names.contains('updated_at')) { - return; - } - - await db.execute('ALTER TABLE sync_cursor RENAME TO sync_cursor_legacy'); - await _createSyncCursor(db); - final legacyRows = await db.query('sync_cursor_legacy'); - final now = DateTime.now().millisecondsSinceEpoch; - for (final row in legacyRows) { - final name = row['name'] as String?; - if (name == null || name.isEmpty) continue; - await db.insert('sync_cursor', { - 'name': name, - 'value': row['value']?.toString(), - 'updated_at': (row['updated_at'] as num?)?.toInt() ?? now, - }, conflictAlgorithm: ConflictAlgorithm.replace); - } - await db.execute('DROP TABLE sync_cursor_legacy'); - } - - static Future _ensureSyncLedgerSchema(Database db) async { - final tables = await db.rawQuery( - "SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'sync_ledger'", - ); - if (tables.isEmpty) { - await _createSyncState(db); - return; - } - final cols = await db.rawQuery("PRAGMA table_info(sync_ledger)"); - final names = { - for (final c in cols) - if (c['name'] is String) c['name'] as String, - }; - if (names.contains('chunk_id')) return; + /// Run a rename → recreate → copy legacy-shape migration ATOMICALLY and + /// IDEMPOTENTLY. + /// + /// The three callers below used to do `ALTER … RENAME`, then `CREATE`, then a + /// row-by-row copy, then `DROP` — all OUTSIDE any transaction. That is fine + /// under `onUpgrade` (sqflite wraps the whole ladder in one exclusive txn) but + /// these also run from `_repairOpenSchema` in `onOpen`, which is NOT wrapped. + /// A crash / OS kill between the rename and the end of the copy left + /// `` present AND correctly shaped, so the next open hit the + /// "already current" early-return and `
_legacy` sat there orphaned with + /// its rows never migrated — losing `strap_trim` / `counter_hw` / `rec_ts_hw`, + /// i.e. the whole resumable-sync cursor and the safe-trim high-water. + /// + /// Now: one transaction — which sqflite JOINS to the already-open `onUpgrade` + /// transaction when called from the ladder, and opens for real from `onOpen` + /// — plus an explicit RESUME of an orphan left behind by any older build. + /// + /// [isCurrent] decides whether an existing `
` is already the new shape. + /// [copy] must be idempotent (INSERT OR REPLACE on a natural key); set + /// [copyOnlyIntoEmpty] for a destination with no natural key (autoincrement + /// id), where re-running a resume would otherwise duplicate rows. + static Future _migrateLegacyTable( + Database db, { + required String table, + required bool Function(Set columns) isCurrent, + required Future Function(DatabaseExecutor ex) create, + required Future Function( + DatabaseExecutor ex, + List> legacyRows, + int nowMs, + ) + copy, + bool copyOnlyIntoEmpty = false, + }) async { + final legacy = '${table}_legacy'; + await db.transaction((txn) async { + Future exists(String t) async => + (await txn.rawQuery( + "SELECT name FROM sqlite_master WHERE type = 'table' AND name = ?", + [t], + )).isNotEmpty; + Future> columnsOf(String t) async { + final info = await txn.rawQuery('PRAGMA table_info($t)'); + return { + for (final c in info) + if (c['name'] is String) c['name'] as String, + }; + } - await db.execute('ALTER TABLE sync_ledger RENAME TO sync_ledger_legacy'); - await _createSyncState(db); - final legacyRows = await db.query('sync_ledger_legacy'); - final now = DateTime.now().millisecondsSinceEpoch; - for (final row in legacyRows) { - final meta = { - 'last_batch_token': row['last_batch_token'], - 'last_batch_id': row['last_batch_id'], - 'last_batch_records': row['last_batch_records'], - 'last_history_complete_at': row['last_history_complete_at'], - 'last_trim_cutoff_ms': row['last_trim_cutoff_ms'], - 'last_trimmed_at': row['last_trimmed_at'], - if (row['note'] != null) 'legacy_note': row['note'], - }; - await db.insert('sync_ledger', { - 'chunk_id': (row['id'] as String?) ?? 'capture', - 'kind': 'historical', - 'status': row['last_history_complete_at'] != null - ? 'complete' - : row['last_batch_acked_at'] != null - ? 'acknowledged' - : 'legacy', - 'created_at': (row['updated_at'] as num?)?.toInt() ?? now, - 'updated_at': (row['updated_at'] as num?)?.toInt() ?? now, - 'acked_at': (row['last_batch_acked_at'] as num?)?.toInt(), - 'last_error': null, - 'meta_json': jsonEncode(meta), - }, conflictAlgorithm: ConflictAlgorithm.replace); - } - await db.execute('DROP TABLE sync_ledger_legacy'); + if (!await exists(legacy)) { + // Normal path. + if (!await exists(table)) { + await create(txn); + return; + } + if (isCurrent(await columnsOf(table))) return; + await txn.execute('ALTER TABLE $table RENAME TO $legacy'); + } + // From here on `
_legacy` holds the rows of record. `
` is + // either absent (crash between RENAME and CREATE) or the new shape + // (possibly half-copied, or fully copied by an older build that then + // died before the DROP) — create it if needed and re-copy; the copy is + // idempotent, so a repeat is a no-op rather than a duplication. + if (!await exists(table)) await create(txn); + final skipCopy = + copyOnlyIntoEmpty && + (Sqflite.firstIntValue( + await txn.rawQuery('SELECT COUNT(*) FROM $table'), + ) ?? + 0) > + 0; + if (!skipCopy) { + await copy( + txn, + await txn.query(legacy), + DateTime.now().millisecondsSinceEpoch, + ); + } + await txn.execute('DROP TABLE $legacy'); + }); } - static Future _ensureSyncQuarantineSchema(Database db) async { - final tables = await db.rawQuery( - "SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'sync_quarantine'", - ); - if (tables.isEmpty) { - await _createSyncState(db); - return; - } - final cols = await db.rawQuery("PRAGMA table_info(sync_quarantine)"); - final names = { - for (final c in cols) - if (c['name'] is String) c['name'] as String, - }; - if (names.contains('payload_json')) return; + static Future _ensureSyncCursorSchema(Database db) => + _migrateLegacyTable( + db, + table: 'sync_cursor', + isCurrent: (c) => + c.contains('name') && + c.contains('value') && + c.contains('updated_at'), + create: _createSyncCursor, + copy: (ex, legacyRows, now) async { + for (final row in legacyRows) { + final name = row['name'] as String?; + if (name == null || name.isEmpty) continue; + await ex.insert('sync_cursor', { + 'name': name, + 'value': row['value']?.toString(), + 'updated_at': (row['updated_at'] as num?)?.toInt() ?? now, + }, conflictAlgorithm: ConflictAlgorithm.replace); + } + }, + ); - await db.execute( - 'ALTER TABLE sync_quarantine RENAME TO sync_quarantine_legacy', - ); - await _createSyncState(db); - final legacyRows = await db.query('sync_quarantine_legacy'); - final now = DateTime.now().millisecondsSinceEpoch; - for (final row in legacyRows) { - await db.insert('sync_quarantine', { - 'kind': (row['source_role'] as String?) ?? 'legacy', - 'payload_json': jsonEncode({ - 'fingerprint': row['fingerprint'], - 'packet_type': row['packet_type'], - 'hex': row['hex'], - 'counter': row['counter'], - 'captured_at': row['captured_at'], - }), - 'reason': (row['reason'] as String?) ?? 'legacy_migrated', - 'created_at': (row['created_at'] as num?)?.toInt() ?? now, - }); - } - await db.execute('DROP TABLE sync_quarantine_legacy'); - } + static Future _ensureSyncLedgerSchema(Database db) => _migrateLegacyTable( + db, + table: 'sync_ledger', + isCurrent: (c) => c.contains('chunk_id'), + create: _createSyncState, + copy: (ex, legacyRows, now) async { + for (final row in legacyRows) { + final meta = { + 'last_batch_token': row['last_batch_token'], + 'last_batch_id': row['last_batch_id'], + 'last_batch_records': row['last_batch_records'], + 'last_history_complete_at': row['last_history_complete_at'], + 'last_trim_cutoff_ms': row['last_trim_cutoff_ms'], + 'last_trimmed_at': row['last_trimmed_at'], + if (row['note'] != null) 'legacy_note': row['note'], + }; + await ex.insert('sync_ledger', { + 'chunk_id': (row['id'] as String?) ?? 'capture', + 'kind': 'historical', + 'status': row['last_history_complete_at'] != null + ? 'complete' + : row['last_batch_acked_at'] != null + ? 'acknowledged' + : 'legacy', + 'created_at': (row['updated_at'] as num?)?.toInt() ?? now, + 'updated_at': (row['updated_at'] as num?)?.toInt() ?? now, + 'acked_at': (row['last_batch_acked_at'] as num?)?.toInt(), + 'last_error': null, + 'meta_json': jsonEncode(meta), + }, conflictAlgorithm: ConflictAlgorithm.replace); + } + }, + ); + + static Future _ensureSyncQuarantineSchema(Database db) => + _migrateLegacyTable( + db, + table: 'sync_quarantine', + isCurrent: (c) => c.contains('payload_json'), + create: _createSyncState, + // `id INTEGER PRIMARY KEY AUTOINCREMENT` — no natural key to REPLACE + // on, so a resume must not re-copy into a destination that already has + // rows or the quarantine log would double on every retry. + copyOnlyIntoEmpty: true, + copy: (ex, legacyRows, now) async { + for (final row in legacyRows) { + await ex.insert('sync_quarantine', { + 'kind': (row['source_role'] as String?) ?? 'legacy', + 'payload_json': jsonEncode({ + 'fingerprint': row['fingerprint'], + 'packet_type': row['packet_type'], + 'hex': row['hex'], + 'counter': row['counter'], + 'captured_at': row['captured_at'], + }), + 'reason': (row['reason'] as String?) ?? 'legacy_migrated', + 'created_at': (row['created_at'] as num?)?.toInt() ?? now, + }); + } + }, + ); // samples — LEGACY header-only record index (counter, ts, hr). Retained only // so pre-v11 databases stay readable if decoded_onehz backfill was partial. @@ -1648,11 +1741,15 @@ class LocalDb { /// Add the additive `rec_ts` column to an EXISTING raw_records table (upgrade /// path only). NOT NULL with a DEFAULT 0 so legacy rows are well-formed until /// the backfill rewrites them. - static Future _addRecTsColumn(Database db) async { - await db.execute( - 'ALTER TABLE raw_records ADD COLUMN rec_ts INTEGER NOT NULL DEFAULT 0', - ); - } + /// GUARDED (see [_addColumnIfMissing]): the step-3 rebuild already creates + /// raw_records from the CURRENT `_createRaw` DDL, which carries rec_ts — so on + /// an oldV <= 2 upgrade this column is already there. + static Future _addRecTsColumn(Database db) => _addColumnIfMissing( + db, + 'raw_records', + 'rec_ts', + 'INTEGER NOT NULL DEFAULT 0', + ); /// Backfill `rec_ts` for every existing raw row by decoding its hex once. Runs /// inside the migration on a populated DB. Falls back to captured_at/1000 when a @@ -1763,6 +1860,29 @@ class LocalDb { } } + /// THE orphan guard for an INSERT-OR-REPLACE into `decoded_onehz`. + /// + /// Queue this onto [batch] IMMEDIATELY BEFORE writing the row for [counter] @ + /// [recTs] — every write path into `decoded_onehz` must go through it, or it + /// strands `decoded_rr` beats (see [_queueDecodedOneHz] for the full + /// derivation of both eviction cases). Returns the number of ops queued. + static int _queueOrphanGuard( + Batch batch, { + required int counter, + required int recTs, + }) { + batch.rawDelete( + 'DELETE FROM decoded_rr WHERE ' + // (a) UNIQUE(rec_ts) eviction — the LOSER counter's beats. + 'counter IN ' + '(SELECT counter FROM decoded_onehz WHERE rec_ts = ? AND counter != ?) ' + // (b) counter-PK eviction — stale-timestamped beats under OUR counter. + 'OR (counter = ? AND rr_ts_ms != ?)', + [recTs, counter, counter, recTs * 1000], + ); + return 1; + } + /// Queues the decoded_onehz + decoded_rr (+ orphan-guard delete) writes for /// one raw onto [batch]. Returns the number of batch operations added, so a /// caller committing a large offload can chunk the batch to bound the native @@ -1788,11 +1908,20 @@ class LocalDb { // REPLACE on UNIQUE(rr_ts_ms, beat_index) only overwrites overlapping beat // indexes, so delete the evicted counter's beats explicitly, in the same // batch/transaction (mirrors the v17 rebuild's decoded_onehz join). - batch.rawDelete( - 'DELETE FROM decoded_rr WHERE counter IN ' - '(SELECT counter FROM decoded_onehz WHERE rec_ts = ? AND counter != ?)', - [recTs, raw.counter], - ); + // + // …AND the COUNTER-PK eviction, which the guard used to miss entirely. + // `decoded_onehz` is `counter INTEGER PRIMARY KEY` as well as + // UNIQUE(rec_ts), and (per the comment above) the strap's counter RESETS to + // ~0 on every reboot — so this same REPLACE also silently DELETES the row + // of an OLDER SECOND that happened to reuse this counter. That older + // second's beats live under OUR counter carrying ITS rr_ts_ms, and only the + // overlapping beat_indexes get overwritten below: any beat at an index past + // the new record's beat count SURVIVES, still stamped days earlier. Neither + // prune path can ever see it (the counter-join finds a fresh rec_ts; the + // orphan sweep finds the counter present), so a later page's RR series was + // polluted with beats from another day — silently wrecking RMSSD/HRV. + // Drop every beat under this counter that is not stamped with THIS second. + var ops = _queueOrphanGuard(batch, counter: raw.counter, recTs: recTs); batch.insert('decoded_onehz', { 'counter': raw.counter, 'rec_ts': recTs, @@ -1804,7 +1933,7 @@ class LocalDb { 'spo2_ir_raw': decoded.spo2IrRaw ?? 0, 'skin_temp_raw': decoded.skinTempRaw ?? 0, }, conflictAlgorithm: ConflictAlgorithm.replace); - var ops = 2; // rawDelete + decoded_onehz insert + ops++; // the decoded_onehz insert for (var i = 0; i < decoded.rrIntervalsMs.length; i++) { final rr = decoded.rrIntervalsMs[i]; if (rr <= 0) continue; @@ -1861,6 +1990,12 @@ class LocalDb { captured_at INTEGER NOT NULL ) '''); + // The PK is the frame hex, so a `ts` window (the timeline's day query, and + // the retention prune) was a full table scan. Cheap to build — `events` is + // pruned to the retention window. + await db.execute( + 'CREATE INDEX IF NOT EXISTS idx_events_ts ON events(ts)', + ); } // band_events / band_battery — structured local history for device-state @@ -1899,19 +2034,8 @@ class LocalDb { /// Additive: add the `millivolts` column to an existing band_battery table. /// Guarded — the column already exists on fresh installs (see _createBandSignals) /// and ALTER … ADD COLUMN throws if it's already there. - static Future _ensureBandBatteryMillivolts(Database db) async { - final info = await db.rawQuery('PRAGMA table_info(band_battery)'); - final has = info.any((c) => c['name'] == 'millivolts'); - if (!has) { - try { - await db.execute( - 'ALTER TABLE band_battery ADD COLUMN millivolts INTEGER', - ); - } catch (_) { - /* another opener won the race — column now exists */ - } - } - } + static Future _ensureBandBatteryMillivolts(Database db) => + _addColumnIfMissing(db, 'band_battery', 'millivolts', 'INTEGER'); /// Durable archive for historical records we received but could not decode /// (unknown/unsupported version). NEVER pruned — the whole point is that a @@ -2119,6 +2243,11 @@ class LocalDb { }; } + /// The OLDEST [limit] queued events — an upload-queue drain head, and ONLY + /// that. Never use it to answer "what happened on day X": once `events` holds + /// more than [limit] rows the page can't reach recent days at all (the same + /// oldest-N-vs-trailing-N shape as the `metricSeries(limit:)` outage). Use + /// [eventsInRange] for a day/window query. static Future>> unuploadedEvents({ int limit = 500, }) async { @@ -2126,6 +2255,25 @@ class LocalDb { return db.query('events', orderBy: 'ts ASC', limit: limit); } + /// Events whose `ts` (epoch SECONDS) is in the half-open window + /// `[fromTs, toTs)`, oldest first. Bounded BY THE WINDOW, not by an unrelated + /// global page, so a day's markers can never be crowded out by older rows. + /// [limit] is a defensive cap on a pathological window only. + static Future>> eventsInRange( + int fromTs, + int toTs, { + int limit = 5000, + }) async { + final db = await instance; + return db.query( + 'events', + where: 'ts >= ? AND ts < ?', + whereArgs: [fromTs, toTs], + orderBy: 'ts ASC', + limit: limit, + ); + } + static Future deleteEvents(List hexes) async { if (hexes.isEmpty) return; final db = await instance; @@ -2410,18 +2558,56 @@ class LocalDb { ); } - /// Sparse RR beats for a contiguous decoded 1 Hz page, keyed by the owning - /// frame counter and ordered by that frame. + /// Sparse RR beats for one contiguous decoded 1 Hz page. + /// + /// [fromCounter] / [toCounter] are the page's FIRST and LAST row counters, as + /// returned by [decodedOneHzBatchByRecTsRange] (which orders `rec_ts ASC, + /// counter ASC`). They are page ENDPOINTS, **not** a monotonic counter span: + /// the strap's counter resets to ~0 on every reboot, so a page straddling a + /// reboot has first = a pre-reboot high and last = a post-reboot low. The old + /// `WHERE counter >= ? AND counter <= ?` then read `>= 1200000 AND <= 5` and + /// returned ZERO rows — the entire page's RR beats vanished with no error, so + /// that window silently produced no RMSSD/HRV at all. + /// + /// Selection is therefore by the page's real TIME window, resolved from those + /// two endpoint counters. `decoded_onehz` is UNIQUE(rec_ts), so + /// `[first.rec_ts, last.rec_ts]` contains exactly the page's rows — no + /// over-fetch — and the join to `decoded_onehz` additionally keeps orphaned + /// beats (whose owning row was evicted) out of the read path. + /// + /// When the endpoints are NOT real rows the caller is asking for a plain + /// counter span (e.g. `0 .. 1<<30` = "everything"); that falls back to a + /// NORMALIZED counter range so an inverted pair still can't return nothing. static Future>> decodedRrByCounterRange({ required int fromCounter, required int toCounter, }) async { final db = await instance; + final bounds = (await db.rawQuery( + 'SELECT COUNT(*) AS n, MIN(rec_ts) AS lo, MAX(rec_ts) AS hi ' + 'FROM decoded_onehz WHERE counter IN (?, ?)', + [fromCounter, toCounter], + )).first; + final n = (bounds['n'] as num?)?.toInt() ?? 0; + final want = fromCounter == toCounter ? 1 : 2; + if (n == want) { + return db.rawQuery( + 'SELECT rr.counter AS counter, rr.beat_index AS beat_index, ' + ' rr.rr_ts_ms AS rr_ts_ms, rr.rr_ms AS rr_ms ' + 'FROM decoded_rr rr ' + 'JOIN decoded_onehz d ON d.counter = rr.counter ' + 'WHERE d.rec_ts >= ? AND d.rec_ts <= ? ' + 'ORDER BY d.rec_ts ASC, rr.beat_index ASC', + [bounds['lo'], bounds['hi']], + ); + } + final lo = fromCounter <= toCounter ? fromCounter : toCounter; + final hi = fromCounter <= toCounter ? toCounter : fromCounter; return db.query( 'decoded_rr', columns: ['counter', 'beat_index', 'rr_ts_ms', 'rr_ms'], where: 'counter >= ? AND counter <= ?', - whereArgs: [fromCounter, toCounter], + whereArgs: [lo, hi], orderBy: 'counter ASC, beat_index ASC', ); } @@ -2512,6 +2698,46 @@ class LocalDb { return [for (final r in rows) _withDate(r)]; } + /// Every day_id that has a `day_result` row at its LATEST algo_version, newest + /// first — WITHOUT touching `payload_json`. + /// + /// `recentDayResults()` does `SELECT r.*`, which drags the whole bundle + /// (hr_curve / hypnogram / HRV series, tens of KB a day) across the isolate + /// boundary. Screens that only need "which days exist" must use this instead: + /// over a multi-year history the payload variant is hundreds of MB. + static Future> dayResultDayIdsDesc() async { + final db = await instance; + final rows = await db.rawQuery( + 'SELECT day_id FROM day_result GROUP BY day_id ORDER BY day_id DESC', + ); + return [ + for (final r in rows) + if (r['day_id'] is String) r['day_id'] as String, + ]; + } + + /// Day labels whose LATEST-version bundle records a real sleep total + /// (`sleep.accounting.value.tst_sec` present). Extracted IN SQLite via + /// json_extract, so only the scalar crosses the boundary — never the payload. + static Future> daysWithSleepTst() async { + final db = await instance; + final rows = await db.rawQuery( + 'SELECT r.day_id FROM day_result r ' + 'JOIN (SELECT day_id, MAX(algo_version) AS v FROM day_result GROUP BY day_id) m ' + ' ON r.day_id = m.day_id AND r.algo_version = m.v ' + // json_valid() first: json_extract() ERRORS on a malformed payload, and a + // corrupt bundle must degrade to "no sleep that day", never take out the + // whole Records screen. + 'WHERE json_valid(r.payload_json) ' + "AND json_extract(r.payload_json, '\$.sleep.accounting.value.tst_sec') " + 'IS NOT NULL', + ); + return { + for (final r in rows) + if (r['day_id'] is String) r['day_id'] as String, + }; + } + /// Every day label ('YYYY-MM-DD') the lookback screen can actually RENDER — /// exactly the days [dayResult]/`_bundleForDate` would return a real bundle /// for, newest first. That is: the LATEST-`algo_version` `day_result` row per @@ -2677,8 +2903,23 @@ class LocalDb { return out; } - static int _localDayStartSec(String dayId) => - DateTime.parse(dayId).millisecondsSinceEpoch ~/ 1000; + /// The half-open LOCAL window `[startSec, endSec)` covering day [dayId]. + /// + /// `endSec` is the NEXT local midnight, NOT `startSec + 86400`: a local + /// calendar day is 23 h on spring-forward and 25 h on fall-back. With the + /// flat +86400 this returned a window that overran into the next day's first + /// hour (deleteDays silently deleted the following day's first hour of + /// decoded_onehz / sessions / band_* / events) or fell an hour short + /// (fall-back left the last hour behind, and the export dropped it). Shared + /// with the UI/coach via day_label.dart so every layer agrees. + static (int, int) _localDayWindow(String dayId) { + final lo = localDayStartSec(dayId); + final hi = localDayEndSec(dayId); + if (lo == null || hi == null) { + throw ArgumentError.value(dayId, 'dayId', 'not a YYYY-MM-DD day label'); + } + return (lo, hi); + } static Future exportDaysDb(Set dayIds) async { final sorted = dayIds.toList()..sort(); @@ -2692,6 +2933,11 @@ class LocalDb { await deleteDatabase(dest); final out = await openDatabase( dest, + // `version:` is MANDATORY here. Without it sqflite throws + // ArgumentError('onCreate must be null if no version is specified') + // before opening anything — so this whole export path (Profile → Data + // history → Export) had never once produced a file. + version: schemaVersion, onCreate: (db, _) async { await _createSamples(db); await _createDecodedStore(db); @@ -2751,25 +2997,28 @@ class LocalDb { for (final row in decoded) if (row['counter'] != null) row['counter'], ]; - if (counters.isNotEmpty) { - final placeholders = List.filled(counters.length, '?').join(','); + // CHUNKED `IN (…)`: a full day is 86 400 counters, two orders of + // magnitude past SQLITE_MAX_VARIABLE_NUMBER — one giant statement can + // never bind. (This never surfaced only because the missing `version:` + // above aborted the export earlier.) + for (final chunk in _sqlVarChunks(counters)) { + final placeholders = List.filled(chunk.length, '?').join(','); final rr = await src.rawQuery( 'SELECT * FROM decoded_rr WHERE counter IN ($placeholders)', - counters, + chunk, ); - if (rr.isNotEmpty) { - await out.transaction((txn) async { - final batch = txn.batch(); - for (final row in rr) { - batch.insert( - 'decoded_rr', - Map.from(row), - conflictAlgorithm: ConflictAlgorithm.replace, - ); - } - await batch.commit(noResult: true); - }); - } + if (rr.isEmpty) continue; + await out.transaction((txn) async { + final batch = txn.batch(); + for (final row in rr) { + batch.insert( + 'decoded_rr', + Map.from(row), + conflictAlgorithm: ConflictAlgorithm.replace, + ); + } + await batch.commit(noResult: true); + }); } } await copyRows( @@ -2805,8 +3054,7 @@ class LocalDb { } for (final dayId in sorted) { - final startSec = _localDayStartSec(dayId); - final endSec = startSec + _daySec; + final (startSec, endSec) = _localDayWindow(dayId); await copyRawRange(startSec, endSec); await copyRows('day_result', where: 'day_id = ?', whereArgs: [dayId]); await copyRows('metric_series', where: 'date = ?', whereArgs: [dayId]); @@ -2839,17 +3087,20 @@ class LocalDb { String column, List values, ) async { - final placeholders = List.filled(values.length, '?').join(','); - deleted += await txn.rawDelete( - 'DELETE FROM $table WHERE $column IN ($placeholders)', - values, - ); + // Chunked: "select all" on a multi-year history binds one day label per + // parameter, which would blow SQLITE_MAX_VARIABLE_NUMBER. + for (final chunk in _sqlVarChunks(values)) { + final placeholders = List.filled(chunk.length, '?').join(','); + deleted += await txn.rawDelete( + 'DELETE FROM $table WHERE $column IN ($placeholders)', + chunk, + ); + } } await db.transaction((txn) async { for (final dayId in sorted) { - final startSec = _localDayStartSec(dayId); - final endSec = startSec + _daySec; + final (startSec, endSec) = _localDayWindow(dayId); deleted += await txn.delete( 'decoded_rr', where: @@ -2881,6 +3132,16 @@ class LocalDb { where: 'ts >= ? AND ts < ?', whereArgs: [startSec, endSec], ); + // CASCADE the GPS route BEFORE its session row disappears — otherwise + // the join key is gone and every lat/lng point of a deleted run stays + // on disk forever (deleteSession cascades explicitly; this path did + // not). Must run first: once `sessions` is deleted the subquery is + // empty and the route is unreachable. + deleted += await txn.rawDelete( + 'DELETE FROM workout_route WHERE session_id IN ' + '(SELECT id FROM sessions WHERE start_ts >= ? AND start_ts < ?)', + [startSec, endSec], + ); deleted += await txn.delete( 'sessions', where: 'start_ts >= ? AND start_ts < ?', @@ -2899,6 +3160,11 @@ class LocalDb { await deleteByIn(txn, 'notifications', 'date', sorted); await deleteByIn(txn, 'sleep_session_candidates', 'day_id', sorted); await deleteByIn(txn, 'wake_day_features', 'day_id', sorted); + // Day-keyed USER rows. Same class of leak as workout_route: "delete this + // day" must not leave the user's own logged health data behind. + await deleteByIn(txn, 'cycle_symptom', 'date', sorted); + await deleteByIn(txn, 'workout_suggestions', 'date', sorted); + await deleteByIn(txn, 'sleep_override', 'day_id', sorted); }); return deleted; } @@ -2976,7 +3242,20 @@ class LocalDb { } var copied = 0; await db.transaction((txn) async { - final batch = txn.batch(); + // CHUNKED, for the same reason commitSyncBatch chunks: sqflite + // serialises a whole batch's args into ONE platform message. A + // full-history import is hundreds of thousands of rows, and the + // orphan guard below adds an op per decoded_onehz row on top. + const chunkOps = 4000; + var batch = txn.batch(); + var ops = 0; + Future flush() async { + if (ops == 0) return; + await batch.commit(noResult: true); + batch = txn.batch(); + ops = 0; + } + for (final r in rows) { final row = { for (final e in r.entries) @@ -2989,10 +3268,24 @@ class LocalDb { )) { continue; // locally finalized — never overwritten by an import } + // ORPHAN GUARD ON THE IMPORT PATH. A plain replace-insert into + // decoded_onehz bypasses _queueDecodedOneHz entirely, so a foreign + // row colliding on UNIQUE(rec_ts) (different counter) or on the + // `counter` PRIMARY KEY (different second) evicted a local row and + // stranded its decoded_rr beats — the exact leak the ingest path is + // guarded against, wide open here. Queue the SAME guard, in the + // same batch/transaction, right before the row. + if (t == 'decoded_onehz') { + final counter = (row['counter'] as num?)?.toInt(); + final recTs = (row['rec_ts'] as num?)?.toInt(); + if (counter == null || recTs == null) continue; + ops += _queueOrphanGuard(batch, counter: counter, recTs: recTs); + } batch.insert(t, row, conflictAlgorithm: ConflictAlgorithm.replace); copied++; + if (++ops >= chunkOps) await flush(); } - await batch.commit(noResult: true); + await flush(); }); counts[t] = copied; } @@ -3862,17 +4155,8 @@ class LocalDb { /// existing workout_route table. Guarded — fresh installs get it from /// _createWorkoutRoute directly once that's updated; ALTER … ADD COLUMN /// throws if it's already there. - static Future _ensureWorkoutRouteSpeed(Database db) async { - final info = await db.rawQuery('PRAGMA table_info(workout_route)'); - final has = info.any((c) => c['name'] == 'speed'); - if (!has) { - try { - await db.execute('ALTER TABLE workout_route ADD COLUMN speed REAL'); - } catch (_) { - /* another opener won the race — column now exists */ - } - } - } + static Future _ensureWorkoutRouteSpeed(Database db) => + _addColumnIfMissing(db, 'workout_route', 'speed', 'REAL'); /// Append a batch of route rows (INSERT OR REPLACE — idempotent on /// (session_id, seq)). Each row is a [RoutePoint.toRow] map. diff --git a/lib/data/local_repository_impl.dart b/lib/data/local_repository_impl.dart index 5397c0d8..b1b3e21e 100644 --- a/lib/data/local_repository_impl.dart +++ b/lib/data/local_repository_impl.dart @@ -679,18 +679,29 @@ class LocalRepositoryImpl extends LocalRepository { final b = await _bundleForDate(date); if (b == null) return const {}; final cov = _sub(b, 'coverage'); - final total = (cov?['hr_samples'] as num?)?.toInt() ?? 0; + final hrSamples = (cov?['hr_samples'] as num?)?.toInt(); // Wear block (on/off segments, first/last on, longest off) computed in the // engine; fall back to the coverage counts when absent. final w = b['wear'] is Map ? (b['wear'] as Map).cast() : null; + // MISSING IS NOT ZERO. This used to collapse "we never measured wear" into + // `worn_min: 0` via `hr_samples ?? 0`, which an imported day (no `wear` + // block, no `coverage` block) hits every time — making it byte-identical to + // a day the strap genuinely sat in a drawer, so the screen asserted "Not + // worn on this day — no wrist contact was recorded" about data it simply + // never had. Resolution order is now: engine wear block, then the day's + // own `worn_min` scalar, then the coverage record count, then absent. + final scalarWornMin = (_sub(b, 'scalars')?['worn_min'] as num?)?.toInt(); + final wornMin = (w?['worn_min'] as num?)?.toInt() ?? + scalarWornMin ?? + (hrSamples == null ? null : (hrSamples / 60).round()); return { // Wear = RECORD presence, not valid HR (HR drops out during daytime // motion). Fall back to the total record count, never hr_valid. - 'worn_min': (w?['worn_min'] as num?)?.toInt() ?? (total / 60).round(), - 'coverage_pct': - (w?['coverage_pct'] as num?)?.toInt() ?? (total > 0 ? 100 : 0), + 'worn_min': wornMin, + 'coverage_pct': (w?['coverage_pct'] as num?)?.toInt() ?? + (hrSamples == null ? null : (hrSamples > 0 ? 100 : 0)), 'segments': w?['segments'] ?? const [], 'first_on': w?['first_on'], 'last_on': w?['last_on'], @@ -860,7 +871,7 @@ class LocalRepositoryImpl extends LocalRepository { // complete day. final bundleDate = (b['date'] as String?) ?? date; final dayStart = _localMidnightSec(bundleDate); - final dayEnd = dayStart + 86400; + final dayEnd = _localDayEndSec(bundleDate); // Sleep span (onset/wake) for the context band + sleep symbol. final sw = _sub(b, 'sleep.window.value'); @@ -876,15 +887,19 @@ class LocalRepositoryImpl extends LocalRepository { // Workouts + device events for that calendar day. final sess = await LocalDb.sessionsInRange(dayStart, dayEnd); - final allEvents = await LocalDb.unuploadedEvents(limit: 2000); + // Bounded BY THE DAY, in SQL. This used to pull `unuploadedEvents(limit: + // 2000)` — `ORDER BY ts ASC LIMIT 2000`, i.e. the OLDEST 2000 rows — and + // then filter that page down to this day. Once `events` held more than 2000 + // rows the page could no longer reach recent days at all, so their markers + // silently vanished from the timeline (the same oldest-N-vs-trailing-N + // shape as the metricSeries(limit:) outage). + final dayEvents = await LocalDb.eventsInRange(dayStart, dayEnd); final events = >[ - for (final e in allEvents) - if (((e['ts'] as num?)?.toInt() ?? -1) >= dayStart && - ((e['ts'] as num?)?.toInt() ?? -1) < dayEnd) - { - 'event_id': (e['event_id'] as num?)?.toInt(), - 'ts': (e['ts'] as num?)?.toInt(), - }, + for (final e in dayEvents) + { + 'event_id': (e['event_id'] as num?)?.toInt(), + 'ts': (e['ts'] as num?)?.toInt(), + }, ]; // Daytime naps (principled detectNaps) as their own bands on the timeline. @@ -969,15 +984,15 @@ class LocalRepositoryImpl extends LocalRepository { } /// Local midnight (epoch sec) of a 'YYYY-MM-DD' date string. - int _localMidnightSec(String ymd) { - final p = ymd.split('-'); - if (p.length != 3) return 0; - final y = int.tryParse(p[0]), - m = int.tryParse(p[1]), - d = int.tryParse(p[2]); - if (y == null || m == null || d == null) return 0; - return DateTime(y, m, d).millisecondsSinceEpoch ~/ 1000; - } + int _localMidnightSec(String ymd) => localDayStartSec(ymd) ?? 0; + + /// End of that local day (epoch sec) — the NEXT local midnight. + /// + /// NOT `_localMidnightSec(ymd) + 86400`: a spring-forward day is 23 h local + /// and a fall-back day is 25 h, so the flat +86400 window pulled in an hour + /// of the next day (or dropped the last hour) on exactly those two days a + /// year. See day_label.dart. + int _localDayEndSec(String ymd) => localDayEndSec(ymd) ?? 0; // ── lists / summaries ───────────────────────────────────────────────────── @@ -1478,7 +1493,7 @@ class LocalRepositoryImpl extends LocalRepository { final b = await _bundleForDate(today); final curve = (_sub(b, 'series')?['hr_curve'] as List?) ?? const []; final dayStart = _localMidnightSec(today); - final dayEnd = dayStart + 86400; + final dayEnd = _localDayEndSec(today); return { 'points': [ for (final e in curve) @@ -1505,18 +1520,17 @@ class LocalRepositoryImpl extends LocalRepository { @override Future> getRecords() async { - final rows = await LocalDb.recentDayResults(3650); - final days = rows.length; - int nights = 0; - final sleepDays = {}; - for (final r in rows) { - final b = _decode(r['payload_json']); - if (_sub(b, 'sleep.accounting.value')?['tst_sec'] != null) { - nights++; - final d = r['date'] as String?; - if (d != null) sleepDays.add(d); - } - } + // PAYLOAD-FREE. This used to be `recentDayResults(3650)` — `SELECT r.*` + // over TEN YEARS of day_result, dragging every bundle's hr_curve / + // hypnogram / HRV series across and `jsonDecode`ing each on the main + // isolate, for a screen that only ever needs scalar extremes. At ~2 years + // of history that is hundreds of MB decoded to compute two counts. Both + // are now answered in SQLite: day labels from an index-only GROUP BY, the + // sleep count via json_extract (only the scalar crosses the boundary). + final dayLabelList = await LocalDb.dayResultDayIdsDesc(); + final days = dayLabelList.length; + final sleepDays = await LocalDb.daysWithSleepTst(); + final nights = sleepDays.length; // Personal records from the day scalars (metric_series) + the sessions // table — computed locally with the record's own date attached. @@ -1598,10 +1612,7 @@ class LocalRepositoryImpl extends LocalRepository { } // Streaks: consecutive most-recent days with derived data / with sleep. - final dayLabels = { - for (final r in rows) - if (r['date'] is String) r['date'] as String, - }; + final dayLabels = dayLabelList.toSet(); int streakOf(Set have) { var streak = 0; var d = DateTime.now(); @@ -2261,8 +2272,18 @@ class LocalRepositoryImpl extends LocalRepository { // Phase + fertile window — only when meanLength is known (else honest unknown). String phase = 'unknown'; String? fertileStart, fertileEnd; - if (meanLength != null && cycleDay != null && lastStart != null) { - final ovDay = (meanLength - 14).round().clamp(10, meanLength.round()); + // A mean cycle shorter than the 10-day ovulation floor makes the clamp + // bounds cross — `clamp(10, 8)` THROWS ArgumentError (lowerLimit > + // upperLimit), and it threw straight out of getCycle() so the entire cycle + // screen errored instead of degrading. Two logged `start` markers 8 days + // apart is enough: a mis-tap the user then corrected, or a genuinely short + // cycle. Below the floor there is no defensible ovulation day to place, so + // be honest — leave `phase: 'unknown'` and publish no fertile window + // (predictedNext / cycleDay / the biometric overlay still render). + final ovDay = (meanLength == null || meanLength.round() < 10) + ? null + : (meanLength - 14).round().clamp(10, meanLength.round()); + if (ovDay != null && cycleDay != null && lastStart != null) { if (cycleDay <= 5) { phase = 'menstrual'; } else if (cycleDay < ovDay) { diff --git a/lib/gps/route_math.dart b/lib/gps/route_math.dart index b4d5f6e1..8de0af88 100644 --- a/lib/gps/route_math.dart +++ b/lib/gps/route_math.dart @@ -203,10 +203,16 @@ List computeSplits( for (var i = 1; i < pts.length; i++) { final prev = pts[i - 1]; final cur = pts[i]; - var segLen = - haversineMeters(prev.lat, prev.lng, cur.lat, cur.lng); final segStartTs = prev.tsMs; final segEndTs = cur.tsMs; + var segLen = haversineMeters(prev.lat, prev.lng, cur.lat, cur.lng); + // SAME filter as [totalDistanceMeters]: a teleport across a recording gap + // (tunnel, screen-off, signal loss) is a SEGMENT BREAK, not distance. + // Without this, a 55 km jump made the headline `distanceMeters` read ~5 km + // while `splitsKm` emitted ~60 mostly-phantom splits on the same screen. + // Time still elapses across the break, so the split it lands in keeps its + // real duration — only the bogus distance is dropped. + if (isImplausibleSegment(segLen, segEndTs - segStartTs)) segLen = 0.0; // A single segment may cross one or more split boundaries. Walk the // boundaries, interpolating the crossing time linearly along the segment. diff --git a/lib/gps/route_tracker.dart b/lib/gps/route_tracker.dart index 7cc15f29..82e40951 100644 --- a/lib/gps/route_tracker.dart +++ b/lib/gps/route_tracker.dart @@ -147,6 +147,9 @@ class RouteTracker { // Surface — a dead location service otherwise looks like eternal // "Waiting for GPS…". The subscription stays up (cancelOnError: false) // so fixes resume seamlessly if the service comes back. + // `_stopped` guard: an event already in flight when stop()/dispose() + // cancelled the subscription must not write to a disposed notifier. + if (_stopped) return; error.value = e.toString(); }, cancelOnError: false, @@ -250,11 +253,21 @@ class RouteTracker { } } - /// Stop tracking and flush any buffered tail. Idempotent. Retries the final - /// flush once — after stop() no later flush ever runs, so a single transient - /// sink failure here used to silently drop the route's tail. + /// Stop tracking and flush any buffered tail, then release the tracker. + /// Idempotent. Retries the final flush once — after stop() no later flush + /// ever runs, so a single transient sink failure here used to silently drop + /// the route's tail. + /// + /// TERMINAL: a tracker is single-use (`start` is a no-op after the first + /// call), and both owners in AppState drop their reference the moment they + /// call this, so stop() also runs [dispose]. Previously nothing ever called + /// dispose(), which leaked the six ValueNotifiers (and their listeners) once + /// per route workout. Read the notifiers BEFORE awaiting stop(). Future stop() async { - if (_stopped) return; + if (_stopped) { + dispose(); + return; + } _stopped = true; _watchdog?.cancel(); _watchdog = null; @@ -262,10 +275,24 @@ class RouteTracker { _sub = null; await _flush(); if (_buffer.isNotEmpty) await _flush(); // one retry for the tail + dispose(); } + bool _disposed = false; + + /// Release every resource the tracker holds. Idempotent, and safe to call + /// WITHOUT stop() — it cancels the GPS subscription too, which the previous + /// implementation did not: it only cancelled the watchdog, so a dispose() + /// without a stop() left the location stream (and the platform's GPS + /// hardware session behind it) running for the life of the process. void dispose() { + if (_disposed) return; + _disposed = true; + _stopped = true; _watchdog?.cancel(); + _watchdog = null; + unawaited(_sub?.cancel()); + _sub = null; path.dispose(); current.dispose(); distanceMeters.dispose(); diff --git a/lib/import/noop_import.dart b/lib/import/noop_import.dart index e6daa1b7..245b9bde 100644 --- a/lib/import/noop_import.dart +++ b/lib/import/noop_import.dart @@ -26,9 +26,21 @@ import '../compute/substrate.dart'; class NoopImportResult { final int days; final int rows; - NoopImportResult(this.days, this.rows); + + /// Rows whose local date had ALREADY been derived and pruned by the time + /// they appeared in the file (a genuinely out-of-order export). They cannot + /// be folded back in, so they are counted here rather than silently dropped + /// — a non-zero value means the export was not fully time-ordered. + final int lateRows; + NoopImportResult(this.days, this.rows, [this.lateRows = 0]); } +/// What to do with an incoming row given the import's high-water date. +/// [advance] closes out the previous date; [buffer] folds the row into the +/// current rolling window; [late] means its day was already derived + pruned, +/// so the row cannot be used (counted in [NoopImportResult.lateRows]). +enum RowOrder { advance, buffer, late } + /// One second's worth of the 1 Hz channels (sparse — only set streams present). class _Sec { int? hr; @@ -63,7 +75,8 @@ class NoopImporter { final rrTs = []; // beat end time (epoch ms) final rrMs = []; String? curDate; - var totalRows = 0, daysDone = 0; + final derived = {}; // dates already derived + pruned out of `secs` + var totalRows = 0, daysDone = 0, lateRows = 0; Future deriveAndPrune(String date) async { // Build a Substrate from everything buffered (prev + current date) and @@ -113,11 +126,34 @@ class NoopImporter { final stream = at(f, 'stream'); final date = localDateLabel(ts); - // Date advanced → the previous date is complete; derive it from the window. - if (curDate != null && date != curDate && _after(date, curDate)) { - await deriveAndPrune(curDate); + // + // OUT-OF-ORDER ROWS. `curDate` is a HIGH-WATER mark and must only ever + // move forward. It used to be assigned unconditionally, so one backwards + // timestamp rewound it; the next forward row then called + // deriveAndPrune(), whose `secs.removeWhere(label != date)` + // discarded every buffered sample for the newer day — silent, unbounded + // loss with nothing surfaced. + // + // Now: a forward row closes out the previous date (derive + prune); a + // backwards row for a day we have NOT derived yet is simply folded into + // the buffer at its own timestamp (the Substrate is rebuilt sorted by ts, + // so arrival order never mattered); and a row for a day already derived + // is genuinely too late to use — counted in + // [NoopImportResult.lateRows] and reported instead of being allowed to + // wipe the current day. + switch (decideRow(date, curDate, derived)) { + case RowOrder.advance: + if (curDate != null) { + await deriveAndPrune(curDate); + derived.add(curDate); + } + curDate = date; + case RowOrder.buffer: + break; + case RowOrder.late: + lateRows++; + continue; } - curDate = date; totalRows++; switch (stream) { @@ -161,7 +197,7 @@ class NoopImporter { } await engine.finalizeImport(profile); - return NoopImportResult(daysDone, totalRows); + return NoopImportResult(daysDone, totalRows, lateRows); } /// Build a Substrate from the buffered seconds + RR beats. Gravity / SpO₂ / @@ -224,4 +260,22 @@ class NoopImporter { /// String date compare 'YYYY-MM-DD' — true when [a] is strictly after [b]. static bool _after(String a, String b) => a.compareTo(b) > 0; + + /// Where a row belongs given the import's high-water date [curDate] and the + /// set of dates already [derived] (and therefore already pruned out of the + /// rolling buffer). + /// + /// This is the whole out-of-order contract, isolated so it can be tested + /// without a database: `curDate` only ever moves FORWARD. It used to be + /// assigned unconditionally, so one backwards timestamp rewound it and the + /// next forward row called `deriveAndPrune()` — whose + /// `secs.removeWhere(label != date)` threw away every buffered sample of the + /// NEWER day, silently and with no error surfaced. + static RowOrder decideRow(String date, String? curDate, Set derived) { + if (curDate == null || _after(date, curDate)) return RowOrder.advance; + if (date == curDate) return RowOrder.buffer; + // Older than the high-water day. Still usable as prior-evening context + // unless its day has already been derived and pruned. + return derived.contains(date) ? RowOrder.late : RowOrder.buffer; + } } diff --git a/lib/import/whoop_import.dart b/lib/import/whoop_import.dart index a2d2f9d3..eb8288ff 100644 --- a/lib/import/whoop_import.dart +++ b/lib/import/whoop_import.dart @@ -22,10 +22,49 @@ import '../data/db.dart'; class WhoopImportResult { final int days; final int workouts; - WhoopImportResult(this.days, this.workouts); + + /// Days present in the export that were NOT written because the device + /// already holds a REAL (1 Hz-derived) day for that date. Vendor snapshots + /// never replace measured data — see [WhoopImporter._writeDay]. + final int skippedExistingDays; + WhoopImportResult(this.days, this.workouts, [this.skippedExistingDays = 0]); +} + +/// One CSV row, addressed BY HEADER NAME. Also exposes WHICH header matched, so +/// unit-bearing columns ("… (cal)" vs "… (kJ)") can be interpreted from their +/// declared unit instead of guessed from the magnitude of the number. +class _Row { + final Map col; + final List f; + const _Row(this.col, this.f); + + String get(List names) => _pick(names).$2; + + /// The header name that matched (lower-cased), or '' if none did. + String header(List names) => _pick(names).$1; + + (String, String) _pick(List names) { + for (final n in names) { + final i = col[n]; + if (i != null && i < f.length) return (n, f[i].trim()); + } + return ('', ''); + } } class WhoopImporter { + /// Column aliases for the energy field. The unit is read from whichever of + /// these actually matched — never inferred from the value (see [_kcal]). + static const List _energyCols = [ + 'energy burned (cal)', + 'energy burned (kcal)', + 'energy burned (kilocalories)', + 'energy burned (kj)', + 'energy burned (kilojoules)', + 'energy burned (kilojoule)', + 'energy burned', + ]; + /// Import one or more WHOOP export CSVs. Derived snapshots only. Pass [engine] /// + [profile] to run the cross-day rollup / baseline refresh once at the end. static Future importFiles( @@ -34,7 +73,17 @@ class WhoopImporter { Profile? profile, void Function(int done)? onProgress, }) async { - var days = 0, workouts = 0; + var days = 0, workouts = 0, skipped = 0; + // Day labels that still have raw 1 Hz substrate on device. An imported + // snapshot for one of these must NEVER be finalized — finalizing locks the + // day out of DerivationEngine forever, so the real signal could never + // replace WHOOP's numbers. + Set rawDays; + try { + rawDays = (await LocalDb.decodedRecTsMaxByDay()).keys.toSet(); + } catch (_) { + rawDays = const {}; + } for (final path in paths) { final rows = await _readCsv(path); if (rows.length < 2) continue; @@ -46,43 +95,77 @@ class WhoopImporter { for (var r = 1; r < rows.length; r++) { final f = rows[r]; if (f.isEmpty) continue; - String get(List names) { - for (final n in names) { - final i = col[n]; - if (i != null && i < f.length) return f[i].trim(); - } - return ''; - } + final row = _Row(col, f); if (kind == _Kind.workout) { - if (await _writeWorkout(get)) workouts++; + if (await _writeWorkout(row)) workouts++; } else if (kind == _Kind.day) { - if (await _writeDay(get)) days++; - onProgress?.call(days); + switch (await _writeDay(row, rawDays)) { + case _DayWrite.written: + days++; + onProgress?.call(days); + case _DayWrite.keptExisting: + skipped++; + case _DayWrite.unusable: + break; + } } } } if (engine != null && profile != null) { await engine.finalizeImport(profile); } - return WhoopImportResult(days, workouts); + return WhoopImportResult(days, workouts, skipped); } // ── per-row writers ────────────────────────────────────────────────────────── - static Future _writeDay(String Function(List) get) async { + /// True when [row] is a day the device DERIVED itself (from real 1 Hz), as + /// opposed to absent, a skip marker, or a previous vendor import. Such a day + /// is never overwritten: `putDayResult` is INSERT OR REPLACE on both + /// `day_result` and `metric_series`, so writing over it would permanently + /// destroy measured data that a returning user cannot get back. + static bool _isRealDerivedDay(Map? row) { + if (row == null) return false; + if (((row['skipped'] as num?) ?? 0).toInt() == 1) return false; + try { + final p = jsonDecode((row['payload_json'] as String?) ?? '{}'); + if (p is Map) { + if (p['skipped'] == true) return false; + // A prior import (this importer, or the cloud one) is replaceable — + // both are vendor snapshots, neither is measured on-device data. + if (p['imported'] == true) return false; + } + } catch (_) { + // Present but unreadable — treat as real and refuse to clobber it. + return true; + } + return true; + } + + static Future<_DayWrite> _writeDay(_Row row, Set rawDays) async { + String get(List names) => row.get(names); final wakeTs = _parseTs(get(['wake onset', 'sleep onset', 'cycle start time'])); final cycleStart = _parseTs(get(['cycle start time', 'sleep onset'])); final anchor = wakeTs ?? cycleStart; - if (anchor == null) return false; + if (anchor == null) return _DayWrite.unusable; final date = localDateLabel(anchor); + // NEVER clobber a real derived day. The import is reachable from onboarding + // AND from Profile, so a returning user with months of band data importing + // their WHOOP export used to have every overlapping day's payload and + // scalars replaced by the vendor's numbers — and `finalized: true` then + // locked the day so DerivationEngine could never rebuild it from raw. + if (_isRealDerivedDay(await LocalDb.dayResult(date))) { + return _DayWrite.keptExisting; + } + num? n(List names) => double.tryParse(get(names)); final recovery = n(['recovery score %', 'recovery score']); final rhr = n(['resting heart rate (bpm)', 'resting heart rate']); final rmssd = n(['heart rate variability (ms)', 'heart rate variability (rmssd) (ms)']); final strain = n(['day strain', 'strain']); - final calories = _kcal(get(['energy burned (cal)', 'energy burned'])); + final calories = _kcal(get(_energyCols), row.header(_energyCols)); final resp = n(['respiratory rate (rpm)', 'respiratory rate']); final spo2 = n(['blood oxygen %', 'blood oxygen']); final skinTempC = n(['skin temp (celsius)', 'skin temperature (celsius)']); @@ -169,7 +252,10 @@ class WhoopImporter { algoVersion: kAlgoVersion, payloadJson: jsonEncode(bundle), windowJson: jsonEncode(win ?? const {}), - finalized: true, + // Finalizing locks a day out of DerivationEngine permanently. Only safe + // when there is no raw substrate left to re-derive from; a day that still + // has 1 Hz raw stays open so the real signal supersedes WHOOP's numbers. + finalized: !rawDays.contains(date), rhr: d(rhr), rmssd: d(rmssd), readiness: d(recovery), @@ -188,10 +274,11 @@ class WhoopImporter { 'efficiency': d(effPct), }, ); - return true; + return _DayWrite.written; } - static Future _writeWorkout(String Function(List) get) async { + static Future _writeWorkout(_Row row) async { + String get(List names) => row.get(names); final start = _parseTs(get(['workout start time', 'start time'])); final end = _parseTs(get(['workout end time', 'end time'])); if (start == null) return false; @@ -203,7 +290,7 @@ class WhoopImporter { 'type': _slug(get(['activity name', 'activity'])), 'status': 'done', 'source': 'whoop', - 'calories': _kcal(get(['energy burned (cal)', 'energy burned']))?.toDouble(), + 'calories': _kcal(get(_energyCols), row.header(_energyCols))?.toDouble(), 'strain': n(['activity strain', 'strain'])?.toDouble(), 'max_hr': n(['max hr (bpm)', 'max heart rate (bpm)'])?.toInt(), 'duration_min': (end != null) ? ((end - start) / 60).round() : null, @@ -247,12 +334,23 @@ class WhoopImporter { return null; } - /// "Energy burned" in WHOOP exports is sometimes kilojoules; values >4000 are - /// almost certainly kJ → convert to kcal. Otherwise treat as kcal. - static num? _kcal(String s) { + /// "Energy burned" is exported in kcal by some WHOOP locales and in + /// kilojoules by others. The unit comes from the COLUMN HEADER that matched, + /// never from the magnitude of the value: the old `v > 4000 ? v / 4.184 : v` + /// heuristic silently rewrote a real 4,500 kcal day (an ultra, a long ride) + /// as 1,076 kcal, and that number then flowed into `metric_series` and into + /// Apple Health / Health Connect as active energy. + /// + /// If the header carries no unit at all, the value is genuinely ambiguous and + /// we drop it rather than guess — a missing calorie figure is honest, a + /// wrong one is not. + static num? _kcal(String s, String header) { final v = double.tryParse(s); if (v == null) return null; - return v > 4000 ? v / 4.184 : v; + final h = header.toLowerCase(); + if (h.contains('kj') || h.contains('kilojoule')) return v / 4.184; + if (h.contains('cal')) return v; // cal / kcal / kilocalories + return null; } static String _slug(String s) { @@ -308,3 +406,7 @@ class WhoopImporter { } enum _Kind { day, workout, unknown } + +/// Outcome of one day row: written, deliberately kept (a real derived day +/// already exists for that date), or unusable (no parseable anchor timestamp). +enum _DayWrite { written, keptExisting, unusable } diff --git a/lib/main.dart b/lib/main.dart index 8fa9eb90..bd57c11b 100644 --- a/lib/main.dart +++ b/lib/main.dart @@ -32,7 +32,9 @@ const _kStartupInitTimeout = Duration(seconds: 6); Future main() async { WidgetsFlutterBinding.ensureInitialized(); - // Initialize Firebase (overridden by dummy values until flutterfire configure) + // Initialize Firebase (overridden by dummy values until flutterfire configure). + // OPTIONAL: a build with no real google-services.json / GoogleService-Info.plist + // throws here and the app carries on without any Firebase at all. try { await Firebase.initializeApp( options: DefaultFirebaseOptions.currentPlatform, @@ -40,6 +42,14 @@ Future main() async { } catch (e) { debugPrint('Firebase init failed (run flutterfire configure!): $e'); } + // ZERO COLLECTION UNTIL CONSENT. The SDKs are already told to stay quiet at + // the platform level (Info.plist / AndroidManifest.xml + // *_collection_enabled=false) so nothing is collected before Dart even runs; + // this restates it programmatically so a build with a stale native config + // still cannot auto-log first_open, an app-start trace, or a startup crash. + // Collection is only ever switched ON later, from the user's loaded consent + // (TelemetryService.applyConsent) — never here. + TelemetryService.instance.enforceCollectionOffUntilConsent(); // Install crash/error hooks (FlutterError.onError + PlatformDispatcher.onError) // BEFORE anything else. Capture is always-on and LOCAL; nothing transmits until diff --git a/lib/notify/notification_center.dart b/lib/notify/notification_center.dart index 74d9c2ff..a99d3b33 100644 --- a/lib/notify/notification_center.dart +++ b/lib/notify/notification_center.dart @@ -11,6 +11,7 @@ import 'dart:async'; import 'package:flutter/foundation.dart'; +import 'package:shared_preferences/shared_preferences.dart'; import '../ai/ai_prefs.dart'; import '../ai/reminder_plan.dart'; @@ -67,15 +68,22 @@ class NotificationCenter { /// background_sync.dart's checkSyncStaleness) MUST pass `false`, so a /// not-yet-decided permission is checked, not requested, and never gets /// permanently mis-cached as "denied" by a background attempt. - Future emit( + /// + /// Returns TRUE only when the event actually reached the OS. Callers that + /// keep their own "already fired today" guard (see [emitOncePerDay]) MUST key + /// it off this, never off the mere fact that emit was called: the event is + /// dropped outright when [NotificationPrefs.shouldFireOs] says no (quiet + /// hours, category muted) or when the OS present fails. + Future emit( NotificationEvent e, { bool allowPermissionPrompt = true, }) async { + var presented = false; try { final prefs = await NotificationPrefs.load(); final now = DateTime.now(); final minuteOfDay = now.hour * 60 + now.minute; - if (!prefs.shouldFireOs(e, minuteOfDay)) return; + if (!prefs.shouldFireOs(e, minuteOfDay)) return false; // Enforce the dedupeKey's "fires at most once" contract (issue #136). // The OS id only REPLACES a prior post of the same key — it still // re-alerts — and derivation re-runs on every BLE sync, so an insight @@ -103,8 +111,44 @@ class NotificationCenter { } finally { if (!shown) await _fired.release(e.dedupeKey); } + presented = shown; }); } catch (_) {/* OS present best-effort */} + return presented; + } + + /// Fire [e] at most once per [dayId], with the persisted day-guard at + /// [prefsKey] consumed ONLY when the notification was actually presented. + /// Returns true iff it fired. + /// + /// The callers of this (recovery-ready, step-goal) used to write the guard + /// FIRST and then emit. [emit] drops the event outright when + /// [NotificationPrefs.shouldFireOs] is false, so a band that syncs at 06:40 — + /// inside the DEFAULT 22:00–07:00 quiet window — computed the new day's + /// recovery, burned the guard, got suppressed, and then had every retry that + /// day blocked by the guard it never earned: "Your recovery is ready" simply + /// never fired. Claiming the guard only on a real present makes the retry + /// (the next derive pass, after 07:00) work. + Future emitOncePerDay({ + required String prefsKey, + required String dayId, + required NotificationEvent e, + bool allowPermissionPrompt = true, + }) async { + SharedPreferences? prefs; + try { + prefs = await SharedPreferences.getInstance(); + } catch (_) { + prefs = null; // no store — [emit]'s own FiredKeyStore still dedupes + } + if (prefs != null && prefs.getString(prefsKey) == dayId) return false; + final shown = await emit(e, allowPermissionPrompt: allowPermissionPrompt); + if (shown && prefs != null) { + try { + await prefs.setString(prefsKey, dayId); + } catch (_) {/* guard is an optimisation; FiredKeyStore is the truth */} + } + return shown; } // Default schedule for standing reminders (user-overridable via prefs UI). diff --git a/lib/notify/notification_event.dart b/lib/notify/notification_event.dart index efacd88e..77aeaa39 100644 --- a/lib/notify/notification_event.dart +++ b/lib/notify/notification_event.dart @@ -42,16 +42,10 @@ class NotificationEvent { this.route, }); - /// Stable OS notification id, partitioned by category so a health alert can - /// never overwrite a reminder (and vice-versa). Bands are 100k apart and start - /// well above the fixed device/insight ids (< 3000) defined in the service. - int get osId { - final base = switch (category) { - NotifCategory.recovery => 200000, - NotifCategory.health => 300000, - NotifCategory.reminders => 400000, - NotifCategory.device => 100000, - }; - return base + (dedupeKey.hashCode.abs() % 100000); - } + // The OS notification id is NOT derived here any more. It used to be + // `categoryBase + dedupeKey.hashCode.abs() % 100000` — a hash modulo, so two + // distinct dedupeKeys in the same category could map onto the same id, and + // `_plugin.show` REPLACES rather than stacks: one notification silently + // vanished. Ids are now allocated collision-free per dedupeKey — see + // notification_ids.dart (NotificationIds.idFor). } diff --git a/lib/notify/notification_ids.dart b/lib/notify/notification_ids.dart new file mode 100644 index 00000000..a3e5bd72 --- /dev/null +++ b/lib/notify/notification_ids.dart @@ -0,0 +1,176 @@ +// notification_ids.dart — collision-free OS notification id ALLOCATION. +// +// The id handed to `FlutterLocalNotificationsPlugin.show` decides which post a +// notification lands on: the SAME id REPLACES the notification already in the +// shade, it does not stack beside it. Ids used to be DERIVED from the dedupeKey +// as `categoryBase + dedupeKey.hashCode.abs() % 100000` — a hash modulo, so two +// DIFFERENT dedupeKeys in the same category whose hashes agree mod 100000 map +// onto the same id and one of the two notifications silently vanishes with no +// trace. The old comment ("partitioned so a health alert can never overwrite a +// reminder") only ever covered CROSS-category collisions; within a category the +// scheme guaranteed nothing. +// +// Ids are now ALLOCATED instead: each dedupeKey takes the next free slot in its +// category's band, recorded in shared_preferences so the id stays stable across +// restarts (a re-post of the same logical event still replaces in place, which +// is the one property the hash gave us for free). A reverse index (slot → key) +// makes occupancy explicit, so an allocation can never land on a slot another +// key already owns. +// +// RETENTION. Allocations are pruned on the same schedule as FiredKeyStore's +// fire-once claims: a date-prefixed dedupeKey older than [retentionDays] can no +// longer be re-posted, so its slot is freed. Undated keys (e.g. +// "alarm_fired:") are rare and left alone. +// +// DEGRADED MODE. With no usable shared_preferences (a plain unit test, a torn +// down background isolate) the in-memory maps below are the whole store: ids +// stay collision-free for the life of the process, they just aren't stable +// across a restart. That is strictly better than the hash it replaces. + +import 'package:flutter/foundation.dart'; +import 'package:shared_preferences/shared_preferences.dart'; + +import '../data/day_label.dart'; +import 'fired_keys.dart'; +import 'notification_event.dart'; + +class NotificationIds { + NotificationIds._(); + static final NotificationIds instance = NotificationIds._(); + + /// Slots per category band. Bands are disjoint and start at [bandBaseFor]. + static const int bandSize = 100000; + + /// How long a dated allocation is kept before its slot is recycled. Matches + /// [FiredKeyStore.retentionDays] — a key that can no longer fire can no + /// longer need its id either. + static const int retentionDays = FiredKeyStore.retentionDays; + + /// How far to probe forward for a free slot before giving up and reusing the + /// counter position. Only reachable with [bandSize] live keys in ONE category + /// (i.e. never, given the prune pass above). + static const int maxProbes = 1024; + + static const String _kSlot = 'notif_osid:'; // ":" → slot + static const String _kOwner = 'notif_osslot:'; // ":" → dedupeKey + static const String _kNext = 'notif_osnext:'; // "" → next slot + + /// The low edge of each category's id band. Kept 100k apart and well above + /// the fixed device/scheduled-reminder ids (< 3000) in NotificationService. + static int bandBaseFor(NotifCategory c) => switch (c) { + NotifCategory.device => 100000, + NotifCategory.recovery => 200000, + NotifCategory.health => 300000, + NotifCategory.reminders => 400000, + }; + + // In-memory mirror of the three prefs namespaces. Also the ONLY store when + // shared_preferences is unavailable (see DEGRADED MODE above). + final Map _slots = {}; + final Map _owners = {}; + final Map _next = {}; + + /// Drop every in-memory allocation. Tests only — a fresh process starts with + /// empty maps and re-reads the persisted ones. + @visibleForTesting + void resetForTest() { + _slots.clear(); + _owners.clear(); + _next.clear(); + } + + /// The stable OS notification id for [e]. Same dedupeKey → same id (replace + /// in place); two different dedupeKeys in the same category → NEVER the same + /// id. Never throws. + Future idFor(NotificationEvent e) async { + final base = bandBaseFor(e.category); + try { + return base + await _slotFor(e); + } catch (_) { + // Absolute last resort: keep the category band correct rather than + // failing the present outright. + return base; + } + } + + Future _slotFor(NotificationEvent e) async { + final cat = e.category.name; + final slotKey = '$_kSlot$cat:${e.dedupeKey}'; + + final memo = _slots[slotKey]; + if (memo != null) return memo; + + SharedPreferences? p; + try { + p = await SharedPreferences.getInstance(); + } catch (_) { + p = null; // no platform prefs — the in-memory maps carry the process + } + if (p != null) { + try { + await p.reload(); // the OTHER isolate may have allocated since + } catch (_) {/* freshness is best-effort */} + final existing = p.getInt(slotKey); + if (existing != null) { + _slots[slotKey] = existing; + return existing; + } + } + + final nextKey = '$_kNext$cat'; + final start = p?.getInt(nextKey) ?? _next[nextKey] ?? 0; + var slot = start % bandSize; + for (var i = 0; i < maxProbes; i++) { + final candidate = (start + i) % bandSize; + final ownerKey = '$_kOwner$cat:$candidate'; + final owner = p?.getString(ownerKey) ?? _owners[ownerKey]; + if (owner == null || owner == e.dedupeKey) { + slot = candidate; + break; + } + } + + final ownerKey = '$_kOwner$cat:$slot'; + _slots[slotKey] = slot; + _owners[ownerKey] = e.dedupeKey; + _next[nextKey] = (slot + 1) % bandSize; + if (p != null) { + try { + await p.setInt(slotKey, slot); + await p.setString(ownerKey, e.dedupeKey); + await p.setInt(nextKey, (slot + 1) % bandSize); + // Never prune the allocation we just made — an event legitimately + // carrying an old date (a backfilled day) would otherwise lose its slot + // the instant it got one. + await _prune(p, keep: slotKey); + } catch (_) {/* the in-memory maps still hold the allocation */} + } + return slot; + } + + /// Free the slots of dated allocations older than [retentionDays], in both + /// directions. Cheap and rare (only after a FRESH allocation). + Future _prune(SharedPreferences p, {String? keep}) async { + try { + final cutoff = + dayLabelOf(DateTime.now().subtract(const Duration(days: retentionDays))); + for (final k in p.getKeys().toList(growable: false)) { + if (!k.startsWith(_kSlot) || k == keep) continue; + final rest = k.substring(_kSlot.length); // ":" + final sep = rest.indexOf(':'); + if (sep < 0) continue; + final cat = rest.substring(0, sep); + final day = FiredKeyStore.leadingDate(rest.substring(sep + 1)); + if (day == null || day.compareTo(cutoff) >= 0) continue; + final slot = p.getInt(k); + await p.remove(k); + _slots.remove(k); + if (slot != null) { + final ownerKey = '$_kOwner$cat:$slot'; + await p.remove(ownerKey); + _owners.remove(ownerKey); + } + } + } catch (_) {/* bounding is best-effort — never break an allocation on it */} + } +} diff --git a/lib/notify/notification_service.dart b/lib/notify/notification_service.dart index 86bdda3f..d3dbd0a2 100644 --- a/lib/notify/notification_service.dart +++ b/lib/notify/notification_service.dart @@ -19,12 +19,53 @@ import 'dart:async'; +import 'package:flutter/foundation.dart'; import 'package:flutter_local_notifications/flutter_local_notifications.dart'; import 'package:flutter_timezone/flutter_timezone.dart'; import 'package:timezone/data/latest_all.dart' as tzdata; import 'package:timezone/timezone.dart' as tz; import 'notification_event.dart'; +import 'notification_ids.dart'; + +/// The next wall-clock instant at [hour]:[minute] — optionally the next +/// [weekday] — strictly after [now], in [now]'s own timezone. +/// +/// CALENDAR arithmetic, never an absolute [Duration]. `d.add(const +/// Duration(days: 1))` adds exactly 24 hours of ELAPSED time, which is NOT "the +/// same wall-clock time tomorrow" across a DST transition: a Sunday-18:00 +/// weekly recap computed over a spring-forward landed at 19:00 (and 17:00 over +/// a fall-back), and the bedtime/hydration dailies drifted the same hour. +/// Rebuilding the [tz.TZDateTime] from its calendar fields pins the wall-clock +/// time and lets the tz database resolve whatever offset that day carries. +@visibleForTesting +tz.TZDateTime nextInstanceOf( + tz.TZDateTime now, + int hour, + int minute, { + int? weekday, +}) { + final loc = now.location; + tz.TZDateTime at(int y, int m, int d) => + tz.TZDateTime(loc, y, m, d, hour, minute); + var d = at(now.year, now.month, now.day); + if (weekday != null) { + // Bounded: any weekday is at most 6 calendar days away. + for (var i = 0; i < 7 && d.weekday != weekday; i++) { + d = at(d.year, d.month, d.day + 1); + } + } + if (!d.isAfter(now)) { + d = at(d.year, d.month, d.day + (weekday != null ? 7 : 1)); + } + return d; +} + +/// The same wall-clock time on the following calendar day (DST-safe — see +/// [nextInstanceOf]). +@visibleForTesting +tz.TZDateTime nextCalendarDay(tz.TZDateTime d) => + tz.TZDateTime(d.location, d.year, d.month, d.day + 1, d.hour, d.minute); class NotificationService { NotificationService._(); @@ -161,28 +202,68 @@ class NotificationService { /// (never requests) via `checkPermissions()` and fails closed to `false` /// rather than attempting to prompt — matching the "in-app feed is ALWAYS /// written, OS presentation is best-effort" contract in NotificationCenter. + /// + /// A cached DENIAL is never final. `_granted` used to latch false for the + /// whole process with nothing to reset it, so a user who denied the prompt + /// (fired at pairing time), went to OS Settings, enabled notifications and + /// came back got ZERO notifications and ZERO scheduled reminders until a full + /// app restart — every presentEvent/scheduleDaily/scheduleWeekly/scheduleOnce + /// early-returned on the stale `false`. Only a GRANT is cached now; a denial + /// is re-read from the live OS state (non-prompting, cheap) on the next call. + /// [invalidatePermissionCache] additionally drops a cached grant so a + /// REVOCATION is noticed too — app.dart calls it on every foreground resume. Future ensurePermission({bool allowPrompt = true}) async { - await init(); - if (_granted != null) return _granted!; - if (!allowPrompt) return hasPermission(); + if (_granted == true) return true; + final request = debugRequestPermission; + if (request == null) await init(); - bool granted = true; - final ios = _plugin.resolvePlatformSpecificImplementation< - IOSFlutterLocalNotificationsPlugin>(); - if (ios != null) { - granted = - await ios.requestPermissions(alert: true, badge: true, sound: true) ?? - false; + if (_granted == false) { + // Denied earlier in this process — re-read the OS rather than trusting a + // stale no. Never re-prompts: once denied, both platforms no-op the + // request anyway, and Settings is the only real path back. + final live = await hasPermission(); + if (live) _granted = true; + return live; } - final android = _plugin.resolvePlatformSpecificImplementation< - AndroidFlutterLocalNotificationsPlugin>(); - if (android != null) { - granted = await android.requestNotificationsPermission() ?? false; + + if (!allowPrompt) return hasPermission(); + + bool granted; + if (request != null) { + granted = await request(); + } else { + granted = true; + final ios = _plugin.resolvePlatformSpecificImplementation< + IOSFlutterLocalNotificationsPlugin>(); + if (ios != null) { + granted = await ios.requestPermissions( + alert: true, badge: true, sound: true) ?? + false; + } + final android = _plugin.resolvePlatformSpecificImplementation< + AndroidFlutterLocalNotificationsPlugin>(); + if (android != null) { + granted = await android.requestNotificationsPermission() ?? false; + } } _granted = granted; return granted; } + /// Drop the cached authorization decision so the next [ensurePermission] / + /// [hasPermission] re-reads the live OS state. Called on every foreground + /// resume (app.dart): the user may have flipped our notification switch + /// either way in Settings while we were backgrounded. + void invalidatePermissionCache() => _granted = null; + + /// Test seams for the platform permission plumbing (there is no plugin to + /// talk to in a unit test). [debugRequestPermission] stands in for the + /// interactive request, [debugProbePermission] for the non-prompting check. + @visibleForTesting + Future Function()? debugRequestPermission; + @visibleForTesting + Future Function()? debugProbePermission; + /// Non-mutating: whether notifications are currently enabled, WITHOUT ever /// showing the OS authorization prompt. Safe to call from any context, /// including headless/background. Does not populate [_granted] — a @@ -190,6 +271,8 @@ class NotificationService { /// "denied" just because a background check happened to run first. Future hasPermission() async { try { + final probe = debugProbePermission; + if (probe != null) return await probe(); await init(); final ios = _plugin.resolvePlatformSpecificImplementation< IOSFlutterLocalNotificationsPlugin>(); @@ -238,8 +321,11 @@ class NotificationService { if (!await ensurePermission(allowPrompt: allowPermissionPrompt)) { return false; } + // Collision-free allocated id (NOT the old hashCode-modulo) — see + // notification_ids.dart. Two same-category events used to be able to + // share an id, and `show` REPLACES: one of them vanished silently. await _plugin.show( - e.osId, + await NotificationIds.instance.idFor(e), e.title, e.body, _details(e.category), @@ -265,19 +351,9 @@ class NotificationService { // ── Scheduling (wall-clock recurring nudges) ──────────────────────────────── - tz.TZDateTime _nextInstanceOf(int hour, int minute, {int? weekday}) { - final now = tz.TZDateTime.now(tz.local); - var d = tz.TZDateTime(tz.local, now.year, now.month, now.day, hour, minute); - if (weekday != null) { - while (d.weekday != weekday) { - d = d.add(const Duration(days: 1)); - } - } - if (!d.isAfter(now)) { - d = d.add(Duration(days: weekday != null ? 7 : 1)); - } - return d; - } + tz.TZDateTime _nextInstanceOf(int hour, int minute, {int? weekday}) => + nextInstanceOf(tz.TZDateTime.now(tz.local), hour, minute, + weekday: weekday); Future scheduleDaily({ required int id, @@ -299,7 +375,8 @@ class NotificationService { if (when.year == now.year && when.month == now.month && when.day == now.day) { - when = when.add(const Duration(days: 1)); + // Calendar day, not +24h — see nextInstanceOf's DST note. + when = nextCalendarDay(when); } } await _plugin.zonedSchedule( diff --git a/lib/state/app_state.dart b/lib/state/app_state.dart index 45294334..b0494a59 100644 --- a/lib/state/app_state.dart +++ b/lib/state/app_state.dart @@ -82,6 +82,27 @@ import 'package:uuid/uuid.dart'; /// personalize (HRmax, calories, TRIMP); it's skipped once those are set. enum AppRoute { loading, welcome, pairing, profile, shell } +/// The healed pairing to persist when the band reports [reportedSerial], or +/// null when nothing should change. +/// +/// HEALS ONLY — it can never CREATE a pairing. The old inline form guarded on +/// `cleanSn != paired?.serial`, which is TRUE when `paired == null`, and then +/// rebuilt a PairedDevice from `paired?.remoteId ?? state.address`. BleEngine's +/// `_teardownSession` never clears `state.serial`/`state.address` (both are set +/// once in `_doConnect`), so a stale engine-state callback arriving AFTER the +/// user unpaired — e.g. the reconnect loop waking from its backoff delay and +/// calling `engine.clearReconnecting()` in its `finally`, which flips the phase +/// to idle and fires `onState` — silently re-created the pairing on disk and +/// bounced the app from Pairing straight back to the Shell. Unpair/sign-out was +/// undone with no user action at all. +PairedDevice? healedPairing(PairedDevice? current, String? reportedSerial) { + if (current == null) return null; // nothing to heal — do NOT pair + final clean = cleanDeviceLabel(reportedSerial); + if (clean == null || clean == current.serial) return null; + if (current.remoteId.isEmpty) return null; + return PairedDevice(current.remoteId, clean); +} + class AppState extends ChangeNotifier { late final BleEngine engine; PairedDevice? paired; @@ -411,7 +432,11 @@ class AppState extends ChangeNotifier { final t = TelemetryService.instance; t.deviceId = deviceId; - t.enabled = telemetryConsent; + // The ONE point where Firebase collection may be switched on, and only + // with the user's AFFIRMATIVELY LOADED consent (the prefs read above). + // Until this runs, TelemetryService.enforceCollectionOffUntilConsent() + // (called from main after Firebase.initializeApp) keeps every SDK off. + t.applyConsent(telemetryConsent); t.consentVersion = termsVersion; t.bandSnapshot = _bandSnapshot; HealthUploader.instance.deviceId = deviceId; @@ -452,7 +477,7 @@ class AppState extends ChangeNotifier { final prefs = await SharedPreferences.getInstance(); await prefs.setBool(_kTelemetryConsent, on); await prefs.setBool(_kConsentChosen, true); - TelemetryService.instance.enabled = on; + TelemetryService.instance.applyConsent(on); notifyListeners(); unawaited( CompanionClient.postConsent( @@ -684,6 +709,32 @@ class AppState extends ChangeNotifier { unawaited(checkPendingSiriRoute()); } + /// Build the object graph WITHOUT running [_init] and without touching a + /// single platform plugin (no DB read, no prefs load, no BLE session, no + /// notification/widget channels), so the state machines above can be + /// unit-tested. Tests only. + /// + /// [engine] lets a test substitute a BleEngine subclass (e.g. one whose + /// stream arming throws). When supplied it is used AS GIVEN — its callbacks + /// are the test's responsibility, not wired back into this AppState. + @visibleForTesting + AppState.forTesting({BleEngine? engine}) { + _background = false; + _gestureDispatcher = GestureDispatcher( + settings: gestureSettings, + log: _log, + onMarkMoment: _markMomentFromGesture, + onWorkoutToggle: _toggleWorkoutFromGesture, + ); + this.engine = engine ?? + BleEngine( + onRecord: _onRecord, + onState: _onEngineState, + log: _log, + onEvent: _onLiveEvent, + ); + } + /// A Siri/Shortcuts App Intent (e.g. "start breathing") may have set a /// pending route in the App Group before launching/foregrounding the app — /// see WidgetService.consumePendingRoute + StartBreathingIntent in @@ -698,17 +749,65 @@ class AppState extends ChangeNotifier { @override void dispose() { + // EVERY timer this object owns, not just three of them. _spotTimer, + // _breathingRecomputeTimer and _workoutTimer used to survive dispose, and + // each of their callbacks ends in notifyListeners() on a disposed + // ChangeNotifier (which throws in release). _tapSub?.cancel(); _stopBackfillTimer(); _alarmGraceTimer?.cancel(); + _alarmGraceTimer = null; + _spotTimer?.cancel(); + _spotTimer = null; + _breathingRecomputeTimer?.cancel(); + _breathingRecomputeTimer = null; + _workoutTimer?.cancel(); + _workoutTimer = null; BandOwnership.markForegroundIntent(false); _releaseForegroundLease(); _deriveScheduler.dispose(); _waterBuzzer.dispose(); + // Owned notifiers/observers. notificationRelay in particular holds a + // WidgetsBindingObserver, a 120 s Timer.periodic and a StreamSubscription — + // its observer accumulated on the binding across every hot restart. + notificationRelay.dispose(); + gestureSettings.dispose(); + navRequest.dispose(); + screenRequest.dispose(); insightsRevision.dispose(); super.dispose(); } + /// Arm every periodic/one-shot timer this object owns, so a test can prove + /// [dispose] actually cancels all of them (an outstanding Timer fails a + /// `testWidgets` case). Tests only — nothing in the app calls this. + @visibleForTesting + void debugArmOwnedTimers() { + _backfillTimer ??= Timer.periodic(_backfillInterval, (_) {}); + _alarmGraceTimer ??= Timer(const Duration(minutes: 5), () {}); + _spotTimer ??= Timer.periodic(const Duration(seconds: 1), (_) {}); + _breathingRecomputeTimer ??= + Timer.periodic(_breathingRecomputeInterval, (_) {}); + _workoutTimer ??= Timer.periodic(const Duration(seconds: 1), (_) {}); + } + + /// True while some foreground feature is holding the live streams open — + /// the gate [_maybeDowngradeLiveForBackground] consults. Tests only. + @visibleForTesting + bool get debugHasLiveConsumer => _hasLiveConsumer; + + /// Run the orphaned-live-workout reconcile directly. Tests only — in the app + /// it is kicked unawaited from [_init]. + @visibleForTesting + Future debugReconcileOrphanedLiveWorkout() => + _reconcileOrphanedLiveWorkout(); + + /// Feed a strap alarm-lifecycle event (56 set / 57–58 fired / 59 cleared) + /// without going through the BLE event path. Tests only. + @visibleForTesting + void debugHandleAlarmEvent(int id) => + _handleAlarmEvent(id, DateTime.now().millisecondsSinceEpoch ~/ 1000); + /// (Re)arm the strap-buzz timer for the hydration reminder from the current /// notification prefs. Call at launch and whenever the prefs change (the /// Notifications screen passes [prefs] so we skip a reload). Slot times come @@ -843,9 +942,16 @@ class AppState extends ChangeNotifier { /* body just omits the slept-for clause */ } - await prefs.setString(_kLastRecoveryNotifDay, dayId); - await NotificationCenter.instance.emit( - NotificationEvent( + // GUARD AFTER PRESENT. Writing _kLastRecoveryNotifDay before the emit + // burned the once-per-day guard on an event that never reached the user: + // a band syncing at 06:40 lands the new day's recovery inside the DEFAULT + // 22:00–07:00 quiet window, emit drops it, and the guard then blocked + // every retry for the rest of the day. emitOncePerDay consumes the guard + // only on a real present, so the next derive pass after 07:00 fires it. + final fired = await NotificationCenter.instance.emitOncePerDay( + prefsKey: _kLastRecoveryNotifDay, + dayId: dayId, + e: NotificationEvent( dedupeKey: '$dayId:recovery_ready', category: NotifCategory.recovery, priority: NotifPriority.normal, @@ -855,7 +961,9 @@ class AppState extends ChangeNotifier { route: '/today', ), ); - _log('[notify] recovery-ready fired for $dayId (score=$score)'); + if (fired) { + _log('[notify] recovery-ready fired for $dayId (score=$score)'); + } } catch (e) { _log('[notify] recovery-ready skipped: $e'); } @@ -957,11 +1065,13 @@ class AppState extends ChangeNotifier { final date = last['date'] as String?; final steps = (last['value'] as num?)?.toInt(); if (date == null || steps == null || steps < goal) return; - final prefs = await SharedPreferences.getInstance(); - if (prefs.getString(_kLastStepGoalDay) == date) return; // already fired - await prefs.setString(_kLastStepGoalDay, date); - await NotificationCenter.instance.emit( - NotificationEvent( + // GUARD AFTER PRESENT — same shape as the recovery-ready fix above: the + // guard used to be written before the emit, so a goal crossed inside + // quiet hours (or with notifications denied) burned the day's only shot. + await NotificationCenter.instance.emitOncePerDay( + prefsKey: _kLastStepGoalDay, + dayId: date, + e: NotificationEvent( dedupeKey: '$date:step_goal', category: NotifCategory.reminders, priority: NotifPriority.low, @@ -1782,14 +1892,12 @@ class AppState extends ChangeNotifier { } // Heal a stale/garbled persisted serial: once the band reports a clean serial // (HELLO body, fixed offset), persist it so the disconnected display stops - // showing any old "?*" junk left by a previous build. - final cleanSn = cleanDeviceLabel(s.serial); - if (cleanSn != null && cleanSn != paired?.serial) { - final rid = paired?.remoteId ?? s.address; - if (rid != null && rid.isNotEmpty) { - paired = PairedDevice(rid, cleanSn); - unawaited(PairedDevice.save(rid, cleanSn)); - } + // showing any old "?*" junk left by a previous build. HEAL ONLY — see + // [healedPairing]: this must never CREATE a pairing. + final healed = healedPairing(paired, s.serial); + if (healed != null) { + paired = healed; + unawaited(PairedDevice.save(healed.remoteId, healed.serial)); } // Keep the lock-screen Band Battery widget current — only when it changed. final battPct = roundedPct ?? -1; @@ -2193,17 +2301,42 @@ class AppState extends ChangeNotifier { case AlarmEffect.fired: _log('[alarm] strap FIRED — EXECUTED (event $id) received.'); unawaited(_notifyAlarmFired()); + // A one-shot alarm is SPENT the moment it fires. This used to only log + // + notify, so `alarmEpoch` kept returning the past epoch across + // relaunches (_init reloads `alarm_epoch`) and Profile's "Smart alarm" + // row went on advertising e.g. "06:30 (7/25)" as the CURRENT alarm + // indefinitely — with live "Test buzz"/"Clear" affordances for an alarm + // that is no longer armed. Clear state AND the persisted epoch. + _clearArmedAlarmState(); break; case AlarmEffect.cleared: - _savedAlarm = null; - device.alarmEpoch = null; - _alarmGraceTimer?.cancel(); + // Same persistence gap on the strap-driven clear (event 59): state was + // nulled but `alarm_epoch` stayed on disk and came back on next launch. + _clearArmedAlarmState(); _log('[alarm] cleared (event $id).'); break; } notifyListeners(); } + /// Drop the armed-alarm state (in-memory + persisted). [AlarmConfirmation]'s + /// `firedAt` deliberately survives `disable()`, so the fired-notification's + /// dedupeKey still resolves after this runs. + void _clearArmedAlarmState() { + _savedAlarm = null; + device.alarmEpoch = null; + _alarm.disable(); + _alarmGraceTimer?.cancel(); + unawaited(() async { + try { + final prefs = await SharedPreferences.getInstance(); + await prefs.remove('alarm_epoch'); + } catch (e) { + _log('[alarm] clearing the persisted epoch failed: $e'); + } + }()); + } + Future _notifyAlarmFired() async { try { await NotificationCenter.instance.emit(NotificationEvent( @@ -2283,15 +2416,26 @@ class AppState extends ChangeNotifier { _setBusy(true); lastError = null; _keepAlive = true; - // Android: start the Edge Tracking foreground service so the live connection keeps - // draining while backgrounded (Android kills background processes otherwise). - EdgeTracking.start(); - // iOS: arm CoreBluetooth restoration so the band can relaunch us when terminated. - // The foreground guard stops a wake from fighting this live session for the band. - IosBleRestore.foregroundActive = true; - IosBleRestore.arm(paired!.remoteId); - _log('===== SESSION START ===== raw=${dbCounts['raw']}'); try { + // INSIDE the guard, and no `paired!`. This block used to sit BETWEEN + // _setBusy(true) and the try, force-unwrapping `paired`. The resume path + // above awaits (setOwnsBand / disconnect), so the user can tap Unpair in + // that window — `paired!` then threw straight past the finally and `busy` + // stayed true for the rest of the process, silently no-opping every + // openSession()/syncNow() ("Sync now" dead until restart). + final band = paired; + if (band == null) { + _log('Session start aborted — band was unpaired mid-resume.'); + return; + } + // Android: start the Edge Tracking foreground service so the live connection keeps + // draining while backgrounded (Android kills background processes otherwise). + EdgeTracking.start(); + // iOS: arm CoreBluetooth restoration so the band can relaunch us when terminated. + // The foreground guard stops a wake from fighting this live session for the band. + IosBleRestore.foregroundActive = true; + IosBleRestore.arm(band.remoteId); + _log('===== SESSION START ===== raw=${dbCounts['raw']}'); await _ensureForegroundLease(); // connect() now subscribes → SET_CLOCK → INIT, so the historical offload is // ALREADY streaming the moment this returns. @@ -2300,7 +2444,7 @@ class AppState extends ChangeNotifier { // config) and live-stream toggles ride the same link as the historical // burst. The per-revision packet accounting counts data-role frames only, // so these command exchanges don't perturb the burst packet counts. - if (!await engine.connectToRemoteId(paired!.remoteId)) { + if (!await engine.connectToRemoteId(band.remoteId)) { lastError = 'Could not reach your band. Is it nearby and free ' '(official WHOOP app force-quit)?'; @@ -2860,23 +3004,38 @@ class AppState extends ChangeNotifier { /// Begin a calibration walk: turn on the live IMU stream and count from zero. Future startStepCalibration() async { if (!isConnected) throw Exception('Connect to your strap first'); + // LATCH SAFELY. `_stepCalActive` is set true BEFORE the stream arming + // below, and the arming can throw (the link dropping mid-write propagates + // straight out to the UI). With no try/finally the latch stuck true for the + // rest of the process — the only reset is _endStepCalStreams(), reachable + // solely from finish/cancel, which the user never gets to because the walk + // never started. A stuck latch pins [_hasLiveConsumer] true, so + // [_maybeDowngradeLiveForBackground] never downgrades and the 100 Hz raw + // flood keeps streaming while backgrounded — exactly the R24-offload + // starvation the downgrade exists to prevent. _stepCalActive = true; - // OWNERSHIP: same rule as the spot check — only claim "we enabled it" when - // live was actually OFF, so ending the walk can never turn off streams the - // open session still expects on. If the background downgrade left live in - // HR-only, upgrade to full (the walk needs the 100 Hz IMU stream) without - // taking ownership. - // - // retryFullLiveStreams (not enableLiveStreams): the walk NEEDS the 100 Hz - // IMU stream, and the sticky standard-HR fallback silently vetoes it — - // every calibration after a fallback trip counted 0 steps forever. An - // explicit user-initiated walk is exactly the moment to give the full - // flood another chance; the detectors re-trip if the radio can't cope. - if (!engine.liveEnabled) { - await engine.retryFullLiveStreams(); - _stepCalEnabledStreams = true; - } else if (engine.liveHrOnly || device.standardHrFallback) { - await engine.retryFullLiveStreams(); + var armed = false; + try { + // OWNERSHIP: same rule as the spot check — only claim "we enabled it" + // when live was actually OFF, so ending the walk can never turn off + // streams the open session still expects on. If the background downgrade + // left live in HR-only, upgrade to full (the walk needs the 100 Hz IMU + // stream) without taking ownership. + // + // retryFullLiveStreams (not enableLiveStreams): the walk NEEDS the 100 Hz + // IMU stream, and the sticky standard-HR fallback silently vetoes it — + // every calibration after a fallback trip counted 0 steps forever. An + // explicit user-initiated walk is exactly the moment to give the full + // flood another chance; the detectors re-trip if the radio can't cope. + if (!engine.liveEnabled) { + await engine.retryFullLiveStreams(); + _stepCalEnabledStreams = true; + } else if (engine.liveHrOnly || device.standardHrFallback) { + await engine.retryFullLiveStreams(); + } + armed = true; + } finally { + if (!armed) _stepCalActive = false; } _resetLivePedometer(); // count this walk from 0 notifyListeners(); @@ -3011,6 +3170,8 @@ class AppState extends ChangeNotifier { 'created_at': start.millisecondsSinceEpoch, }), ); + // Never leak a previous periodic tick by overwriting the reference. + _workoutTimer?.cancel(); _workoutTimer = Timer.periodic( const Duration(seconds: 1), (_) => _tickWorkout(), @@ -3133,6 +3294,16 @@ class AppState extends ChangeNotifier { // just-started activeWorkout and leak its timer. if (activeWorkout != null) return; final rows = await LocalDb.liveSessions(); + // RE-CHECK AFTER THE AWAIT. This is kicked unawaited from _init(), one + // line before `initialized = true` makes the shell interactive — so the + // user can tap "Start workout" INSIDE this DB round-trip. The pre-await + // guard alone let us then overwrite a genuinely live `activeWorkout` with + // the stale row AND assign a second `_workoutTimer` over the live one: + // the first timer became unreachable, was never cancelled, and kept + // running _tickWorkout at 2 Hz for the rest of the session — double + // counting calories/strain/zone-seconds against a workout the user never + // started. + if (activeWorkout != null) return; if (rows.isEmpty) return; final nowMs = DateTime.now().millisecondsSinceEpoch; var resumed = false; @@ -3157,6 +3328,8 @@ class AppState extends ChangeNotifier { // snapshot: steps count from zero going forward, same as // calories/strain/zone-minutes already (honestly) do here. _workoutRawBase = _liveRaw; + // Never overwrite a live timer reference without cancelling it. + _workoutTimer?.cancel(); _workoutTimer = Timer.periodic( const Duration(seconds: 1), (_) => _tickWorkout(), diff --git a/lib/sync/headless_boot.dart b/lib/sync/headless_boot.dart index e0f93494..98f174ea 100644 --- a/lib/sync/headless_boot.dart +++ b/lib/sync/headless_boot.dart @@ -17,16 +17,50 @@ // IosBleRestore (CB state restoration wake). iOS does NOT have an Activity concept, // so this Android-only path is guarded by Platform.isAndroid. +import 'dart:async'; import 'dart:io'; import 'package:flutter/widgets.dart'; import 'android_boot_signal.dart'; import 'band_ownership.dart'; +import 'headless_gate.dart'; import '../sync/background_sync.dart'; import '../sync/edge_tracking.dart'; import '../sync/paired_device.dart'; +/// The [HeadlessSyncGate] owner name for the Android post-boot wake. +const String kBootWakeGateOwner = 'android_boot_wake'; + +/// Run the boot drain THROUGH the process-wide headless gate. +/// +/// The stated invariant is that EVERY headless entry point serialises through +/// [HeadlessSyncGate] (skip, don't queue). The three iOS entry points did; this +/// Android boot path called `runHeadlessSync` directly and fire-and-forget, so +/// `HeadlessSyncGate.busy` read false for the entire duration of a boot drain +/// and any iOS-style wake landing in the same process would have run +/// concurrently with it. +/// +/// Returns the runner's result, or null when the gate was busy and this cycle +/// was skipped — in which case the band lease is released, since nothing will +/// use it. [runner] is injectable for tests. +Future runBootSyncThroughGate( + BandLease lease, { + Future Function(BandLease lease)? runner, +}) async { + final run = runner ?? ((l) => runHeadlessSync(lease: l)); + final result = await HeadlessSyncGate.tryRun( + kBootWakeGateOwner, + () => run(lease), + ); + if (result == null) { + // Skipped — release the lease we acquired up front, or the band stays + // owned by a headless run that never happened. + BandOwnership.release(lease); + } + return result; +} + /// Guards headless boot so we only run once per process lifetime. Prevents /// re-entry if main() is somehow invoked twice on the same engine. bool _booted = false; @@ -76,7 +110,16 @@ Future maybeHeadlessBoot() async { // disconnect). This catches up the offline backlog accumulated while the phone // was powered off. Errors are swallowed inside runHeadlessSync. debugPrint('[headless-boot] starting headless sync for ${paired.remoteId}'); - runHeadlessSync(lease: lease).then((_) { - debugPrint('[headless-boot] headless sync complete'); - }); + // Fire-and-forget (see the doc above) but SERIALISED through the shared + // headless gate, so `HeadlessSyncGate.busy` is true for the whole boot drain. + unawaited( + runBootSyncThroughGate(lease).then((ran) { + debugPrint( + ran == null + ? '[headless-boot] boot wake skipped — another headless sync ' + 'holds the gate; lease released.' + : '[headless-boot] headless sync complete', + ); + }), + ); } diff --git a/lib/sync/sync_policy.dart b/lib/sync/sync_policy.dart index 39f6540a..8ebec03b 100644 --- a/lib/sync/sync_policy.dart +++ b/lib/sync/sync_policy.dart @@ -115,6 +115,21 @@ const int kRecTsGridSeconds = 300; // 5-minute grid int snapToGrid(int ts, [int grid = kRecTsGridSeconds]) => (ts ~/ grid) * grid; class ClockPolicy { + /// Whether a GET_CLOCK `clock_epoch` read may be trusted enough to become + /// this session's strap-RTC ↔ wall correlation ([ClockRef]). + /// + /// `range_newest` (GET_DATA_RANGE) is already guarded by [isCorruptFutureRtc] + /// before it is allowed to tighten the per-record plausibility window; + /// `clock_epoch` had NO such gate, even though it feeds something strictly + /// more dangerous. A corrupt far-future RTC read yields a large NEGATIVE + /// [ClockRef.driftSec], and [AlarmPayloads.toStrapFrame] arms the wake alarm + /// at `when - driftSec` — i.e. years in the future, where it silently never + /// fires. Reject the read instead: with no correlation the alarm falls back + /// to the raw wall epoch, and the bounded SET_CLOCK retry budget is not + /// spent chasing a value that was never real. + static bool acceptsClockRead(int deviceClock, int wallNow) => + !isCorruptFutureRtc(deviceClock, wallNow); + /// Re-issue SET_CLOCK if the strap clock has drifted > 1 day or is frozen in /// the pre-2023 past (an unset RTC). static bool shouldSetClock(int deviceClock, int wallNow) { diff --git a/lib/telemetry/telemetry_service.dart b/lib/telemetry/telemetry_service.dart index 7ad73c0a..02b86ae3 100644 --- a/lib/telemetry/telemetry_service.dart +++ b/lib/telemetry/telemetry_service.dart @@ -45,6 +45,38 @@ class TelemetryService { bool get enabled => _enabled; set enabled(bool value) { _enabled = value; + _consentResolved = true; + _applyFirebaseCollection(value); + } + + // ── consent-gated Firebase collection ────────────────────────────────────── + // + // The Firebase SDKs auto-collect from process start unless told otherwise at + // the PLATFORM level, which is why ios/Runner/Info.plist and + // android/app/src/main/AndroidManifest.xml both ship + // _collection_enabled = false. Those flags + // are the real guarantee (they land before any Dart runs); everything here is + // the seam that turns collection back ON, and only after the user's stored + // consent has actually been read. + + /// False until the user's persisted telemetry consent has been LOADED — not + /// merely defaulted. Nothing is transmitted and no Firebase SDK is enabled + /// while this is false, no matter what else happens during startup. + bool _consentResolved = false; + bool get consentResolved => _consentResolved; + + /// Test seam: replaces the real `set*CollectionEnabled` fan-out (the Firebase + /// SDKs can't be initialized in a unit test). Receives the value that would + /// have been pushed to Crashlytics/Performance/Analytics. + @visibleForTesting + static void Function(bool enabled)? debugCollectionSink; + + void _applyFirebaseCollection(bool value) { + final sink = debugCollectionSink; + if (sink != null) { + sink(value); + return; + } try { if (Firebase.apps.isNotEmpty) { FirebaseCrashlytics.instance.setCrashlyticsCollectionEnabled(value); @@ -54,6 +86,34 @@ class TelemetryService { } catch (_) {} } + /// Apply the user's AFFIRMATIVELY LOADED telemetry consent. Preferred over + /// assigning [enabled] directly because it reads as what it is: the single + /// point where collection may be switched on. + void applyConsent(bool consent) => enabled = consent; + + /// Belt-and-braces: force every Firebase SDK's collection OFF at startup, + /// before consent is known. The platform flags already do this, but a build + /// with a stale plist/manifest (or a future SDK that defaults differently) + /// must still not collect. No-op once consent has been resolved, so a hot + /// restart / re-entry can't silently revoke an opt-in. + /// + /// Called from main() right after Firebase.initializeApp. Safe when Firebase + /// is absent entirely (no google-services.json / GoogleService-Info.plist) — + /// Firebase is OPTIONAL and the app must run without it. + void enforceCollectionOffUntilConsent() { + if (_consentResolved) return; + _enabled = false; + _applyFirebaseCollection(false); + } + + /// Test seam: return the singleton to its fresh-install state (no consent + /// read yet, nothing transmitted). + @visibleForTesting + void debugResetConsent() { + _consentResolved = false; + _enabled = false; + } + /// Anchors + version stamped onto each batch (AppState sets these on load). String? deviceId; String? userId; diff --git a/lib/ui/ai/ai_breakdown_screen.dart b/lib/ui/ai/ai_breakdown_screen.dart index 8e5b6b7e..419e3cef 100644 --- a/lib/ui/ai/ai_breakdown_screen.dart +++ b/lib/ui/ai/ai_breakdown_screen.dart @@ -41,6 +41,12 @@ class _AiBreakdownScreenState extends State { _Phase _phase = _Phase.busy; Briefing? _brief; String _error = ''; + // Regenerate is a plain button on a screen whose generate can take many + // seconds. Two taps = two in-flight calls; whichever settles LAST wins, so a + // first call that errors after the second succeeds flips ready → error and + // throws away the briefing already on screen (and the reverse re-renders the + // older one as current). + bool _generating = false; @override void initState() { @@ -71,11 +77,13 @@ class _AiBreakdownScreenState extends State { } Future _generate() async { + if (_generating) return; // a generation is already in flight final engine = _engine(); if (engine == null || !engine.configured) { setState(() => _phase = _Phase.noKey); return; } + _generating = true; setState(() => _phase = _Phase.busy); try { final b = await engine.generate(widget.period); @@ -94,6 +102,8 @@ class _AiBreakdownScreenState extends State { _phase = _Phase.error; _error = e.toString().replaceFirst('CoachException: ', ''); }); + } finally { + _generating = false; } } diff --git a/lib/ui/ai/ai_settings_screen.dart b/lib/ui/ai/ai_settings_screen.dart index a48e91bd..83414c83 100644 --- a/lib/ui/ai/ai_settings_screen.dart +++ b/lib/ui/ai/ai_settings_screen.dart @@ -20,6 +20,10 @@ import '../design/design.dart'; class AiSettingsScreen extends StatefulWidget { const AiSettingsScreen({super.key}); + /// Test seam for the time picker — production uses [showTimePicker]. + @visibleForTesting + static Future Function(BuildContext, TimeOfDay)? pickerOverride; + @override State createState() => _AiSettingsScreenState(); } @@ -44,6 +48,10 @@ class _AiSettingsScreenState extends State { } Future _update(AiPrefs next) async { + // Reached from _pickTime AFTER an awaited showTimePicker, so by the time + // this runs the screen may already be gone — the setState below was the + // only unguarded one on the path. + if (!mounted) return; setState(() => _p = next); await next.save(); if (mounted) await context.read().refreshAiReminders(); @@ -53,12 +61,16 @@ class _AiSettingsScreenState extends State { TimeOfDay(hour: (min ~/ 60) % 24, minute: min % 60).format(context); Future _pickTime(int current, ValueChanged apply) async { - final picked = await showTimePicker( - context: context, - initialTime: - TimeOfDay(hour: (current ~/ 60) % 24, minute: current % 60), - ); - if (picked == null) return; + final initial = TimeOfDay(hour: (current ~/ 60) % 24, minute: current % 60); + final override = AiSettingsScreen.pickerOverride; + // Build the future synchronously — no BuildContext across an async gap. + final Future pending = override != null + ? override(context, initial) + : showTimePicker(context: context, initialTime: initial); + final picked = await pending; + // The dialog's future resolves after its exit transition, which is long + // enough for the screen underneath to have been popped. + if (!mounted || picked == null) return; apply(picked.hour * 60 + picked.minute); } diff --git a/lib/ui/coach/ai_coach_screen.dart b/lib/ui/coach/ai_coach_screen.dart index f65a6232..ee4a1a7e 100644 --- a/lib/ui/coach/ai_coach_screen.dart +++ b/lib/ui/coach/ai_coach_screen.dart @@ -15,6 +15,7 @@ import '../../coach/coach_engine.dart'; import '../../state/app_state.dart'; import '../../theme/theme_switcher.dart'; import '../design/design.dart'; +import '../widgets/async_guards.dart'; import 'coach_chart.dart'; import 'coach_render.dart'; import 'coach_settings_screen.dart'; @@ -81,17 +82,21 @@ class _AiCoachScreenState extends State { _input.clear(); setState(() => _busy = true); try { + // engine.send can block for up to 120 s. Every callback below, and the + // catch, can therefore land after the user has popped the screen — the + // finally already knew this, the rest didn't. await engine.send( t, onItem: (it) { - setState(() => _items.add(it)); + if (!mounted) return; + setStateIfMounted(() => _items.add(it)); _scrollDown(); }, - onStatus: (s) => setState(() => _status = s), + onStatus: (s) => setStateIfMounted(() => _status = s), confirm: _confirm, ); } catch (e) { - setState(() => _items.add(CoachItem.error( + setStateIfMounted(() => _items.add(CoachItem.error( e is CoachException ? e.message : 'Something went wrong: $e'))); } finally { if (mounted) { @@ -246,8 +251,9 @@ class _AiCoachScreenState extends State { } void _scrollDown() { + if (!mounted) return; // _scroll is disposed with the State WidgetsBinding.instance.addPostFrameCallback((_) { - if (_scroll.hasClients) { + if (mounted && _scroll.hasClients) { _scroll.animateTo(_scroll.position.maxScrollExtent, duration: const Duration(milliseconds: 250), curve: Curves.easeOut); } diff --git a/lib/ui/coach/coach_chart.dart b/lib/ui/coach/coach_chart.dart index 94fc8986..4b2b862a 100644 --- a/lib/ui/coach/coach_chart.dart +++ b/lib/ui/coach/coach_chart.dart @@ -41,7 +41,10 @@ class CoachChart extends StatelessWidget { const SizedBox(height: Sp.x4), if (spec.type == 'bar' && single) LabeledBars( - values: spec.series.first.values.map((v) => v ?? 0).toList(), + // Nulls stay null. `?? 0` here drew a real bar for every point the + // coach's own query returned no value for — the same absent-as-zero + // fabrication as the trend boards, in a chart the model narrates. + values: spec.series.first.values.toList(), labels: _fitLabels(spec.xLabels, spec.series.first.values.length), color: DomainAccent.heart, height: 160, diff --git a/lib/ui/design/recap_card.dart b/lib/ui/design/recap_card.dart index 9c58a046..3f4e5555 100644 --- a/lib/ui/design/recap_card.dart +++ b/lib/ui/design/recap_card.dart @@ -51,7 +51,11 @@ class RecapCard extends StatelessWidget { @override Widget build(BuildContext context) { final a = accent ?? AppColors.accent; - final barsClean = bars?.whereType().toList() ?? const []; + // Nulls go THROUGH to MiniBars, which keeps their slots empty. Stripping + // them here compacted the strip: a week missing Wednesday drew six bars + // with Thu–Sun shifted a day left, silently re-dating every value after + // the gap. + final barStrip = bars ?? const []; return BentoTile( tone: BentoTone.paper, accent: a, @@ -104,11 +108,14 @@ class RecapCard extends StatelessWidget { size: BigStatSize.md, ), ), - if (barsClean.length >= 2) ...[ + // Gate on how many slots actually CARRY a value (a strip of + // one real bar plus six gaps isn't a trend), but draw the + // full-length strip so the bars keep their days. + if (barStrip.whereType().length >= 2) ...[ const SizedBox(width: Sp.x3), SizedBox( width: 96, - child: MiniBars(barsClean, color: a, height: 34), + child: MiniBars(barStrip, color: a, height: 34), ), ], ], diff --git a/lib/ui/kit/charts.dart b/lib/ui/kit/charts.dart index d56976c0..67c36040 100644 --- a/lib/ui/kit/charts.dart +++ b/lib/ui/kit/charts.dart @@ -177,7 +177,11 @@ class BaselineProgress extends StatelessWidget { /// Tiny sparkline bars (for inside cards). Values normalized to their own max. class MiniBars extends StatelessWidget { - final List values; + /// Bar values. A NULL entry is a documented gap (nothing was measured for + /// that slot) and renders as empty space — it is never compacted away (which + /// would shift every later bar left onto the wrong day) and never drawn as a + /// bar (which would read as a measured zero). + final List values; final Color? color; final double height; final double gap; @@ -192,7 +196,9 @@ class MiniBars extends StatelessWidget { Widget build(BuildContext context) { final color = this.color ?? AppColors.coral; if (values.isEmpty) return SizedBox(height: height); - final maxV = values.reduce(math.max); + final present = values.whereType(); + if (present.isEmpty) return SizedBox(height: height); + final maxV = present.reduce(math.max); return SizedBox( height: height, child: Row( @@ -200,23 +206,32 @@ class MiniBars extends StatelessWidget { children: [ for (int i = 0; i < values.length; i++) ...[ Expanded( - child: TweenAnimationBuilder( - duration: Motion.med, - curve: Motion.curve, - tween: Tween(begin: 0, end: maxV == 0 ? 0 : (values[i] / maxV)), - builder: (_, v, _) => Align( - alignment: Alignment.bottomCenter, - child: Container( - height: math.max(3, v * height), - decoration: BoxDecoration( - color: color.withValues( - alpha: 0.4 + 0.6 * (maxV == 0 ? 0 : values[i] / maxV), + // A gap keeps its slot (so the bars stay aligned to their days) + // but draws nothing at all. + child: values[i] == null + ? const SizedBox.shrink() + : TweenAnimationBuilder( + duration: Motion.med, + curve: Motion.curve, + tween: Tween( + begin: 0, + end: maxV == 0 ? 0 : (values[i]! / maxV), + ), + builder: (_, v, _) => Align( + alignment: Alignment.bottomCenter, + child: Container( + height: math.max(3, v * height), + decoration: BoxDecoration( + color: color.withValues( + alpha: + 0.4 + + 0.6 * (maxV == 0 ? 0 : values[i]! / maxV), + ), + borderRadius: BorderRadius.circular(R.pill), + ), + ), ), - borderRadius: BorderRadius.circular(R.pill), ), - ), - ), - ), ), if (i != values.length - 1) SizedBox(width: gap), ], @@ -230,7 +245,11 @@ class MiniBars extends StatelessWidget { /// Shows the numeric value above each bar (set [showValues] false to hide). /// [onTapBar] makes a bar tappable (drill-down in the Metric Explorer). class LabeledBars extends StatelessWidget { - final List values; + /// Bar values. A NULL entry is an explicit GAP — no measurement exists for + /// that period — and renders as an em-dash with NO bar. It must never be + /// coerced to 0.0: the bar floor (`heightFactor 0.02`) would otherwise draw + /// a real, accent-coloured, tappable bar for a day the strap wasn't worn. + final List values; final List labels; final Color? color; final double height; @@ -250,7 +269,10 @@ class LabeledBars extends StatelessWidget { this.onTapBar, }); - String _fmt(double v) { + /// Label above a bar. Null = absent → the honest em-dash (NOT '' and NOT + /// '0': a missing measurement must never be printed as a number). + String _fmt(double? v) { + if (v == null) return '—'; if (valueFmt != null) return valueFmt!(v); // tidy default: integers when whole, else one decimal; blank for exact 0. if (v == 0) return ''; @@ -260,7 +282,10 @@ class LabeledBars extends StatelessWidget { @override Widget build(BuildContext context) { final color = this.color ?? AppColors.coral; - final maxV = values.isEmpty ? 1.0 : math.max(1.0, values.reduce(math.max)); + final present = values.whereType(); + final maxV = present.isEmpty + ? 1.0 + : math.max(1.0, present.reduce(math.max)); return SizedBox( height: height, child: Row( @@ -281,7 +306,9 @@ class LabeledBars extends StatelessWidget { _fmt(values[i]), style: AppText.caption.copyWith( fontWeight: FontWeight.w600, - color: (highlight == null || highlight == i) + color: values[i] == null + ? AppColors.inkMuted + : (highlight == null || highlight == i) ? AppColors.ink : AppColors.inkMuted, ), @@ -290,23 +317,31 @@ class LabeledBars extends StatelessWidget { ), if (showValues) const SizedBox(height: 2), Expanded( - child: TweenAnimationBuilder( - duration: Motion.med, - curve: Motion.emphatic, - tween: Tween(begin: 0, end: values[i] / maxV), - builder: (_, v, _) => FractionallySizedBox( - heightFactor: v.clamp(0.02, 1.0), - alignment: Alignment.bottomCenter, - child: Container( - decoration: BoxDecoration( - color: (highlight == null || highlight == i) - ? color - : color.withValues(alpha: 0.28), - borderRadius: BorderRadius.circular(R.pill), + // Absent → draw NOTHING. The 0.02 floor below exists so + // a genuine tiny value stays visible; applying it to a + // missing day fabricates a bar out of no measurement. + child: values[i] == null + ? const SizedBox.shrink() + : TweenAnimationBuilder( + duration: Motion.med, + curve: Motion.emphatic, + tween: Tween(begin: 0, end: values[i]! / maxV), + builder: (_, v, _) => FractionallySizedBox( + heightFactor: v.clamp(0.02, 1.0), + alignment: Alignment.bottomCenter, + child: Container( + decoration: BoxDecoration( + color: + (highlight == null || highlight == i) + ? color + : color.withValues(alpha: 0.28), + borderRadius: BorderRadius.circular( + R.pill, + ), + ), + ), + ), ), - ), - ), - ), ), const SizedBox(height: Sp.x2), Text( @@ -933,53 +968,60 @@ class _HrReplayOverlayState extends State @override Widget build(BuildContext context) { - final playing = _c.isAnimating; + // The AnimatedBuilder must sit OUTSIDE the `0 < t < 1` gate. It used to be + // inside it, so the only thing that could ever rebuild past the gate was + // the setState in _toggle — which runs while _c.value is still 0. The gate + // was therefore false on every rebuild and the replay dot never mounted. return Positioned.fill( - child: Stack( - children: [ - if (_c.value > 0 && _c.value < 1) - Positioned.fill( - child: IgnorePointer( - child: AnimatedBuilder( - animation: _c, - builder: (context, _) => CustomPaint( - painter: _ReplayDotPainter( - points: widget.points, - t: _c.value, - loX: widget.loX, - hiX: widget.hiX, - loY: widget.loY, - hiY: widget.hiY, - leftPad: widget.leftPad, - topInset: widget.topInset, - plotH: widget.chartHeight - widget.bottomPad, - color: widget.color, + child: AnimatedBuilder( + animation: _c, + builder: (context, _) { + final t = _c.value; + final playing = _c.isAnimating; + return Stack( + children: [ + if (t > 0 && t < 1) + Positioned.fill( + child: IgnorePointer( + child: CustomPaint( + painter: _ReplayDotPainter( + points: widget.points, + t: t, + loX: widget.loX, + hiX: widget.hiX, + loY: widget.loY, + hiY: widget.hiY, + leftPad: widget.leftPad, + topInset: widget.topInset, + plotH: widget.chartHeight - widget.bottomPad, + color: widget.color, + ), ), ), ), - ), - ), - Positioned( - right: 0, - top: widget.topInset, - child: Material( - color: widget.color, - shape: const CircleBorder(), - child: InkWell( - customBorder: const CircleBorder(), - onTap: _toggle, - child: Padding( - padding: const EdgeInsets.all(7), - child: Icon( - playing ? Icons.pause : Icons.play_arrow, - size: 16, - color: Colors.white, + Positioned( + right: 0, + top: widget.topInset, + child: Material( + color: widget.color, + shape: const CircleBorder(), + child: InkWell( + customBorder: const CircleBorder(), + onTap: _toggle, + child: Padding( + padding: const EdgeInsets.all(7), + child: Icon( + playing ? Icons.pause : Icons.play_arrow, + size: 16, + color: Colors.white, + ), + ), ), ), ), - ), - ), - ], + ], + ); + }, ), ); } diff --git a/lib/ui/kit/kit.dart b/lib/ui/kit/kit.dart index d41e0704..4b832eb8 100644 --- a/lib/ui/kit/kit.dart +++ b/lib/ui/kit/kit.dart @@ -359,6 +359,11 @@ class SegToggle extends StatelessWidget { } /// A ▲ +3.2% / ▼ −5% colored delta chip. Pass null to hide. +/// +/// [pct] is a PERCENTAGE and is rendered with a literal '%'. Never hand it an +/// absolute difference in the metric's own unit (e.g. `/trend`'s +/// `delta_vs_prev`, which is `avg - prevAvg`) — "+20 min of sleep" would print +/// as "▲ 20.0%". Use [BaselineDeltaChip] for absolute deltas. class DeltaChip extends StatelessWidget { final num? pct; final String suffix; @@ -414,12 +419,19 @@ class BaselineDeltaChip extends StatelessWidget { final String unit; // e.g. 'bpm', 'ms', '' final bool goodIsUp; // RHR: down is good → false final bool showVsNormal; + + /// What the delta is measured AGAINST, when it isn't the personal baseline + /// (e.g. 'vs prev' on a trend board comparing two adjacent windows). Ignored + /// while [showVsNormal] is true. Naming the comparand is not decoration — + /// an unlabelled signed number invites the reader to guess a unit. + final String? vsLabel; const BaselineDeltaChip( this.delta, { super.key, this.unit = '', this.goodIsUp = true, this.showVsNormal = true, + this.vsLabel, }); @override Widget build(BuildContext context) { @@ -443,7 +455,8 @@ class BaselineDeltaChip extends StatelessWidget { borderRadius: BorderRadius.circular(R.pill), ), child: Text( - '$sign$shownMag${unit.isNotEmpty ? ' $unit' : ''}${showVsNormal ? ' vs normal' : ''}', + '$sign$shownMag${unit.isNotEmpty ? ' $unit' : ''}' + '${showVsNormal ? ' vs normal' : (vsLabel != null ? ' $vsLabel' : '')}', style: AppText.caption.copyWith(color: c, fontWeight: FontWeight.w700), ), ); diff --git a/lib/ui/screens/detail_cards.dart b/lib/ui/screens/detail_cards.dart index a927897a..b8eaed78 100644 --- a/lib/ui/screens/detail_cards.dart +++ b/lib/ui/screens/detail_cards.dart @@ -13,6 +13,7 @@ import '../../models/metric.dart' import '../../data/day_label.dart'; import '../../state/app_state.dart'; import '../design/design.dart'; +import '../widgets/async_guards.dart' show LatestRequestGate; import 'metric_row.dart'; import 'trend_screen.dart'; @@ -98,6 +99,10 @@ class _FetchState extends State<_Fetch> { AppState? _app; VoidCallback? _insightsListener; int _lastInsightsRevision = -1; + // ...which then introduced its own race: two revisions in quick succession + // (rollup, then derive) leave two loads in flight and the later-STARTING one + // can complete first, letting the stale payload land last and win. + final LatestRequestGate _gate = LatestRequestGate(); @override void initState() { @@ -129,16 +134,17 @@ class _FetchState extends State<_Fetch> { Future _go() async { final api = context.read().repo; if (api == null) return; + final token = _gate.begin(); try { final d = await widget.load(api); - if (mounted) { - setState(() { - _d = d; - _loading = false; - }); - } + if (!mounted || !_gate.isCurrent(token)) return; // superseded + setState(() { + _d = d; + _loading = false; + }); } catch (_) { - if (mounted) setState(() => _loading = false); + if (!mounted || !_gate.isCurrent(token)) return; + setState(() => _loading = false); } } @@ -1480,6 +1486,22 @@ Widget _legendPill(String label, Color color) { ); } + // A grade needs at least ONE computed dip metric. These used to default to + // 0, so a night with trusted coverage but nothing computed fell all the way + // through to "Quiet — No meaningful overnight oxygen dips were detected" — + // an affirmative all-clear derived from three absent numbers. Currently + // unreachable only because `spo2.disabled` short-circuits upstream; it goes + // live the moment SpO₂ decoding is re-enabled. + if (odiPerHour == null && maxDipPct == null && burdenPct == null) { + return ( + label: 'Not graded', + color: AppColors.inkSoft, + reason: + 'The overnight dip metrics weren’t computed for this night, so ' + 'there is nothing to grade — this is not an all-clear.', + ); + } + final odi = odiPerHour ?? 0; final maxDip = maxDipPct ?? 0; final burden = burdenPct ?? 0; @@ -1539,6 +1561,7 @@ class _OxygenRecentStrip extends StatefulWidget { class _OxygenRecentStripState extends State<_OxygenRecentStrip> { Map? _trend; bool _loading = true; + final LatestRequestGate _gate = LatestRequestGate(); @override void initState() { @@ -1549,19 +1572,20 @@ class _OxygenRecentStripState extends State<_OxygenRecentStrip> { Future _load() async { final api = context.read().repo; if (api == null) return; + final token = _gate.begin(); try { final trend = await api.getTrend( 'spo2', scale: 'week', anchor: widget.date, ); - if (!mounted) return; + if (!mounted || !_gate.isCurrent(token)) return; // superseded setState(() { _trend = trend; _loading = false; }); } catch (_) { - if (!mounted) return; + if (!mounted || !_gate.isCurrent(token)) return; setState(() => _loading = false); } } @@ -1601,7 +1625,7 @@ class _OxygenRecentStripState extends State<_OxygenRecentStrip> { return ( label: 'spike', color: AppColors.warn, - reason: 'Tonight stands well above your recent oxygen-dip pattern.', + reason: 'Tonight stands well above your recent oxygen-index pattern.', ); } if (drift >= 1.0) { @@ -1664,7 +1688,18 @@ class _OxygenRecentStripState extends State<_OxygenRecentStrip> { mainAxisSize: MainAxisSize.min, children: [ Tag(pattern.label, color: pattern.color), - InfoDot(title: 'Last 7 nights', body: pattern.reason), + InfoDot( + title: 'Last 7 nights', + body: pattern.reason, + // The `spo2` SERIES is not the ODI shown in the hero above: + // the on-device pipeline writes null into it, so its only + // writer is the cloud importer's vendor saturation index. + // It must not be captioned "/h" as if it were a dip rate. + methodNote: + 'These bars are the imported vendor oxygen index, not ' + 'the dip rate in the hero above and not an absolute ' + 'SpO₂. Nights with no imported value are left blank.', + ), ], ), ), @@ -1684,9 +1719,9 @@ class _OxygenRecentStripState extends State<_OxygenRecentStrip> { spacing: Sp.x2, runSpacing: Sp.x1, children: [ - StatusChip('Latest ${latest.toStringAsFixed(1)}/h', + StatusChip('Latest ${latest.toStringAsFixed(1)}', tone: ChipTone.accent), - StatusChip('Avg ${avg.toStringAsFixed(1)}/h'), + StatusChip('Avg ${avg.toStringAsFixed(1)}'), StatusChip( '${(latest - avg >= 0 ? '+' : '')}${(latest - avg).toStringAsFixed(1)} vs avg', ), @@ -1896,6 +1931,20 @@ class WearDayContent extends StatelessWidget { num? _n(Object? v) => v is num ? v : null; + /// How many separate on-wrist stretches the day had. + /// + /// `getDayWear` emits `segments` as a LIST of on/off segment maps (the + /// engine builds it as a list). The old `(_n(d['segments']) ?? 0).toInt()` + /// therefore always saw a List, always fell through to `?? 0`, and printed + /// "Wear stretches 0" on every day on every device — including a 24 h-worn + /// day with three stretches. An EMPTY list means no wear block was stored at + /// all (a worn day necessarily has ≥1 stretch), which is absence, not zero. + static int? _segmentCount(Object? v) { + if (v is List) return v.isEmpty ? null : v.length; + if (v is num) return v.toInt(); // legacy/precomputed count + return null; + } + // unix seconds → local "h:mm AM/PM" String _clock(num? ts) { if (ts == null) return '—'; @@ -1909,16 +1958,32 @@ class WearDayContent extends StatelessWidget { Widget build(BuildContext context) { final d = data; final accent = AppColors.coralDeep; - final worn = (_n(d['worn_min']) ?? 0).toInt(); - final cov = (_n(d['coverage_pct']) ?? 0).toInt(); + // NULLABLE on purpose. "The strap was off all day" and "this day has no + // wear measurement" are different claims, and only one of them is ours to + // make. A cloud-imported day carries no wear/coverage block, so the old + // `?? 0` turned silence into the affirmative "No wrist contact was + // recorded" — on a day whose own Week bars showed hours of wear. + final worn = _n(d['worn_min'])?.toInt(); + final cov = _n(d['coverage_pct'])?.toInt(); final hourly = ((d['hourly'] as List?) ?? const []) - .map((e) => (e as num).toDouble()) + .whereType() + .map((e) => e.toDouble()) .toList(); final firstOn = _n(d['first_on']); final lastOn = _n(d['last_on']); - final segments = (_n(d['segments']) ?? 0).toInt(); - final longestOff = (_n(d['longest_off_min']) ?? 0).toInt(); + final segments = _segmentCount(d['segments']); + final longestOff = _n(d['longest_off_min'])?.toInt(); + if (d.isEmpty || worn == null) { + return const _QuietState( + icon: OsIcon.wear, + title: 'Wear time wasn’t recorded', + message: + 'This day has no wear measurement stored — imported days don’t ' + 'carry one. That is not the same as the strap being off, so ' + 'nothing is claimed either way.', + ); + } if (worn == 0) { return const _QuietState( icon: OsIcon.wear, @@ -1959,17 +2024,23 @@ class WearDayContent extends StatelessWidget { Expanded( child: BigStat( value: hm(worn), - caption: '$cov% of the day', + caption: cov == null + ? 'share of day not recorded' + : '$cov% of the day', size: BigStatSize.xl, ), ), const SizedBox(width: Sp.x3), ArcGauge( - value: (cov / 100).clamp(0.0, 1.0), + // NaN → the muted empty ring; a missing coverage figure + // must not be drawn as a 0% ring. + value: cov == null + ? double.nan + : (cov / 100).clamp(0.0, 1.0), color: AppColors.coralDeep, size: 96, stroke: 10, - valueText: '$cov%', + valueText: cov == null ? '—' : '$cov%', label: 'of day', ), ], @@ -2012,6 +2083,29 @@ class WearDayContent extends StatelessWidget { ], ), ), + ] else ...[ + // The hourly array is NOT stored per day (getDayWear hard-codes + // `'hourly': const []`), so for a recent day both branches above + // used to be false and the card vanished with no explanation — + // the one outcome the honesty contract forbids: rendering nothing + // silently. Say so instead. + const SizedBox(height: Sp.x3), + ProCard( + child: Row( + children: [ + AppIcon(OsIcon.wear, size: 20, color: AppColors.inkMuted), + const SizedBox(width: Sp.x3), + Expanded( + child: Text( + 'Hour-by-hour wear isn’t stored for this day — only the ' + 'day totals above are. Nothing has been estimated to ' + 'fill the gap.', + style: AppText.caption.copyWith(color: AppColors.inkSoft), + ), + ), + ], + ), + ), ], // ── when + how continuous ──────────────────────────────────────────── @@ -2039,7 +2133,11 @@ class WearDayContent extends StatelessWidget { children: [ const TileHeader('Wear stretches'), const SizedBox(height: Sp.x2), - BigStat(value: '$segments', size: BigStatSize.md), + // Null → BigStat's honest em-dash. + BigStat( + value: segments?.toString(), + size: BigStatSize.md, + ), ], ), ), @@ -2066,8 +2164,14 @@ class WearDayContent extends StatelessWidget { children: [ const TileHeader('Longest off'), const SizedBox(height: Sp.x2), + // 'none' is a positive claim ("never removed"). Only make it + // from a MEASURED zero — a day whose bundle has coverage but + // no engine wear block has no off-time measurement at all, + // and its raw may already be pruned, so it can never get one. BigStat( - value: longestOff > 0 ? hm(longestOff) : 'none', + value: longestOff == null + ? null + : (longestOff > 0 ? hm(longestOff) : 'none'), size: BigStatSize.md, ), ], diff --git a/lib/ui/screens/metric_screen.dart b/lib/ui/screens/metric_screen.dart index 7a7a0c95..fe8d23f7 100644 --- a/lib/ui/screens/metric_screen.dart +++ b/lib/ui/screens/metric_screen.dart @@ -225,7 +225,13 @@ class _DrillLevelState extends State<_DrillLevel> { } /// Bar label for a /trend bucket at a given scale ('week' → weekday initials, -/// 'month' → W1..W5, 'quarter' → month names). Pure; shared with tests. +/// 'month' → the window's END date, 'quarter' → month names). Pure; shared +/// with tests. +/// +/// 'month' buckets are NOT calendar weeks — the repository builds four ROLLING +/// 7-day windows ending at the anchor day, so 'W1…W4' was a lie (W1 of a month +/// it may not even overlap). Labelling by the window's last day is the only +/// honest short form. String trendBarLabel(String scale, int i, Map b) { final ts = (b['t_start'] as num?)?.toInt(); if (ts == null) return b['label']?.toString() ?? ''; @@ -234,12 +240,59 @@ String trendBarLabel(String scale, int i, Map b) { case 'week': return _wd[(d.weekday - 1) % 7]; case 'month': - return 'W${i + 1}'; + final end = trendBucketEnd(b) ?? d.add(const Duration(days: 6)); + return '${_mon[end.month - 1]} ${end.day}'; default: // quarter → month return _mon[d.month - 1]; } } +/// Last day (inclusive, UTC) covered by a /trend bucket. `t_end` is exclusive. +DateTime? trendBucketEnd(Map b) { + final end = (b['t_end'] as num?)?.toInt(); + if (end == null) return null; + return DateTime.fromMillisecondsSinceEpoch((end - 86400) * 1000, isUtc: true); +} + +/// The value a /trend bucket actually carries, or NULL when the repository +/// marked the bucket absent. +/// +/// The repository writes `{'value': v ?? 0.0, 'has': v != null}` — the 0.0 is +/// pure padding and `has` is the only truth. Reading `value` alone turns every +/// unworn day into a measured zero. Payloads that predate `has` fall back to +/// value-presence. +double? trendBucketValue(Map b) { + final v = (b['value'] as num?)?.toDouble(); + if (b.containsKey('has')) return b['has'] == true ? v : null; + return v; +} + +/// Display (label, unit) for a trend series, correcting the repository's +/// nominal naming where the stored series does not contain what the name says. +/// +/// `spo2` is the only such series: the on-device 1 Hz pipeline writes +/// `scalars['spo2'] = null` and `odi_per_hour = null`, so nothing local ever +/// populates it. Its ONLY writer is the cloud importer, which stores the +/// vendor's `spo2_idx` — a relative saturation index in the 90s, not a dip +/// rate. Presenting it as "95.3 dips/h" is a category error on top of a +/// three-orders-of-magnitude wrong number. +({String label, String unit}) trendDisplay( + String metric, + String label, + String unit, +) { + if (metric == 'spo2') { + return (label: 'imported oxygen index', unit: ''); + } + return (label: label, unit: unit); +} + +/// Metrics where a DOWNWARD move is the good one (for delta-chip colouring). +bool trendGoodIsUp(String metric) => switch (metric) { + 'resting_hr' || 'stress' || 'spo2' || 'debt' => false, + _ => true, +}; + /// TrendBoard — the pure over-time hero of the rebuilt MetricScreen: one /// BentoTile with a whispered header + (i) definition, a BigStat average with /// its week-over-week delta, and the period's tappable bars underneath. @@ -269,11 +322,32 @@ class TrendBoard extends StatelessWidget { this.onTapBar, }); + /// Generic name for the window's LENGTH. Never "this week"/"this month": + /// with no explicit anchor the repository anchors on the LAST DAY WITH DATA, + /// so after five unsynced days "this week" is a fortnight ago. [_periodOf] + /// prefers the buckets' real dates; this is only the fallback. String get _period => scale == 'week' - ? 'this week' + ? '7 days' : scale == 'month' - ? 'this month' - : 'last 3 months'; + ? '4 weeks' + : '3 months'; + + /// The window the bars ACTUALLY cover, read off the buckets ('Jul 15–21'). + String _periodOf(List buckets) { + if (buckets.isEmpty) return _period; + final firstTs = ((buckets.first as Map)['t_start'] as num?)?.toInt(); + final last = trendBucketEnd(buckets.last as Map); + if (firstTs == null || last == null) return _period; + final start = DateTime.fromMillisecondsSinceEpoch( + firstTs * 1000, + isUtc: true, + ); + final s = '${_mon[start.month - 1]} ${start.day}'; + final e = start.month == last.month + ? '${last.day}' + : '${_mon[last.month - 1]} ${last.day}'; + return '$s–$e'; + } String _fmtAvg(num v) { // Sleep + wear avgs come in minutes → show as Hh Mm in the hero. @@ -288,23 +362,38 @@ class TrendBoard extends StatelessWidget { @override Widget build(BuildContext context) { final buckets = (data['buckets'] as List?) ?? const []; - final unit = data['unit']?.toString() ?? ''; - final label = data['label']?.toString() ?? title; + final display = trendDisplay( + metric, + data['label']?.toString() ?? title, + data['unit']?.toString() ?? '', + ); + final unit = display.unit; + final label = display.label; final summary = (data['summary'] as Map?)?.cast(); - final values = [ - for (final b in buckets) ((b as Map)['value'] as num?)?.toDouble() ?? 0.0, + // NULL for an absent bucket — see [trendBucketValue]. Coercing to 0.0 here + // is what made an unworn Tuesday draw a real bar (and, for oxygen, print a + // literal "0" SpO₂) under a tooltip promising "empty periods stay empty". + final List values = [ + for (final b in buckets) trendBucketValue(b as Map), ]; final labels = [ for (var i = 0; i < buckets.length; i++) trendBarLabel(scale, i, buckets[i] as Map), ]; - final allZero = values.every((v) => v == 0); + // "No data" means NO bucket carried a measurement — not "every measurement + // happened to be zero", which is itself a finding worth drawing. + final noData = values.every((v) => v == null); final avg = summary?['avg'] as num?; final delta = summary?['delta_vs_prev'] as num?; final met = summary?['met_count'] as num?; final total = summary?['total'] as num?; final showUnit = unit.isNotEmpty && metric != 'sleep' && metric != 'wear'; final info = infoFor(metric); + final period = _periodOf(buckets); + // /trend's `delta_vs_prev` is `avg - prevAvg` — an ABSOLUTE difference in + // the metric's own unit. Sleep/wear averages travel in minutes even though + // their display unit is 'h', so the chip must say 'min'. + final deltaUnit = (metric == 'sleep' || metric == 'wear') ? 'min' : unit; return BentoTile( accent: accent, @@ -314,7 +403,7 @@ class TrendBoard extends StatelessWidget { mainAxisSize: MainAxisSize.min, children: [ TileHeader( - '$label · $_period', + '$label · $period', icon: icon, trailing: Row( mainAxisSize: MainAxisSize.min, @@ -325,10 +414,17 @@ class TrendBoard extends StatelessWidget { InfoDot( title: title, body: info ?? - 'Your $label, averaged across $_period. Tap a bar to ' - 'drill into a finer period.', - methodNote: 'Bars show each period’s value; empty periods ' - 'stay empty.', + 'Your $label, averaged across the last $_period. Tap a ' + 'bar to drill into a finer period.', + methodNote: metric == 'spo2' + ? 'This series is the IMPORTED vendor oxygen index — a ' + 'relative saturation number, not a dip rate and ' + 'not an absolute SpO₂. On-device decoding writes ' + 'nothing here. Bars show each period’s value; ' + 'periods with no measurement stay empty.' + : 'Bars show each period’s value; periods with no ' + 'measurement stay empty — a gap is a gap, never ' + 'a zero.', ), ], ), @@ -347,12 +443,18 @@ class TrendBoard extends StatelessWidget { ), if (delta != null && delta != 0) ...[ const SizedBox(width: Sp.x3), - DeltaChip(delta), + BaselineDeltaChip( + delta, + unit: deltaUnit, + goodIsUp: trendGoodIsUp(metric), + showVsNormal: false, + vsLabel: 'vs prev', + ), ], ], ), const SizedBox(height: Sp.x5), - if (allZero) + if (noData) SizedBox( height: 120, child: Center( diff --git a/lib/ui/screens/screens.dart b/lib/ui/screens/screens.dart index 9fb1cc06..07e31a76 100644 --- a/lib/ui/screens/screens.dart +++ b/lib/ui/screens/screens.dart @@ -36,8 +36,10 @@ class SleepScreen extends StatelessWidget { title: 'Sleep', metric: 'sleep', accent: DomainAccent.sleep, - valueFmt: (v) => - v == 0 ? '' : (v / 60).toStringAsFixed(1), // minutes → hours on bars + // minutes → hours on bars. A MEASURED zero prints '0.0'; an absent night + // never reaches here (LabeledBars renders nulls as an em-dash gap). The + // old `v == 0 ? ''` blanked both cases identically. + valueFmt: (v) => (v / 60).toStringAsFixed(1), // Sleep Coach now renders INSIDE SleepDetailScreen/SleepNightContent // (below Cycles, above Nocturnal heart) rather than as a separate // leading card here — same scroll, just reordered so the night's own @@ -83,7 +85,11 @@ class OxygenScreen extends StatelessWidget { title: 'Overnight oxygen', metric: 'spo2', accent: DomainAccent.oxygen, - valueFmt: (v) => v == 0 ? '0' : v.toStringAsFixed(1), + // Was `v == 0 ? '0' : …` — it PRINTED a literal "0" above the bar for a + // night with no measurement at all, i.e. an SpO₂ of zero. Absent buckets + // no longer reach a formatter (they render as a gap); what does reach it + // is the imported vendor oxygen index, formatted plainly. + valueFmt: (v) => v.toStringAsFixed(1), todayDetail: (ctx) => OxygenDayCard(date: todayLabel()), dayDetail: (ctx, date) => OxygenDayCard(date: date), ); @@ -99,8 +105,8 @@ class WearScreen extends StatelessWidget { title: 'Wear time', metric: 'wear', accent: AppColors.coralDeep, - valueFmt: (v) => - v == 0 ? '' : (v / 60).toStringAsFixed(1), // minutes → hours on bars + // minutes → hours on bars; a measured zero is a real (bad) day, not a gap. + valueFmt: (v) => (v / 60).toStringAsFixed(1), todayDetail: (ctx) => WearDayCard(date: todayLabel()), dayDetail: (ctx, date) => WearDayCard(date: date), ); diff --git a/lib/ui/screens/trend_screen.dart b/lib/ui/screens/trend_screen.dart index 193325dd..7c66f3d1 100644 --- a/lib/ui/screens/trend_screen.dart +++ b/lib/ui/screens/trend_screen.dart @@ -114,7 +114,11 @@ class _TrendTodayCardState extends State<_TrendTodayCard> { child: Center(child: SizedBox(width: 22, height: 22, child: CircularProgressIndicator(strokeWidth: 2))))); } final buckets = (_d?['buckets'] as List?) ?? const []; - final unit = _d?['unit']?.toString() ?? ''; + final unit = trendDisplay( + widget.metric, + _d?['label']?.toString() ?? '', + _d?['unit']?.toString() ?? '', + ).unit; final summary = (_d?['summary'] as Map?)?.cast(); final isDay = widget.date != null; @@ -128,15 +132,16 @@ class _TrendTodayCardState extends State<_TrendTodayCard> { final dstr = DateTime.fromMillisecondsSinceEpoch(ts * 1000, isUtc: true) .toIso8601String().substring(0, 10); if (dstr == widget.date) { - if (bm['has'] == true && bm['value'] is num) value = (bm['value'] as num).toDouble(); + value = trendBucketValue(bm); break; } } } else { - // Today leaf: latest day with a value. + // Today leaf: latest day that actually CARRIES a measurement. `value` is + // padded to 0.0 on absent buckets, so `v is num` matched every day. for (final b in buckets.reversed) { - final v = (b as Map)['value']; - if (v is num) { value = v.toDouble(); break; } + final v = trendBucketValue(b as Map); + if (v != null) { value = v; break; } } } @@ -165,10 +170,24 @@ class _TrendTodayCardState extends State<_TrendTodayCard> { const SizedBox(width: Sp.x2), Padding(padding: const EdgeInsets.only(bottom: 8), child: Text(unit, style: AppText.bodySoft)), ], - // Week-over-week delta only makes sense on the latest (non-day) leaf. + // Week-over-week delta only makes sense on the latest (non-day) + // leaf. It is an ABSOLUTE difference in the metric's own unit + // (`avg - prevAvg`), so it goes in a BaselineDeltaChip — DeltaChip + // would stamp a '%' on it and turn "20 min more sleep" into "20%". if (!isDay && delta is num && delta != 0) ...[ const SizedBox(width: Sp.x3), - Padding(padding: const EdgeInsets.only(bottom: 8), child: DeltaChip(delta)), + Padding( + padding: const EdgeInsets.only(bottom: 8), + child: BaselineDeltaChip( + delta, + unit: (widget.metric == 'sleep' || widget.metric == 'wear') + ? 'min' + : unit, + goodIsUp: trendGoodIsUp(widget.metric), + showVsNormal: false, + vsLabel: 'vs prev', + ), + ), ], ]), if (info != null) ...[ diff --git a/lib/ui/widgets/async_guards.dart b/lib/ui/widgets/async_guards.dart new file mode 100644 index 00000000..bdbc54e4 --- /dev/null +++ b/lib/ui/widgets/async_guards.dart @@ -0,0 +1,51 @@ +// Two small guards for the same class of bug: state written back from a future +// that is no longer the one the screen is waiting for. +// +// • [LatestRequestGate] — a NEWER request has started; drop the older reply. +// • [setStateIfMounted] — the State is GONE; drop the reply entirely. +// +// Both exist because `await` has no memory: the continuation runs whenever the +// future happens to complete, with no relationship to what the widget still +// wants (or to whether it still exists). + +import 'package:flutter/widgets.dart'; + +/// Request-generation guard for widgets that can have several fetches in +/// flight at once (e.g. an `insightsRevision` listener firing again while the +/// previous load is still awaiting). +/// +/// Futures do NOT complete in the order they were started: a cross-day rollup +/// followed immediately by a derive leaves two loads racing, and the LATER one +/// can finish FIRST — after which the earlier, staler payload lands last and +/// wins. That is exactly the stale-day bug the revision listener was added to +/// fix, reintroduced by the listener itself. Take a token before awaiting and +/// drop the result if a newer request has started since. +/// +/// final token = _gate.begin(); +/// final d = await load(); +/// if (!mounted || !_gate.isCurrent(token)) return; // superseded +class LatestRequestGate { + int _gen = 0; + + /// Start a request and get its token. Every earlier token is now stale. + int begin() => ++_gen; + + /// True only for the most recently started request. + bool isCurrent(int token) => token == _gen; +} + +/// `setState` that is a no-op once the State has been disposed. +/// +/// Any `setState` reached AFTER an `await` needs this. A provider call can +/// block for up to two minutes; the user pops the screen; the call then errors +/// (or its progress callbacks fire) and the handler crashes the frame with +/// "setState() called after dispose()". The `mounted` check is not defensive +/// noise — for a long await it is the normal case. +extension SafeSetState on State { + void setStateIfMounted(VoidCallback fn) { + if (mounted) { + // ignore: invalid_use_of_protected_member + setState(fn); + } + } +} diff --git a/lib/widget/widget_service.dart b/lib/widget/widget_service.dart index 2ca03c44..d5fb5019 100644 --- a/lib/widget/widget_service.dart +++ b/lib/widget/widget_service.dart @@ -78,7 +78,12 @@ class WidgetService { s.isEmpty ? -1.0 : s.value!.toDouble(), ); await setI('sleep_min', sleep.isEmpty ? -1 : sleep.value!.round()); - await setI('sleep_need_min', need.isEmpty ? 480 : need.value!.round()); + // -1, like every other int key here. This used to write a hard 480 — + // a fabricated 8h00m sleep need shown on the home widget AND mirrored to + // the watch, with the sleep ring's fill computed as a fraction of an + // invented denominator. The native readers gate their ring on + // `needMin > 0`, so the sentinel simply leaves it empty. + await setI('sleep_need_min', need.isEmpty ? -1 : need.value!.round()); await setI('rhr', rhr.isEmpty ? -1 : rhr.value!.round()); await HomeWidget.saveWidgetData( 'coach_line', diff --git a/pubspec.lock b/pubspec.lock index 1de73eb6..f1caf073 100644 --- a/pubspec.lock +++ b/pubspec.lock @@ -266,7 +266,7 @@ packages: source: hosted version: "1.3.3" ffi: - dependency: transitive + dependency: "direct dev" description: name: ffi sha256: "6d7fd89431262d8f3125e81b50d3847a091d846eafcd4fdb88dd06f36d705a45" @@ -1063,7 +1063,7 @@ packages: source: hosted version: "2.2.1" path_provider_platform_interface: - dependency: transitive + dependency: "direct dev" description: name: path_provider_platform_interface sha256: "88f5779f72ba699763fa3a3b06aa4bf6de76c8e5de842cf6f29e2e06476c2334" diff --git a/pubspec.yaml b/pubspec.yaml index c837c36c..8e919c0c 100644 --- a/pubspec.yaml +++ b/pubspec.yaml @@ -181,6 +181,14 @@ dev_dependencies: # RouteTracker's stall-watchdog tests drive virtual time (was a transitive # dep via flutter_test; declared directly since test/ now imports it). fake_async: ^1.3.1 + # day_window_dst_test moves the PROCESS timezone via libc setenv/tzset so the + # DST day-length regressions are real on a non-DST host (transitive via + # sqflite_common_ffi; declared directly since test/ now imports it). + ffi: ^2.1.0 + # db_p0_fixes_test swaps PathProviderPlatform.instance for a temp-dir fake so + # exportDaysDb can run without a platform plugin (transitive via + # path_provider; declared directly since test/ now imports it). + path_provider_platform_interface: ^2.1.0 flutter_launcher_icons: android: "launcher_icon" diff --git a/test/absent_not_zero_test.dart b/test/absent_not_zero_test.dart new file mode 100644 index 00000000..79487172 --- /dev/null +++ b/test/absent_not_zero_test.dart @@ -0,0 +1,548 @@ +// The honesty contract, under test: an ABSENT measurement renders as "—" / +// nothing / an explicit gap — NEVER as 0, and never as a placeholder that +// reads as measured. +// +// Every test here fails against the pre-fix behaviour. The repository already +// tells the truth (`has` flags, nullable scalars, `segments` as a list); these +// pin down the UI actually reading it instead of `?? 0`-ing it away. + +import 'package:flutter/material.dart'; +import 'package:flutter_test/flutter_test.dart'; + +import 'package:openstrap_edge/theme/theme.dart'; +import 'package:openstrap_edge/theme/tokens.dart'; +import 'package:openstrap_edge/ui/design/recap_card.dart' show RecapCard; +import 'package:openstrap_edge/ui/kit/charts.dart' + show HrReplayOverlay, LabeledBars, MiniBars, TimeSeriesPoint; +import 'package:openstrap_edge/ui/kit/kit.dart' show OsIcon; +import 'package:openstrap_edge/ui/screens/detail_cards.dart' + show OxygenNightContent, WearDayContent; +import 'package:openstrap_edge/ui/screens/metric_screen.dart' + show TrendBoard, trendBarLabel, trendBucketValue, trendDisplay; +import 'package:openstrap_edge/ui/widgets/async_guards.dart' + show LatestRequestGate, SafeSetState; + +Widget _host(Widget child, {Palette palette = kLightPalette}) { + AppColors.active = palette; + return MaterialApp( + theme: buildOpenStrapTheme(palette), + home: Scaffold(body: SingleChildScrollView(child: child)), + ); +} + +void _phone(WidgetTester t, {double height = 1400}) { + t.view.physicalSize = Size(390, height); + t.view.devicePixelRatio = 1.0; + addTearDown(t.view.reset); +} + +String _today() { + final d = DateTime.now(); + return '${d.year.toString().padLeft(4, '0')}-' + '${d.month.toString().padLeft(2, '0')}-' + '${d.day.toString().padLeft(2, '0')}'; +} + +/// A week /trend payload; [absent] lists the bucket indices the repository +/// marked as having no measurement (it still pads their `value` to 0.0). +Map _week({ + Set absent = const {}, + String unit = 'bpm', + String label = 'resting HR', + num? delta, + double Function(int i)? value, +}) { + final monday = DateTime.utc(2026, 7, 13); + return { + 'label': label, + 'unit': unit, + 'summary': {'avg': 52.4, 'delta_vs_prev': delta, 'total': 7}, + 'buckets': [ + for (var i = 0; i < 7; i++) + { + 't_start': monday.add(Duration(days: i)).millisecondsSinceEpoch ~/ 1000, + 't_end': + monday.add(Duration(days: i + 1)).millisecondsSinceEpoch ~/ 1000, + // The repository's exact shape: 0.0 padding + the `has` flag. + 'value': absent.contains(i) ? 0.0 : (value?.call(i) ?? (50 + i)), + 'has': !absent.contains(i), + }, + ], + }; +} + +/// Bars actually PAINTED by a LabeledBars (a gap paints nothing). +int _paintedBars(WidgetTester t) => t + .widgetList( + find.descendant( + of: find.byType(LabeledBars), + matching: find.byType(FractionallySizedBox), + ), + ) + .length; + +Map _wearDay({ + Object? worn = 900, + Object? cov = 62, + Object? segments, + Object? longestOff, + List? hourly, +}) { + final m = { + 'first_on': 1752300000, + 'last_on': 1752380000, + 'hourly': hourly ?? const [], + }; + // Keys are OMITTED (not set to null) when absent — exactly how a bundle with + // no engine wear block reaches the UI. + if (worn != null) m['worn_min'] = worn; + if (cov != null) m['coverage_pct'] = cov; + if (segments != null) m['segments'] = segments; + if (longestOff != null) m['longest_off_min'] = longestOff; + return m; +} + +void main() { + tearDown(() => AppColors.active = kLightPalette); + + // ── 3 · the `has` flag, all the way down ────────────────────────────────── + group('trend buckets: `has` is the only truth', () { + test('trendBucketValue returns null for a bucket the repo marked absent', + () { + // Exactly what local_repository_impl writes: `value: v ?? 0.0`. + expect(trendBucketValue({'value': 0.0, 'has': false}), isNull); + expect(trendBucketValue({'value': 0.0, 'has': true}), 0.0); + expect(trendBucketValue({'value': 51.0, 'has': true}), 51.0); + // Payloads that predate `has` still work off value-presence. + expect(trendBucketValue({'value': 51.0}), 51.0); + expect(trendBucketValue(const {}), isNull); + }); + + testWidgets('LabeledBars draws NO bar for a null value and labels it —', + (t) async { + _phone(t); + await t.pumpWidget(_host(const SizedBox( + height: 220, + child: LabeledBars( + values: [5.0, null, 7.0], + labels: ['Mon', 'Tue', 'Wed'], + ), + ))); + await t.pump(const Duration(milliseconds: 700)); + // Two real bars, not three: the 0.02 height floor must not manufacture + // a bar out of a missing measurement. + expect(_paintedBars(t), 2); + expect(find.text('—'), findsOneWidget); + expect(t.takeException(), isNull); + }); + + testWidgets('TrendBoard leaves an unworn day empty instead of drawing ' + 'an accent bar at the floor', (t) async { + _phone(t); + await t.pumpWidget(_host(TrendBoard( + data: _week(absent: {3}), + title: 'Heart', + icon: OsIcon.heart, + metric: 'resting_hr', + scale: 'week', + accent: AppColors.coral, + ))); + await t.pump(const Duration(milliseconds: 900)); + expect(_paintedBars(t), 6); // 7 days, 1 gap + expect(find.text('—'), findsWidgets); + expect(t.takeException(), isNull); + }); + + testWidgets('an all-absent period says "No data", an all-ZERO one draws ' + 'its real zeros', (t) async { + _phone(t); + await t.pumpWidget(_host(TrendBoard( + data: _week(absent: {0, 1, 2, 3, 4, 5, 6}), + title: 'Steps', + icon: OsIcon.activity, + metric: 'steps', + scale: 'week', + accent: AppColors.coral, + ))); + await t.pump(const Duration(milliseconds: 700)); + expect(find.text('No data in this period'), findsOneWidget); + + // Seven measured zeros are a finding, not an absence. + await t.pumpWidget(_host(TrendBoard( + data: _week(value: (_) => 0.0), + title: 'Steps', + icon: OsIcon.activity, + metric: 'steps', + scale: 'week', + accent: AppColors.coral, + ))); + await t.pump(const Duration(milliseconds: 700)); + expect(find.text('No data in this period'), findsNothing); + expect(_paintedBars(t), 7); + expect(t.takeException(), isNull); + }); + }); + + // ── 4 · the fabricated oxygen "0" ───────────────────────────────────────── + group('oxygen bars', () { + testWidgets('an unmeasured night never prints a literal 0 SpO₂', (t) async { + _phone(t); + await t.pumpWidget(_host(TrendBoard( + data: _week(absent: {1}, unit: 'dips/h', label: 'oxygen dips'), + title: 'Overnight oxygen', + icon: OsIcon.hydration, + metric: 'spo2', + scale: 'week', + accent: AppColors.coral, + // The screen's OLD formatter, kept here on purpose: if an absent + // bucket ever reaches a value formatter again, this fails loudly. + valueFmt: (v) => v == 0 ? '0' : v.toStringAsFixed(1), + ))); + await t.pump(const Duration(milliseconds: 900)); + expect(find.text('0'), findsNothing); + expect(find.text('—'), findsWidgets); + expect(t.takeException(), isNull); + }); + + // ── 6 · the series is not what its name says ──────────────────────────── + test('trendDisplay renames the spo2 series to what it actually holds', () { + final d = trendDisplay('spo2', 'oxygen dips', 'dips/h'); + expect(d.label, 'imported oxygen index'); + expect(d.unit, isEmpty); // never "dips/h" + // Every other series is passed through untouched. + final hr = trendDisplay('resting_hr', 'resting HR', 'bpm'); + expect(hr.label, 'resting HR'); + expect(hr.unit, 'bpm'); + }); + + testWidgets('the spo2 board is not captioned as a dip rate', (t) async { + _phone(t); + await t.pumpWidget(_host(TrendBoard( + data: _week(unit: 'dips/h', label: 'oxygen dips', + value: (i) => 95.0 + i * 0.1), + title: 'Overnight oxygen', + icon: OsIcon.hydration, + metric: 'spo2', + scale: 'week', + accent: AppColors.coral, + ))); + await t.pump(const Duration(milliseconds: 900)); + expect(find.text('dips/h'), findsNothing); + // TileHeader uppercases its title. + expect(find.textContaining(RegExp('oxygen dips', caseSensitive: false)), + findsNothing); + expect( + find.textContaining( + RegExp('imported oxygen index', caseSensitive: false)), + findsOneWidget); + expect(t.takeException(), isNull); + }); + }); + + // ── 5 · an absolute delta is not a percentage ───────────────────────────── + group('trend delta chip', () { + testWidgets('renders the absolute delta in its own unit, never as %', + (t) async { + _phone(t); + await t.pumpWidget(_host(TrendBoard( + data: _week(delta: -1.2), + title: 'Heart', + icon: OsIcon.heart, + metric: 'resting_hr', + scale: 'week', + accent: AppColors.coral, + ))); + await t.pump(const Duration(milliseconds: 900)); + expect(find.text('−1.2 bpm vs prev'), findsOneWidget); + expect(find.textContaining('1.2%'), findsNothing); + expect(t.takeException(), isNull); + }); + + testWidgets('sleep deltas travel in MINUTES and say so', (t) async { + _phone(t); + await t.pumpWidget(_host(TrendBoard( + data: _week(delta: 20, unit: 'h', label: 'sleep', + value: (i) => 420 + i.toDouble()), + title: 'Sleep', + icon: OsIcon.activity, + metric: 'sleep', + scale: 'week', + accent: AppColors.coral, + ))); + await t.pump(const Duration(milliseconds: 900)); + // The bug: "20 minutes more sleep" rendered as "▲ 20.0%". + expect(find.text('+20 min vs prev'), findsOneWidget); + expect(find.textContaining('20.0%'), findsNothing); + expect(t.takeException(), isNull); + }); + }); + + // ── window labelling ────────────────────────────────────────────────────── + group('window labels name the real window', () { + test('month buckets are rolling 7-day windows, labelled by their end', () { + final b = { + 't_start': DateTime.utc(2026, 7, 8).millisecondsSinceEpoch ~/ 1000, + 't_end': DateTime.utc(2026, 7, 15).millisecondsSinceEpoch ~/ 1000, + }; + expect(trendBarLabel('month', 0, b), 'Jul 14'); // was the false 'W1' + // week + quarter are unchanged. + final mon = { + 't_start': DateTime.utc(2026, 7, 6).millisecondsSinceEpoch ~/ 1000, + }; + expect(trendBarLabel('week', 0, mon), 'Mon'); + expect(trendBarLabel('quarter', 0, mon), 'Jul'); + }); + + testWidgets('the board header dates the window instead of saying ' + '"this week"', (t) async { + _phone(t); + await t.pumpWidget(_host(TrendBoard( + data: _week(), + title: 'Heart', + icon: OsIcon.heart, + metric: 'resting_hr', + scale: 'week', + accent: AppColors.coral, + ))); + await t.pump(const Duration(milliseconds: 700)); + // /trend anchors on the LAST DAY WITH DATA when no anchor is given, so + // "this week" can be a fortnight ago. + expect(find.textContaining(RegExp('this week', caseSensitive: false)), + findsNothing); + expect(find.textContaining(RegExp('Jul 13.19', caseSensitive: false)), + findsOneWidget); + expect(t.takeException(), isNull); + }); + }); + + // ── 1, 2, 7, 8 · the wear day board ─────────────────────────────────────── + group('WearDayContent', () { + testWidgets('counts the segments LIST instead of printing 0 forever', + (t) async { + _phone(t, height: 2400); + await t.pumpWidget(_host(WearDayContent( + data: _wearDay(segments: [ + {'start': 1, 'end': 2}, + {'start': 3, 'end': 4}, + {'start': 5, 'end': 6}, + ]), + date: _today(), + ))); + await t.pump(const Duration(milliseconds: 900)); + expect(find.text('WEAR STRETCHES'), findsOneWidget); + expect(find.text('3'), findsOneWidget); + // The old `_n(List) ?? 0` printed this on every worn day, every device. + expect(find.text('0'), findsNothing); + expect(t.takeException(), isNull); + }); + + testWidgets('an unrecorded stretch count is —, not 0', (t) async { + _phone(t, height: 2400); + await t.pumpWidget(_host(WearDayContent( + // Exactly what getDayWear emits with no engine wear block. + data: _wearDay(segments: const []), + date: _today(), + ))); + await t.pump(const Duration(milliseconds: 900)); + expect(find.text('WEAR STRETCHES'), findsOneWidget); + expect(find.text('0'), findsNothing); + expect(find.text('—'), findsWidgets); + expect(t.takeException(), isNull); + }); + + testWidgets('a day with NO wear measurement does not assert "not worn"', + (t) async { + _phone(t); + await t.pumpWidget(_host(WearDayContent( + data: _wearDay(worn: null, cov: null), // imported day: no wear block + date: _today(), + ))); + await t.pump(const Duration(milliseconds: 700)); + expect(find.text('Not worn on this day'), findsNothing); + expect(find.text('Wear time wasn’t recorded'), findsOneWidget); + + // A MEASURED zero still says so — the two claims stay distinguishable. + await t.pumpWidget(_host( + WearDayContent(data: const {'worn_min': 0}, date: _today()), + )); + await t.pump(const Duration(milliseconds: 700)); + expect(find.text('Not worn on this day'), findsOneWidget); + expect(t.takeException(), isNull); + }); + + testWidgets('"Longest off: none" is only claimed from a measured zero', + (t) async { + _phone(t, height: 2400); + await t.pumpWidget(_host(WearDayContent( + data: _wearDay(segments: const []), // no longest_off_min at all + date: _today(), + ))); + await t.pump(const Duration(milliseconds: 900)); + expect(find.text('LONGEST OFF'), findsOneWidget); + expect(find.text('none'), findsNothing); + + await t.pumpWidget(_host(WearDayContent( + data: _wearDay(segments: const [], longestOff: 0), + date: _today(), + ))); + await t.pump(const Duration(milliseconds: 900)); + expect(find.text('none'), findsOneWidget); + expect(t.takeException(), isNull); + }); + + testWidgets('a recent day with no hourly array explains itself instead of ' + 'rendering nothing', (t) async { + _phone(t, height: 2400); + await t.pumpWidget(_host(WearDayContent( + data: _wearDay(segments: const []), // hourly: [] — the real payload + date: _today(), + ))); + await t.pump(const Duration(milliseconds: 900)); + expect(find.text('Hourly coverage'), findsNothing); + expect( + find.textContaining('Hour-by-hour wear isn’t stored'), + findsOneWidget, + ); + expect(t.takeException(), isNull); + }); + }); + + // ── 9 · oxygen severity from absent inputs ──────────────────────────────── + group('OxygenNightContent severity', () { + testWidgets('trusted coverage with NO computed dip metrics is "Not ' + 'graded", never an all-clear', (t) async { + _phone(t, height: 3600); + await t.pumpWidget(_host(OxygenNightContent( + data: const { + 'spo2': { + 'trusted_coverage': 0.71, + 'signal_coverage': 0.82, + 'analyzed_hours': 6.8, + 'dip_count': 0, + // odi_per_hour / max_dip_pct / burden_pct all absent. + }, + }, + date: _today(), + ))); + await t.pump(const Duration(milliseconds: 1200)); + expect(find.text('Not graded'), findsOneWidget); + expect(find.text('Quiet'), findsNothing); + expect( + find.textContaining('No meaningful overnight oxygen dips'), + findsNothing, + ); + expect(t.takeException(), isNull); + }); + }); + + // ── recap strip gaps ────────────────────────────────────────────────────── + group('RecapCard week strip', () { + testWidgets('keeps a missing day in place instead of shifting the week ' + 'left', (t) async { + _phone(t); + await t.pumpWidget(_host(const RecapCard( + title: 'Weekly recap', + value: '7h 12m', + caption: 'daily average', + bars: [420.0, 430.0, null, 445.0, 455.0, 460.0, 470.0], + ))); + await t.pump(const Duration(milliseconds: 700)); + final bars = t.widget(find.byType(MiniBars)); + expect(bars.values.length, 7); // was 6 — Thu–Sun slid onto Wed–Sat + expect(bars.values[2], isNull); + expect(t.takeException(), isNull); + }); + }); + + // ── the workout HR replay dot ───────────────────────────────────────────── + group('HrReplayOverlay', () { + testWidgets('mounts the replay dot once playing', (t) async { + _phone(t); + await t.pumpWidget(_host(SizedBox( + height: 200, + child: Stack( + children: [ + HrReplayOverlay( + points: const [ + TimeSeriesPoint(0, 60), + TimeSeriesPoint(1, 90), + TimeSeriesPoint(2, 70), + ], + loX: 0, + hiX: 2, + loY: 60, + hiY: 90, + chartHeight: 200, + ), + ], + ), + ))); + await t.pump(const Duration(milliseconds: 100)); + final dot = find.descendant( + of: find.byType(HrReplayOverlay), + matching: find.byType(IgnorePointer), + ); + expect(dot, findsNothing); + + await t.tap(find.byIcon(Icons.play_arrow)); + await t.pump(); + await t.pump(const Duration(milliseconds: 600)); + // The AnimatedBuilder used to live INSIDE the `0 < t < 1` gate, so the + // only rebuild (the setState in _toggle, at t == 0) never passed it and + // the dot never mounted. + expect(dot, findsOneWidget); + await t.pump(const Duration(seconds: 6)); + expect(t.takeException(), isNull); + }); + }); + + // ── 10 · stale-response overwrite ───────────────────────────────────────── + group('LatestRequestGate', () { + test('only the newest request may write back', () { + final gate = LatestRequestGate(); + final first = gate.begin(); + expect(gate.isCurrent(first), isTrue); + + final second = gate.begin(); // a second revision arrives + // The first load completes LAST (futures do not settle in start order)… + expect(gate.isCurrent(first), isFalse); // …and is dropped. + expect(gate.isCurrent(second), isTrue); + + // A third supersedes the second in turn. + final third = gate.begin(); + expect(gate.isCurrent(second), isFalse); + expect(gate.isCurrent(third), isTrue); + }); + }); + + // ── 13, 14 · setState after dispose ─────────────────────────────────────── + group('setStateIfMounted', () { + testWidgets('a callback that lands after dispose is dropped, not thrown', + (t) async { + await t.pumpWidget(const MaterialApp(home: _LateSetState())); + final state = t.state<_LateSetStateState>(find.byType(_LateSetState)); + await t.pumpWidget(const MaterialApp(home: SizedBox())); + expect(state.mounted, isFalse); + + // What a 120 s provider call's onItem/onStatus/catch does when the user + // has already popped the screen. + state.setStateIfMounted(() {}); + expect(t.takeException(), isNull); + + // The control: the unguarded call this replaced does throw. + expect(() => state.callSetState(() {}), throwsFlutterError); + }); + }); +} + +class _LateSetState extends StatefulWidget { + const _LateSetState(); + @override + State<_LateSetState> createState() => _LateSetStateState(); +} + +class _LateSetStateState extends State<_LateSetState> { + void callSetState(VoidCallback fn) => setState(fn); + @override + Widget build(BuildContext context) => const SizedBox(); +} diff --git a/test/ai_screen_async_guards_test.dart b/test/ai_screen_async_guards_test.dart new file mode 100644 index 00000000..4d06af27 --- /dev/null +++ b/test/ai_screen_async_guards_test.dart @@ -0,0 +1,128 @@ +// Async-lifecycle guards on the AI surfaces. All three bugs share a shape: a +// callback lands after the world has moved on (a newer request started, or the +// screen is gone) and clobbers what's on screen — or crashes the frame. + +import 'dart:async'; + +import 'package:flutter/material.dart'; +import 'package:flutter_test/flutter_test.dart'; +import 'package:provider/provider.dart'; +import 'package:shared_preferences/shared_preferences.dart'; + +import 'package:openstrap_edge/ai/briefing.dart'; +import 'package:openstrap_edge/ai/briefing_engine.dart'; +import 'package:openstrap_edge/coach/coach_config.dart'; +import 'package:openstrap_edge/data/local_repository.dart'; +import 'package:openstrap_edge/state/prefs.dart'; +import 'package:openstrap_edge/theme/theme.dart'; +import 'package:openstrap_edge/theme/tokens.dart'; +import 'package:openstrap_edge/ui/ai/ai_breakdown_screen.dart'; +import 'package:openstrap_edge/ui/ai/ai_settings_screen.dart'; + +class _FakeRepo extends LocalRepository { + @override + Future> getToday() async => { + 'daily': { + 'readiness': {'value': 74}, + }, + 'status': const {}, + }; + @override + Future> getDaySleep(String date) async => + {'has_sleep': false}; +} + +Widget _host(Widget child) { + AppColors.active = kLightPalette; + return MultiProvider( + providers: [ + ChangeNotifierProvider(create: (_) => CoachConfig()), + ], + child: MaterialApp(theme: buildOpenStrapTheme(kLightPalette), home: child), + ); +} + +void main() { + setUp(() async { + SharedPreferences.setMockInitialValues({}); + await Prefs.ensureLoaded(); + AppColors.active = kLightPalette; + AiSettingsScreen.pickerOverride = null; + }); + tearDown(() { + AiSettingsScreen.pickerOverride = null; + AppColors.active = kLightPalette; + }); + + testWidgets('regenerate is single-flight: a double tap does not put two ' + 'briefings in the air', (t) async { + t.view.physicalSize = const Size(390, 2000); + t.view.devicePixelRatio = 1.0; + addTearDown(t.view.reset); + + var calls = 0; + final engine = BriefingEngine( + config: CoachConfig(), + repo: _FakeRepo(), + complete: ({required system, required user}) async { + calls++; + await Future.delayed(const Duration(milliseconds: 40)); + return 'Recovered and ready.\n---\n- Readiness sits at 74'; + }, + ); + + await t.pumpWidget(_host(AiBreakdownScreen( + period: BriefingPeriod.morning, + engineOverride: engine, + ))); + await t.pump(); // post-frame _load → _generate + await t.pump(const Duration(milliseconds: 200)); + await t.pump(const Duration(milliseconds: 700)); + expect(calls, 1); + expect(find.text('Recovered and ready.'), findsOneWidget); + + // Two taps inside one frame — the button is still on screen for both. + final regen = find.byType(InkWell).first; + await t.tap(regen, warnIfMissed: false); + await t.tap(regen, warnIfMissed: false); + await t.pump(); + await t.pump(const Duration(milliseconds: 200)); + await t.pump(const Duration(milliseconds: 700)); + + // Without the in-flight guard both taps issued a generate, and whichever + // settled LAST won — an error arriving after a success discards the fresh + // briefing (and vice versa shows a stale one as current). + expect(calls, 2); + expect(find.text('Recovered and ready.'), findsOneWidget); + expect(t.takeException(), isNull); + }); + + testWidgets('AI settings: a time picked after the screen is gone is dropped', + (t) async { + t.view.physicalSize = const Size(390, 2400); + t.view.devicePixelRatio = 1.0; + addTearDown(t.view.reset); + + final picked = Completer(); + AiSettingsScreen.pickerOverride = (_, _) => picked.future; + + await t.pumpWidget(_host(const AiSettingsScreen())); + await t.pump(); + await t.pump(const Duration(milliseconds: 700)); + + await t.tap(find.text('Time').first); + await t.pump(); + + // The user leaves while the picker is still up. showTimePicker's future + // resolves after the dialog's exit transition, i.e. plausibly now. + await t.pumpWidget(_host(const SizedBox())); + await t.pump(); + + picked.complete(const TimeOfDay(hour: 7, minute: 30)); + await t.pump(); + await t.pump(const Duration(milliseconds: 300)); + + // Unguarded this is "setState() called after dispose()". + expect(t.takeException(), isNull); + }); +} diff --git a/test/app_state_regressions_test.dart b/test/app_state_regressions_test.dart new file mode 100644 index 00000000..395505ef --- /dev/null +++ b/test/app_state_regressions_test.dart @@ -0,0 +1,306 @@ +// Regression tests for a batch of AppState state-machine bugs. +// +// AppState.forTesting() builds the object graph WITHOUT running _init() and +// without touching a single platform plugin, so the logic below can be driven +// directly. Each group names the bug it guards. + +import 'package:flutter/foundation.dart'; +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:shared_preferences/shared_preferences.dart'; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; + +import 'package:openstrap_edge/ble/ble_engine.dart'; +import 'package:openstrap_edge/data/db.dart'; +import 'package:openstrap_edge/notify/notification_center.dart'; +import 'package:openstrap_edge/notify/notification_event.dart'; +import 'package:openstrap_edge/state/app_state.dart'; +import 'package:openstrap_edge/sync/paired_device.dart'; + +/// A BleEngine whose live-stream arming always fails — the "link dropped +/// mid-write" case that used to latch _stepCalActive true forever. +class _ThrowingEngine extends BleEngine { + _ThrowingEngine() + : super( + onRecord: _noRecord, + onState: _noState, + ); + static Future _noRecord(Object? sample, Object? raw) async {} + static void _noState(Object state) {} + + @override + Future retryFullLiveStreams() async => + throw StateError('link dropped mid-write'); +} + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + LocalDb.dbName = 'openstrap_app_state_regressions_test.db'; + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + tearDownAll(() async { + await LocalDb.close(); + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + setUp(() => SharedPreferences.setMockInitialValues({})); + + // ── 1. the serial heal must never RE-PAIR a band the user just unpaired ───── + group('healedPairing (stale engine-state callback after unpair)', () { + test('an unpaired app is NEVER re-paired from a stale engine state', () { + // BleEngine._teardownSession leaves state.serial/state.address set, so a + // late onState (e.g. the reconnect loop's finally → clearReconnecting → + // _setPhase(idle) → onState) arrives with a perfectly clean serial long + // after unpair() ran. The old guard (`cleanSn != paired?.serial`) was + // TRUE for paired == null and rebuilt a PairedDevice from state.address, + // silently re-pairing the removed band and bouncing the app back to the + // Shell. + expect(healedPairing(null, '4C2248092'), isNull); + expect(healedPairing(null, "Abdul's WHOOP"), isNull); + }); + + test('an EXISTING pairing still gets its junk serial healed', () { + final healed = healedPairing(PairedDevice('r-1', '?*?*'), '4C2248092'); + expect(healed, isNotNull); + expect(healed!.remoteId, 'r-1'); + expect(healed.serial, '4C2248092'); + }); + + test('a pairing with no serial yet gets one', () { + expect(healedPairing(PairedDevice('r-1', null), '4C2248092')?.serial, + '4C2248092'); + }); + + test('no change when the serial already matches, or the report is junk', + () { + expect(healedPairing(PairedDevice('r-1', '4C2248092'), '4C2248092'), + isNull); + expect(healedPairing(PairedDevice('r-1', '4C2248092'), '?*?*'), isNull); + expect(healedPairing(PairedDevice('r-1', '4C2248092'), null), isNull); + expect(healedPairing(PairedDevice('r-1', '4C2248092'), ' '), isNull); + }); + + test('the remoteId is never invented — it always comes from the pairing', + () { + // Even with a clean serial, an empty remoteId means there is nothing + // legitimate to write back. + expect(healedPairing(PairedDevice('', null), '4C2248092'), isNull); + }); + }); + + // ── 5. _stepCalActive must not latch true when the arming throws ──────────── + group('startStepCalibration (live-consumer latch)', () { + test('a throwing stream arm leaves no phantom live consumer', () async { + final engine = _ThrowingEngine(); + engine.state.connection = 'connected'; + final app = AppState.forTesting(engine: engine); + addTearDown(app.dispose); + + expect(app.debugHasLiveConsumer, isFalse); + await expectLater( + app.startStepCalibration(), throwsA(isA())); + // Pre-fix this stayed true for the rest of the process, pinning + // _hasLiveConsumer and permanently disabling + // _maybeDowngradeLiveForBackground — the 100 Hz raw flood then kept + // streaming while backgrounded and starved the R24 offload. + expect(app.debugHasLiveConsumer, isFalse); + }); + }); + + // ── 6. `busy` must not latch true forever ────────────────────────────────── + group('openSession (busy latch)', () { + test('unpairing while the session is opening does not wedge busy', + () async { + final app = AppState.forTesting(); + addTearDown(app.dispose); + app.paired = PairedDevice('r-1', '4C2248092'); + // Simulate the user tapping Unpair inside openSession's own resume + // window: the first thing openSession does after flipping busy is + // notify, and unpair() nulls `paired`. + app.addListener(() => app.paired = null); + + // Pre-fix this THREW (`paired!` sat outside the try) and left busy true, + // so every later openSession()/syncNow() no-opped — "Sync now" was dead + // until the process restarted. + await app.openSession(); + + expect(app.busy, isFalse); + expect(app.paired, isNull); + // And the state machine is genuinely usable again. + await app.syncNow(); + expect(app.busy, isFalse); + }); + }); + + // ── 7. the orphan-workout reconcile must not clobber a live workout ──────── + group('_reconcileOrphanedLiveWorkout (startWorkout race)', () { + test('a workout started inside the DB round-trip is not overwritten', + () async { + final nowSec = DateTime.now().millisecondsSinceEpoch ~/ 1000; + await LocalDb.putSession({ + 'id': 'stale-from-a-killed-run', + 'start_ts': nowSec - 600, + 'end_ts': null, + 'type': 'other', + 'status': 'live', + 'source': 'manual', + 'created_at': (nowSec - 600) * 1000, + }); + + final app = AppState.forTesting(); + addTearDown(app.dispose); + + // Kicked unawaited from _init(), one line before `initialized = true` + // makes the shell interactive — so the user can start a workout inside + // the round-trip. + final reconcile = app.debugReconcileOrphanedLiveWorkout(); + app.activeWorkout = LiveWorkoutState( + startTime: DateTime.now(), + targetKcal: 300, + workoutId: 'user-just-started-this', + type: 'run', + ); + await reconcile; + + expect(app.activeWorkout?.workoutId, 'user-just-started-this', + reason: 'the stale row must never replace a genuinely live workout ' + '(the old timer became unreachable and double-counted at 2 Hz)'); + }); + + test('with nothing live, a recent orphan is still resumed', () async { + final nowSec = DateTime.now().millisecondsSinceEpoch ~/ 1000; + await LocalDb.putSession({ + 'id': 'resumable', + 'start_ts': nowSec - 300, + 'end_ts': null, + 'type': 'run', + 'status': 'live', + 'source': 'manual', + 'created_at': (nowSec - 300) * 1000, + }); + + final app = AppState.forTesting(); + addTearDown(app.dispose); + await app.debugReconcileOrphanedLiveWorkout(); + expect(app.activeWorkout?.workoutId, 'resumable'); + }); + }); + + // ── 9. a fired alarm must be cleared from state AND prefs ────────────────── + group('alarm lifecycle (fired / strap-cleared)', () { + final originalSink = NotificationCenter.instance.presentSink; + tearDown(() => NotificationCenter.instance.presentSink = originalSink); + + Future silenceOsPresent() async { + NotificationCenter.instance.presentSink = + (NotificationEvent e, {bool allowPermissionPrompt = true}) async => + true; + } + + test('EXECUTED (event 57) clears the armed alarm and its persisted epoch', + () async { + SharedPreferences.setMockInitialValues({'alarm_epoch': 1785000000}); + await silenceOsPresent(); + final app = AppState.forTesting(); + addTearDown(app.dispose); + app.device.alarmEpoch = 1785000000; + expect(app.alarmEpoch, 1785000000); + + app.debugHandleAlarmEvent(57); + await Future.delayed(const Duration(milliseconds: 20)); + + // Pre-fix this only logged + notified: alarmEpoch kept returning the past + // epoch across relaunches (_init reloads `alarm_epoch`) and Profile's + // "Smart alarm" row advertised a spent one-shot as the CURRENT alarm. + expect(app.alarmEpoch, isNull); + final prefs = await SharedPreferences.getInstance(); + await prefs.reload(); + expect(prefs.getInt('alarm_epoch'), isNull); + expect(app.alarmFiredAt, isNotNull, reason: 'firedAt must survive'); + }); + + test('the app-side EXECUTED id (58) clears it too', () async { + SharedPreferences.setMockInitialValues({'alarm_epoch': 1785000000}); + await silenceOsPresent(); + final app = AppState.forTesting(); + addTearDown(app.dispose); + app.device.alarmEpoch = 1785000000; + + app.debugHandleAlarmEvent(58); + await Future.delayed(const Duration(milliseconds: 20)); + + expect(app.alarmEpoch, isNull); + final prefs = await SharedPreferences.getInstance(); + await prefs.reload(); + expect(prefs.getInt('alarm_epoch'), isNull); + }); + + test('the strap-driven clear (event 59) also drops the persisted epoch', + () async { + SharedPreferences.setMockInitialValues({'alarm_epoch': 1785000000}); + final app = AppState.forTesting(); + addTearDown(app.dispose); + app.device.alarmEpoch = 1785000000; + + app.debugHandleAlarmEvent(59); + await Future.delayed(const Duration(milliseconds: 20)); + + expect(app.alarmEpoch, isNull); + final prefs = await SharedPreferences.getInstance(); + await prefs.reload(); + expect(prefs.getInt('alarm_epoch'), isNull, + reason: 'state was nulled but the epoch used to stay on disk and ' + 'came back on the next launch'); + }); + + test('ALARM_SET (event 56) leaves the armed alarm alone', () async { + SharedPreferences.setMockInitialValues({'alarm_epoch': 1785000000}); + final app = AppState.forTesting(); + addTearDown(app.dispose); + app.device.alarmEpoch = 1785000000; + + app.debugHandleAlarmEvent(56); + await Future.delayed(const Duration(milliseconds: 20)); + + expect(app.alarmEpoch, 1785000000); + expect(app.alarmConfirmed, isTrue); + final prefs = await SharedPreferences.getInstance(); + await prefs.reload(); + expect(prefs.getInt('alarm_epoch'), 1785000000); + }); + }); + + // ── 10. dispose must release EVERYTHING AppState owns ────────────────────── + group('dispose', () { + testWidgets('cancels every owned timer', (t) async { + final app = AppState.forTesting(); + // _spotTimer, _breathingRecomputeTimer and _workoutTimer used to survive + // dispose; each callback ends in notifyListeners() on a disposed + // ChangeNotifier. An outstanding Timer fails this test outright. + app.debugArmOwnedTimers(); + app.dispose(); + }); + + testWidgets('disposes every owned notifier/observer', (t) async { + final app = AppState.forTesting(); + app.dispose(); + void addTo(void Function(VoidCallback) add) => + expect(() => add(() {}), throwsA(isA())); + addTo(app.navRequest.addListener); + addTo(app.screenRequest.addListener); + addTo(app.insightsRevision.addListener); + addTo(app.gestureSettings.addListener); + // NotificationRelay holds a WidgetsBindingObserver, a 120 s + // Timer.periodic and a StreamSubscription — its observer accumulated on + // the binding across every hot restart. + addTo(app.notificationRelay.addListener); + }); + }); +} diff --git a/test/ble_engine_test.dart b/test/ble_engine_test.dart index eac141b5..6acce803 100644 --- a/test/ble_engine_test.dart +++ b/test/ble_engine_test.dart @@ -122,41 +122,6 @@ void main() { }); }); - group('history-end settle streak', () { - test('resets while queue is not empty', () { - final streak = nextBurstStablePollStreak( - queueEmpty: false, - currentCount: 79, - previousCount: 79, - stableStreak: 2, - ); - - expect(streak, 0); - }); - - test('increments only when queue is empty and count is unchanged', () { - final streak = nextBurstStablePollStreak( - queueEmpty: true, - currentCount: 79, - previousCount: 79, - stableStreak: 1, - ); - - expect(streak, 2); - }); - - test('resets when traffic count changes between polls', () { - final streak = nextBurstStablePollStreak( - queueEmpty: true, - currentCount: 80, - previousCount: 79, - stableStreak: 2, - ); - - expect(streak, 0); - }); - }); - group('maintenance traffic gating', () { test('maintenance traffic is paused while offload is active', () { expect(shouldPauseMaintenanceTraffic(offloadActive: true), isTrue); diff --git a/test/ble_safe_trim_test.dart b/test/ble_safe_trim_test.dart new file mode 100644 index 00000000..65000cc3 --- /dev/null +++ b/test/ble_safe_trim_test.dart @@ -0,0 +1,370 @@ +// Regression tests for the SAFE-TRIM invariant and the offload-transport +// guards around it. +// +// THE INVARIANT: the durable commit (decoded_onehz + decoded_rr + cursor + +// raw_archive, one transaction) must be durable BEFORE the engine echoes the +// verbatim 8-byte HISTORY_END token, because that echo is what makes the band +// trim the chunk out of its flash. The happy path always honoured it; every +// test here covers a FAILURE path that did not. + +import 'dart:async'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_edge/ble/ble_engine.dart'; +import 'package:openstrap_edge/ble/ble_state.dart'; +import 'package:openstrap_edge/data/models.dart'; +import 'package:openstrap_edge/sync/sync_policy.dart'; + +/// A well-formed inner-frame hex: [0]=0x2f historical, [1]=0x18 (revision 24), +/// then the u32 record counter. BurstStats re-parses this, so it must be real +/// hex, not a label. +String _hex(int counter) => + '2f18${counter.toRadixString(16).padLeft(8, '0')}'; + +RawRecord _raw(int counter) => RawRecord( + counter: counter, + packetType: 0x2f, + hex: _hex(counter), + capturedAt: 1780000000000 + counter, + recTs: 1780000000 + counter, +); + +Sample _sample(int counter) => + Sample(tsEpoch: 1780000000 + counter, counter: counter, hr: 60); + +ArchiveRecord _archive(int counter) => ArchiveRecord( + counter: counter, + hex: _hex(counter), + packetType: 0x2f, + capturedAt: 1780000000000 + counter, + reason: 'undecodable_rec_v99', +); + +const _tokenA = [1, 2, 3, 4, 5, 6, 7, 8]; +const _tokenB = [9, 9, 9, 9, 9, 9, 9, 9]; + +/// A drain controller wired to [onCommit], the atomic-commit sink the real +/// engine points at `LocalDb.commitSyncBatch`. +DrainController _drainWith(CommitSyncBatchSink onCommit, {List? logs}) => + DrainController( + onRecord: (sample, raw) async {}, + onRecordsBatch: null, + onCommit: onCommit, + onArchive: null, + log: (line) => logs?.add(line), + ); + +void main() { + group('P0 — a durable commit that fails must not let the caller ACK', () { + test('commit() REPORTS failure instead of swallowing the exception', () async { + final d = _drainWith( + (raws, samples, token, {archives}) async => + throw StateError('OOM in SqlCommand.getSqlArguments'), + ); + d.onHistoricalRecord(_raw(1), _sample(1)); + + final durable = await d.commit(_tokenA); + + // OLD BEHAVIOUR: commit() was Future — it logged 'offload commit + // error' and returned normally, and the caller unconditionally built and + // wrote buildHistoryResultOk(), so the band trimmed a chunk that had + // rolled back. There was no success signal to check at all. + expect(durable, isFalse); + }); + + test('a failed commit RE-BUFFERS the records instead of losing them', () async { + final d = _drainWith( + (raws, samples, token, {archives}) async => throw StateError('rollback'), + ); + d.onHistoricalRecord(_raw(1), _sample(1)); + d.onHistoricalRecord(_raw(2), _sample(2)); + d.onUndecodableRecord(_archive(3)); + + expect(d.bufferedRecords, 2); + final durable = await d.commit(_tokenA); + + expect(durable, isFalse); + // OLD BEHAVIOUR: the buffer was snapshotted and CLEARED before the + // commit, and the exception was swallowed — so with the transaction + // rolled back and the cursor unadvanced, those rows existed nowhere. + expect(d.bufferedRecords, 2); + + // Proof they are really still there: a later successful commit ships + // every record AND the archived one. + final seenRaws = []; + final seenArchives = []; + final d2 = _drainWith((raws, samples, token, {archives}) async { + seenRaws.addAll(raws.map((r) => r.hex)); + seenArchives.addAll((archives ?? const []).map((a) => a.hex)); + }); + // (rebuild the same state on a controller whose commit succeeds) + d2.onHistoricalRecord(_raw(1), _sample(1)); + d2.onHistoricalRecord(_raw(2), _sample(2)); + d2.onUndecodableRecord(_archive(3)); + expect(await d2.commit(_tokenA), isTrue); + expect(seenRaws, [_hex(1), _hex(2)]); + expect(seenArchives, [_hex(3)]); + }); + + test( + 're-buffered records keep arrival order ahead of records that landed ' + 'during the failing await', + () async { + final gate = Completer(); + var fail = true; + final shipped = []; + final d = DrainController( + onRecord: (sample, raw) async {}, + onRecordsBatch: null, + onCommit: (raws, samples, token, {archives}) async { + if (fail) { + await gate.future; + throw StateError('rollback'); + } + shipped.addAll(raws.map((r) => r.hex)); + }, + onArchive: null, + log: (_) {}, + ); + + d.onHistoricalRecord(_raw(1), _sample(1)); + final inFlight = d.commit(_tokenA); + // A record arrives while the commit is parked mid-await. + d.onHistoricalRecord(_raw(2), _sample(2)); + gate.complete(); + expect(await inFlight, isFalse); + + expect(d.bufferedRecords, 2); + fail = false; + expect(await d.commit(_tokenA), isTrue); + expect(shipped, [_hex(1), _hex(2)]); + }, + ); + + test('a failed commit rolls back the trim-advance bookkeeping', () async { + var fail = false; + final d = DrainController( + onRecord: (sample, raw) async {}, + onRecordsBatch: null, + onCommit: (raws, samples, token, {archives}) async { + if (fail) throw StateError('rollback'); + }, + onArchive: null, + log: (_) {}, + ); + + expect(await d.commit(_tokenA), isTrue); + expect(d.lastTrimAdvanced, isTrue); + + fail = true; + d.onHistoricalRecord(_raw(1), _sample(1)); + expect(await d.commit(_tokenB), isFalse); + // The cursor did NOT move to tokenB, so nothing may claim it did. + expect(d.lastTrimAdvanced, isTrue, reason: 'rolled back to the tokenA state'); + + // And when tokenB is finally committed for real it still counts as an + // ADVANCE. Without the rollback, _lastAckedToken was already tokenB, so + // the real commit reported "no advance" and BackfillContinuation refused + // to auto-continue a drain that had in fact progressed. + fail = false; + expect(await d.commit(_tokenB), isTrue); + expect(d.lastTrimAdvanced, isTrue); + }); + + test('a successful commit clears the buffer and reports durable', () async { + final d = _drainWith((raws, samples, token, {archives}) async {}); + d.onHistoricalRecord(_raw(1), _sample(1)); + + expect(await d.commit(_tokenA), isTrue); + expect(d.bufferedRecords, 0); + }); + }); + + group('P0 — TrimAckPolicy gates the one irreversible act', () { + test('a commit that did not become durable blocks the ACK', () { + expect( + TrimAckPolicy.evaluate( + sessionCurrent: true, + burstDiscarded: false, + commitDurable: false, + ), + TrimAckVerdict.blockedCommitFailed, + ); + }); + + test('a stale session blocks the ACK', () { + expect( + TrimAckPolicy.evaluate( + sessionCurrent: false, + burstDiscarded: false, + commitDurable: true, + ), + TrimAckVerdict.blockedStaleSession, + ); + }); + + test('a discarded burst blocks the ACK even when the commit succeeded', () { + expect( + TrimAckPolicy.evaluate( + sessionCurrent: true, + burstDiscarded: true, + commitDurable: true, + ), + TrimAckVerdict.blockedDiscardedBurst, + ); + }); + + test('a stale session outranks every other reason — nothing may touch the ' + 'new link', () { + expect( + TrimAckPolicy.evaluate( + sessionCurrent: false, + burstDiscarded: true, + commitDurable: false, + ), + TrimAckVerdict.blockedStaleSession, + ); + }); + + test('a discarded burst outranks a durable commit result', () { + expect( + TrimAckPolicy.evaluate( + sessionCurrent: true, + burstDiscarded: true, + commitDurable: false, + ), + TrimAckVerdict.blockedDiscardedBurst, + ); + }); + + test('every precondition holding is the ONLY way to send', () { + expect( + TrimAckPolicy.evaluate( + sessionCurrent: true, + burstDiscarded: false, + commitDurable: true, + ), + TrimAckVerdict.send, + ); + }); + }); + + group('P0 — a discarded burst poisons its HISTORY_END token', () { + test('discardOpenChunk marks the open burst un-ACKable', () async { + final d = _drainWith((raws, samples, token, {archives}) async {}); + d.onHistoricalRecord(_raw(1), _sample(1)); + expect(d.burstDiscarded, isFalse); + + d.discardOpenChunk(); + + // OLD BEHAVIOUR: the records were dropped and NOTHING recorded it, so + // the straggler HISTORY_END (already in flight when the idle watchdog + // fired) committed an empty buffer and echoed the token verbatim — the + // band then trimmed exactly the records that had just been thrown away. + expect(d.burstDiscarded, isTrue); + expect(d.bufferedRecords, 0); + expect( + TrimAckPolicy.evaluate( + sessionCurrent: true, + burstDiscarded: d.burstDiscarded, + commitDurable: true, + ), + TrimAckVerdict.blockedDiscardedBurst, + ); + }); + + test('poisons even when the open buffer is already empty', () { + final d = _drainWith((raws, samples, token, {archives}) async {}); + d.discardOpenChunk(); + expect(d.burstDiscarded, isTrue); + }); + + test('a fresh burst (rearm / HISTORY_START) clears the poison', () { + final d = _drainWith((raws, samples, token, {archives}) async {}); + d.discardOpenChunk(); + expect(d.burstDiscarded, isTrue); + + d.rearm(); + + expect(d.burstDiscarded, isFalse); + expect( + TrimAckPolicy.evaluate( + sessionCurrent: true, + burstDiscarded: d.burstDiscarded, + commitDurable: true, + ), + TrimAckVerdict.send, + ); + }); + + test('poisonedBursts counts once per burst, not once per discard call', () { + final d = _drainWith((raws, samples, token, {archives}) async {}); + d.discardOpenChunk(); + d.discardOpenChunk(); + expect(d.poisonedBursts, 1); + d.rearm(); + d.discardOpenChunk(); + expect(d.poisonedBursts, 2); + }); + }); + + group('BurstTrimGuard', () { + test('starts un-poisoned', () { + expect(BurstTrimGuard().discarded, isFalse); + }); + + test('a discard latches until the next burst begins', () { + final g = BurstTrimGuard(); + g.discardOpenChunk(); + expect(g.discarded, isTrue); + g.beginBurst(); + expect(g.discarded, isFalse); + expect(g.poisonedBursts, 1); + }); + }); + + group('P2 — a corrupt far-future RTC read is not a clock correlation', () { + const wallNow = 1780000000; + + test('a read implausibly far in the future is refused', () { + expect( + ClockPolicy.acceptsClockRead(wallNow + 20 * 365 * 86400, wallNow), + isFalse, + ); + }); + + test('a normal read is accepted', () { + expect(ClockPolicy.acceptsClockRead(wallNow - 12, wallNow), isTrue); + }); + + test('an unset/behind RTC is still accepted — this gate is future-only', () { + // Those go to ClockPolicy.shouldSetClock, which corrects them; refusing + // them here would break the ordinary drift-correction path. + expect(ClockPolicy.acceptsClockRead(1600000000, wallNow), isTrue); + }); + + test('small clock skew inside the future margin is accepted', () { + expect(ClockPolicy.acceptsClockRead(wallNow + 60, wallNow), isTrue); + }); + + test( + 'trusting a corrupt read would arm the wake alarm years out — the exact ' + 'failure the gate prevents', + () { + final corrupt = wallNow + 20 * 365 * 86400; + final ref = ClockRef(device: corrupt, wall: wallNow); + expect(ref.driftSec, lessThan(0)); + + final target = DateTime.fromMillisecondsSinceEpoch(wallNow * 1000) + .add(const Duration(hours: 8)); + final armed = AlarmPayloads.toStrapFrame(target, ref.driftSec); + + // setAlarm does when.subtract(driftSec); a negative drift pushes the + // armed epoch decades into the future, where it silently never fires. + expect(armed.difference(target).inDays, greaterThan(5 * 365)); + // …which is why the read never becomes a ClockRef in the first place. + expect(ClockPolicy.acceptsClockRead(corrupt, wallNow), isFalse); + }, + ); + }); +} diff --git a/test/ble_transport_guards_test.dart b/test/ble_transport_guards_test.dart new file mode 100644 index 00000000..b6f64a92 --- /dev/null +++ b/test/ble_transport_guards_test.dart @@ -0,0 +1,217 @@ +// Regression tests for the BLE transport guards around the offload: +// the process-wide single-owner band claim, link-down teardown, and inbound +// frame routing. + +import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_edge/ble/ble_engine.dart'; +import 'package:openstrap_edge/ble/ble_state.dart'; + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + + group('P1 — the band claim is released when no link ever came up', () { + setUp(BleEngine.resetBandClaimForTest); + tearDown(BleEngine.resetBandClaimForTest); + + test('a failed connect does not leave the claim held', () async { + final logs = []; + final engine = BleEngine( + onRecord: (sample, raw) async {}, + onState: (_) {}, + log: logs.add, + ); + + // flutter_blue_plus is unsupported in the test host, so this exercises + // the real failure path (the throw happens in _doConnect, OUTSIDE the + // block that used to guard it). + final connected = await engine.connectToRemoteId('AA:BB:CC:DD:EE:FF'); + + expect(connected, isFalse); + // OLD BEHAVIOUR: _claimBand() ran BEFORE the link was up and only + // disconnect() ever released it — which nothing calls on a failed + // connect — so _bandOwner stayed pointing at an engine with no link for + // the rest of the process lifetime. + expect(BleEngine.bandClaimed, isFalse); + expect(engine.holdsBandLink, isFalse); + }); + + test( + 'a background drainer is not starved by an earlier failed foreground ' + 'connect', + () async { + final foreground = BleEngine( + onRecord: (sample, raw) async {}, + onState: (_) {}, + ); + await foreground.connectToRemoteId('AA:BB:CC:DD:EE:FF'); + + final drainerLogs = []; + final drainer = BleEngine( + onRecord: (sample, raw) async {}, + onState: (_) {}, + log: drainerLogs.add, + isBackgroundDrainer: true, + ); + await drainer.connectToRemoteId('AA:BB:CC:DD:EE:FF'); + + // OLD BEHAVIOUR: the stale foreground claim was non-null, so every + // later background drain yielded — "strap not reachable this cycle", + // forever. + expect( + drainerLogs.where((l) => l.contains('yielding')), + isEmpty, + reason: 'the drainer must actually attempt the band', + ); + }, + ); + }); + + group('P1 — BandClaimPolicy arbitration', () { + test('an unclaimed band is claimed outright', () { + expect( + BandClaimPolicy.decide( + incumbentPresent: false, + incumbentLive: false, + isBackgroundDrainer: true, + ), + BandClaimDecision.claim, + ); + }); + + test('a STALE claim (owner has no link) is taken, not yielded to', () { + expect( + BandClaimPolicy.decide( + incumbentPresent: true, + incumbentLive: false, + isBackgroundDrainer: true, + ), + BandClaimDecision.claim, + ); + }); + + test('a background drainer yields to a LIVE owner', () { + expect( + BandClaimPolicy.decide( + incumbentPresent: true, + incumbentLive: true, + isBackgroundDrainer: true, + ), + BandClaimDecision.yieldToOwner, + ); + }); + + test('a foreground engine preempts a LIVE owner', () { + expect( + BandClaimPolicy.decide( + incumbentPresent: true, + incumbentLive: true, + isBackgroundDrainer: false, + ), + BandClaimDecision.preemptThenClaim, + ); + }); + + test('a foreground engine takes a stale claim without a preempt round-trip', + () { + expect( + BandClaimPolicy.decide( + incumbentPresent: true, + incumbentLive: false, + isBackgroundDrainer: false, + ), + BandClaimDecision.claim, + ); + }); + }); + + group('P1 — a dropped link tears its session down', () { + test('the current session is torn down, not merely flagged', () { + // OLD BEHAVIOUR: link-down only set connected=false and surfaced `idle`; + // teardown happened solely on the NEXT connect()/disconnect(). When + // BondRefusalGiveUp pauses auto-reconnect neither ever runs, so the dead + // session's five timers kept firing and its four notification + // subscriptions stayed registered — one more set leaked per drop. + expect( + LinkDownPolicy.evaluate(sessionIsCurrent: true), + LinkDownAction.tearDownSession, + ); + }); + + test('a stale session\'s link-down is ignored entirely', () { + expect( + LinkDownPolicy.evaluate(sessionIsCurrent: false), + LinkDownAction.ignoreStaleSession, + ); + }); + }); + + group('P2 — metadata always takes the serialized offload queue', () { + test('metadata on the events characteristic is queued, not run inline', () { + // OLD BEHAVIOUR: only role=='data' metadata reached the queue; metadata + // reassembled on cmd_from/events was fired unawaited on the immediate + // path — the ONE route that could run a HISTORY_END handler concurrently + // with the queued drain, i.e. two handlers on the same DrainController. + expect( + FrameRoutePolicy.route( + isMetadata: true, + isHistorical: false, + isDataRole: false, + ), + FrameRoute.serializedQueue, + ); + }); + + test('metadata on the data characteristic is queued', () { + expect( + FrameRoutePolicy.route( + isMetadata: true, + isHistorical: false, + isDataRole: true, + ), + FrameRoute.serializedQueue, + ); + }); + + test('historical records on the data characteristic are queued', () { + expect( + FrameRoutePolicy.route( + isMetadata: false, + isHistorical: true, + isDataRole: true, + ), + FrameRoute.serializedQueue, + ); + }); + + test('a historical frame off a non-data role keeps the immediate fallback', + () { + expect( + FrameRoutePolicy.route( + isMetadata: false, + isHistorical: true, + isDataRole: false, + ), + FrameRoute.immediate, + ); + }); + + test('live/command frames stay on the immediate path', () { + expect( + FrameRoutePolicy.route( + isMetadata: false, + isHistorical: false, + isDataRole: true, + ), + FrameRoute.immediate, + ); + expect( + FrameRoutePolicy.route( + isMetadata: false, + isHistorical: false, + isDataRole: false, + ), + FrameRoute.immediate, + ); + }); + }); +} diff --git a/test/coach_provider_response_test.dart b/test/coach_provider_response_test.dart new file mode 100644 index 00000000..88d1574c --- /dev/null +++ b/test/coach_provider_response_test.dart @@ -0,0 +1,178 @@ +// CoachEngine.postChat — the ONE provider call every LLM feature in the app +// goes through (coach tool loop, briefings, journal chat). +// +// • Every failure must surface as the documented CoachException. Reaching for +// `choices.first['message'] as Map` unchecked meant any +// OpenAI-compatible proxy returning a streaming (`delta`) or legacy (`text`) +// shape blew up with a raw TypeError, and the user was shown +// "type 'Null' is not a subtype of type 'Map'". +// • The request has a hard size ceiling. The coach's tools read the on-device +// health database and every result is resent on every later turn, so without +// a ceiling a runaway tool loop could serialize the whole database into a +// prompt bound for a third-party endpoint. The ceiling is FAIL-CLOSED: the +// request is refused, never truncated and sent anyway. +import 'dart:convert'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:http/http.dart' as http; +import 'package:http/testing.dart'; +import 'package:openstrap_edge/coach/coach_config.dart'; +import 'package:openstrap_edge/coach/coach_engine.dart'; + +http.Client _json(Object body, {int status = 200}) => MockClient( + (_) async => http.Response( + body is String ? body : jsonEncode(body), + status, + headers: {'content-type': 'application/json; charset=utf-8'}, + ), + ); + +Future> _post(CoachConfig cfg, http.Client c) => + CoachEngine.postChat(cfg, { + 'model': 'gpt-4o-mini', + 'messages': [ + {'role': 'user', 'content': 'hi'} + ], + }, client: c); + +void main() { + late CoachConfig cfg; + setUp(() => cfg = CoachConfig()); + + group('postChat response shapes', () { + test('standard message shape is returned', () async { + final msg = await _post( + cfg, + _json({ + 'choices': [ + { + 'message': {'role': 'assistant', 'content': 'hello'} + } + ] + }), + ); + expect(msg['content'], 'hello'); + }); + + test('a streaming-style `delta` chunk is accepted, not a TypeError', + () async { + final msg = await _post( + cfg, + _json({ + 'choices': [ + { + 'delta': {'role': 'assistant', 'content': 'partial'} + } + ] + }), + ); + expect(msg['content'], 'partial'); + }); + + test('a legacy completions `text` choice is accepted', () async { + final msg = await _post( + cfg, + _json({ + 'choices': [ + {'text': 'legacy'} + ] + }), + ); + expect(msg['content'], 'legacy'); + }); + + for (final entry in { + 'choice with neither message nor delta': { + 'choices': [{}] + }, + 'choice that is not an object': { + 'choices': ['just a string'] + }, + 'message that is not an object': { + 'choices': [ + {'message': 'oops'} + ] + }, + 'no choices key at all': {}, + 'empty choices list': {'choices': []}, + 'top-level JSON array': [1, 2, 3], + }.entries) { + test('throws CoachException (never a TypeError) for ${entry.key}', + () async { + await expectLater( + _post(cfg, _json(entry.value)), + throwsA(isA()), + ); + }); + } + + test('a non-JSON body (an HTML error page / wrong base URL) is a ' + 'CoachException', () async { + await expectLater( + _post(cfg, _json('502 Bad Gateway')), + throwsA(isA()), + ); + }); + + test('a non-200 is a CoachException carrying the provider message', + () async { + await expectLater( + _post( + cfg, + _json({ + 'error': {'message': 'invalid api key'} + }, status: 401), + ), + throwsA(isA().having( + (e) => e.toString(), 'message', contains('invalid api key'))), + ); + }); + }); + + group('postChat request size ceiling', () { + test('refuses an oversized request WITHOUT contacting the provider', + () async { + var called = false; + final client = MockClient((_) async { + called = true; + return http.Response('{}', 200); + }); + final huge = 'x' * (CoachEngine.kMaxRequestBytes + 1024); + await expectLater( + CoachEngine.postChat(cfg, { + 'model': 'gpt-4o-mini', + 'messages': [ + {'role': 'user', 'content': huge} + ], + }, client: client), + throwsA(isA()), + ); + expect(called, isFalse, + reason: 'health data must never leave the device once over the cap'); + }); + + test('a request just under the ceiling still goes out', () async { + final body = 'y' * (CoachEngine.kMaxRequestBytes ~/ 2); + final msg = await CoachEngine.postChat(cfg, { + 'model': 'gpt-4o-mini', + 'messages': [ + {'role': 'user', 'content': body} + ], + }, client: _json({ + 'choices': [ + { + 'message': {'content': 'ok'} + } + ] + })); + expect(msg['content'], 'ok'); + }); + + test('the ceilings are ordered so history can never exceed one request', () { + expect(CoachEngine.kMaxToolResultChars, + lessThan(CoachEngine.kMaxHistoryChars)); + expect(CoachEngine.kMaxHistoryChars, + lessThan(CoachEngine.kMaxRequestBytes)); + }); + }); +} diff --git a/test/coach_sql_guard_adversarial_test.dart b/test/coach_sql_guard_adversarial_test.dart new file mode 100644 index 00000000..9453af17 --- /dev/null +++ b/test/coach_sql_guard_adversarial_test.dart @@ -0,0 +1,147 @@ +// ADVERSARIAL suite for the coach's read-only SQL guard. +// +// The guard is the boundary between on-device health data and a third-party +// LLM endpoint: anything it lets through is serialized into a prompt and +// leaves the device. It is an ALLOW-LIST (only the v_* coach views and CTEs +// declared in the same statement may appear in a table position), so a table +// added to the schema tomorrow is closed by default. +// +// Every case below is an escape attempt. The first one is a verified working +// exploit against the previous FROM/JOIN-only regex: a comma-separated +// implicit cross-join whose SECOND member (on-device GPS coordinates) was +// never examined at all. +import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_edge/coach/coach_db.dart'; + +void main() { + group('CoachDb.guardAndPrepare — allow-list', () { + test('accepts a plain SELECT over an allowed view + auto-LIMITs', () { + final out = CoachDb.guardAndPrepare( + "SELECT date, value FROM v_metric WHERE key='rhr'"); + expect(out.toLowerCase(), contains('from v_metric')); + expect(out.toLowerCase(), contains('limit 200')); + }); + + test('accepts a comma cross-join when BOTH members are allowed views', () { + final out = CoachDb.guardAndPrepare( + 'SELECT d.date FROM v_daily d, v_metric m WHERE d.date = m.date'); + expect(out.toLowerCase(), contains('v_daily')); + }); + + test('accepts a LEFT JOIN between two views with AS aliases', () { + final out = CoachDb.guardAndPrepare('SELECT s.type FROM v_sessions AS s ' + 'LEFT JOIN v_daily AS d ON d.date = s.date'); + expect(out.toLowerCase(), contains('left join v_daily')); + }); + + test('accepts multiple CTEs', () { + final out = CoachDb.guardAndPrepare( + "WITH a AS (SELECT value v FROM v_metric WHERE key='strain'), " + "b AS (SELECT value v FROM v_metric WHERE key='rhr') " + 'SELECT (SELECT AVG(v) FROM a) - (SELECT AVG(v) FROM b)'); + expect(out.toLowerCase(), startsWith('with')); + }); + + test('respects an explicit LIMIT', () { + final out = CoachDb.guardAndPrepare('SELECT * FROM v_daily LIMIT 7'); + expect(RegExp(r'limit\s+200', caseSensitive: false).hasMatch(out), isFalse); + }); + }); + + group('CoachDb.guardAndPrepare — escape attempts', () { + final attempts = { + // ── THE verified exploit: comma cross-join reaching raw GPS. ── + 'comma cross-join to workout_route (raw GPS)': + 'SELECT r.lat, r.lng, r.ts_ms FROM v_sessions s, workout_route r LIMIT 50', + 'comma cross-join to raw_archive': + 'SELECT * FROM v_metric, raw_archive', + 'comma cross-join to sleep_override': + 'SELECT * FROM v_daily d, sleep_override o', + 'comma cross-join to notif_fired': + 'SELECT * FROM v_metric, notif_fired', + 'comma cross-join to sleep_session_candidates': + 'SELECT * FROM v_metric, sleep_session_candidates', + 'comma cross-join to sqlite_schema': + 'SELECT * FROM v_metric, sqlite_schema', + 'comma cross-join across newlines': + 'SELECT *\nFROM v_sessions\n,workout_route', + 'comma cross-join in UPPERCASE': + 'SELECT * FROM V_SESSIONS S, WORKOUT_ROUTE R', + 'three-way comma list with the payload last': + 'SELECT * FROM v_daily a, v_metric b, workout_route c', + // ── explicit joins ── + 'CROSS JOIN to workout_route': + 'SELECT * FROM v_metric CROSS JOIN workout_route', + 'INNER JOIN to decoded_onehz': + 'SELECT * FROM v_metric JOIN decoded_onehz ON 1=1', + // ── subqueries / derived tables ── + 'subquery in FROM': 'SELECT * FROM (SELECT lat FROM workout_route)', + 'subquery in FROM over an allowed view': + 'SELECT * FROM (SELECT date FROM v_daily)', + 'correlated subquery in WHERE reaching a base table': + 'SELECT * FROM v_daily WHERE date IN (SELECT date FROM workout_route)', + 'UNION with a base-table arm': + 'SELECT date FROM v_metric UNION ALL SELECT ts_ms FROM workout_route', + // ── CTE shadowing ── + 'CTE shadowing a real table name': + 'WITH workout_route AS (SELECT 1 x) SELECT * FROM workout_route', + 'CTE shadowing an allowed view': + 'WITH v_metric AS (SELECT 1 x) SELECT * FROM v_metric', + // ── name mangling ── + 'schema-qualified base table': 'SELECT * FROM main.workout_route', + 'schema-qualified view': 'SELECT * FROM main.v_metric', + 'double-quoted identifier': 'SELECT * FROM "workout_route"', + 'bracket-quoted identifier': 'SELECT * FROM [workout_route]', + 'backtick-quoted identifier': 'SELECT * FROM `workout_route`', + // ── functions / internals ── + 'table-valued function in FROM': "SELECT * FROM json_each('[1,2]')", + 'pragma_ function table': "SELECT * FROM pragma_table_info('sessions')", + 'dbstat virtual table': 'SELECT * FROM dbstat', + 'sqlite_master': 'SELECT * FROM sqlite_master', + // ── unknown-by-default (the whole point of an allow-list) ── + 'a table that does not exist yet': 'SELECT * FROM future_secrets', + 'unknown table behind an alias': 'SELECT * FROM v_daily, future_secrets f', + // ── statement shape ── + 'second statement': 'SELECT * FROM v_metric; DROP TABLE sessions', + 'trailing line comment': 'SELECT * FROM v_metric -- , workout_route', + 'block comment': 'SELECT * FROM v_metric /* , workout_route */', + 'unterminated string literal': "SELECT * FROM v_metric WHERE key='rhr", + 'no table reference at all': 'SELECT 1', + 'DELETE': 'DELETE FROM v_metric', + 'UPDATE': 'UPDATE v_daily SET hrv=0', + 'INSERT': 'INSERT INTO v_metric VALUES (1)', + 'PRAGMA': 'PRAGMA table_info(sessions)', + 'ATTACH': 'ATTACH DATABASE x AS y', + 'EXPLAIN prefix': 'EXPLAIN SELECT * FROM v_metric', + }; + + attempts.forEach((label, sql) { + test('rejects: $label', () { + expect(() => CoachDb.guardAndPrepare(sql), + throwsA(isA()), + reason: 'ESCAPED THE GUARD: $sql'); + }); + }); + }); + + group('CoachDb reserved-name net', () { + // These five were verified present in the live schema and absent from the + // old deny-list — the allow-list closes them regardless, but they are + // named here so a rename shows up as a test failure rather than silence. + for (final t in const [ + 'workout_route', + 'raw_archive', + 'notif_fired', + 'sleep_override', + 'sleep_session_candidates', + ]) { + test('reserves the real table $t', () { + expect(CoachDb.reservedTableNames, contains(t)); + }); + } + + test('does not carry the stale non-existent primitive_artifacts entry', () { + expect(CoachDb.reservedTableNames, isNot(contains('primitive_artifacts'))); + }); + }); +} diff --git a/test/coach_sql_structural_test.dart b/test/coach_sql_structural_test.dart new file mode 100644 index 00000000..0ed4ce73 --- /dev/null +++ b/test/coach_sql_structural_test.dart @@ -0,0 +1,109 @@ +// Layer 2 of the coach's read-only surface: the STRUCTURAL btree gate. +// +// The text-level guard (coach_sql_guard_adversarial_test.dart) is a parser, and +// a parser is a model of SQL rather than SQL itself. This gate doesn't parse at +// all — it asks SQLite which btrees a statement would actually open (EXPLAIN's +// OpenRead/ReopenIdx root pages) and refuses anything outside the base tables +// the coach's own views are built from. These tests drive it DIRECTLY, past the +// parser, so a future parser regression can never be the only thing standing +// between an LLM prompt and on-device GPS coordinates. +import 'dart:convert'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; +import 'package:openstrap_edge/coach/coach_db.dart'; +import 'package:openstrap_edge/data/db.dart'; + +void main() { + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + LocalDb.dbName = 'openstrap_coach_structural_test.db'; + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + tearDownAll(() async { + await CoachDb.close(); + await LocalDb.close(); + }); + + test('seeds a session + a route + derived scalars', () async { + final db = await LocalDb.instance; + await db.insert('metric_series', + {'date': '2026-07-01', 'key': 'rhr', 'value': 52.0}); + await db.insert('sessions', { + 'id': 's1', + 'start_ts': 1782000000, + 'end_ts': 1782003600, + 'type': 'run', + 'status': 'done', + 'source': 'local', + 'created_at': 0, + }); + await LocalDb.appendRoutePoints('s1', [ + { + 'session_id': 's1', + 'seq': 0, + 'ts_ms': 1782000000000, + 'lat': 51.5007, + 'lng': -0.1246, + }, + ]); + final n = await db.rawQuery('SELECT COUNT(*) c FROM workout_route'); + expect((n.first['c'] as num).toInt(), 1); + }); + + test('allows a legitimate view query through the btree gate', () async { + await expectLater( + CoachDb.debugAssertAllowedBtrees( + "SELECT date, value FROM v_metric WHERE key='rhr' LIMIT 10"), + completes, + ); + await expectLater( + CoachDb.debugAssertAllowedBtrees( + 'SELECT s.type, d.readiness FROM v_sessions s ' + 'JOIN v_daily d ON d.date = s.date LIMIT 10'), + completes, + ); + }); + + for (final sql in const [ + // THE exploit — comma cross-join onto on-device GPS coordinates. + 'SELECT r.lat, r.lng, r.ts_ms FROM v_sessions s, workout_route r LIMIT 50', + 'SELECT * FROM workout_route LIMIT 50', + 'SELECT * FROM raw_archive LIMIT 50', + 'SELECT * FROM sleep_override LIMIT 50', + 'SELECT name, sql FROM sqlite_master LIMIT 50', + 'SELECT * FROM v_daily WHERE date IN (SELECT session_id FROM workout_route)', + 'SELECT lat FROM workout_route UNION ALL SELECT value FROM v_metric', + ]) { + test('btree gate rejects (parser bypassed): $sql', () async { + await expectLater( + CoachDb.debugAssertAllowedBtrees(sql), + throwsA(isA()), + reason: 'REACHED STORAGE OUTSIDE THE COACH VIEWS: $sql', + ); + }); + } + + test('runCoachSql returns an error (not rows) for the GPS exploit', () async { + final out = await CoachDb.runCoachSql( + 'SELECT r.lat, r.lng, r.ts_ms FROM v_sessions s, workout_route r LIMIT 50'); + final j = jsonDecode(out) as Map; + expect(j.containsKey('error'), isTrue); + expect(j.containsKey('rows'), isFalse); + // No coordinate ever appears in what would be sent to the provider. + expect(out.contains('51.5'), isFalse); + expect(out.contains('-0.12'), isFalse); + }); + + test('runCoachSql still serves the allowed views', () async { + final out = await CoachDb.runCoachSql( + "SELECT date, value FROM v_metric WHERE key='rhr'"); + final j = jsonDecode(out) as Map; + expect(j['row_count'], 1); + expect((j['rows'] as List).first['value'], 52.0); + }); +} diff --git a/test/crossday_pipeline_test.dart b/test/crossday_pipeline_test.dart index 195d97e0..676bc163 100644 --- a/test/crossday_pipeline_test.dart +++ b/test/crossday_pipeline_test.dart @@ -178,5 +178,126 @@ void main() { expect((out['regularity'] as Map)['value'], '—'); expect((out['social_jetlag'] as Map)['value'], '—'); }); + + // ── the SRI grid must WRAP around midnight, not drop the segment ───────── + // + // Segment bounds are mapped to clock-minute-of-day in [0,1440). A segment + // crossing local midnight therefore reads start > end (e.g. 1430 → 20), and + // the old `for (m = startMin; m < endMin; m++)` never executed — silently + // dropping it, despite a comment claiming it "clamps into grid". EVERY + // night has exactly one such segment, so sleep-regularity was always + // computed with a hole right at the boundary. + test('a hypnogram segment crossing local midnight is not dropped from the ' + 'SRI grid', () { + // Local 23:30 → 00:30 the next day, expressed as epoch seconds. + final onset = DateTime(2024, 3, 4, 23, 30).millisecondsSinceEpoch ~/ 1000; + final wake = DateTime(2024, 3, 5, 0, 30).millisecondsSinceEpoch ~/ 1000; + + List> nights({required bool crossMidnight}) => [ + for (var i = 0; i < 14; i++) + { + 'date': '2024-03-${(4 + i).toString().padLeft(2, '0')}', + 'onset_sec': onset + i * 86400, + 'wake_sec': wake + i * 86400, + 'tst_min': 60, + 'hypnogram': [ + { + 'start': onset + i * 86400, + 'end': (crossMidnight ? wake : onset + 1800) + i * 86400, + 'stage': 'nrem', + }, + ], + }, + ]; + + // A midnight-crossing segment must produce REAL coverage — under the old + // clamp the grid stayed entirely uncovered and SRI came back absent. + final crossing = buildCrossDayBundle( + nights(crossMidnight: true), + const {}, + ); + final reg = (crossing['regularity'] as Map).cast(); + expect(reg['value'], isNot('—'), + reason: 'the only segment of each night crosses midnight; dropping ' + 'it leaves the SRI with zero valid epochs'); + + // Sanity: a same-day segment (no wrap) was always handled, and still is. + final sameDay = buildCrossDayBundle( + nights(crossMidnight: false), + const {}, + ); + expect((sameDay['regularity'] as Map)['value'], isNot('—')); + }); + + // ── the resting-HR CUSUM notification's input must actually be emitted ─── + test('recent rows carry rhr so the resting-HR trend notification can fire', + () { + final out = buildCrossDayBundle(_synthDays(30), const {}); + final recent = (out['recent'] as List).cast(); + // DerivationEngine._runNotifications collects `r['rhr'] is num` off these + // rows and needs >= 10 of them; the builder never emitted the field, so + // the series was always empty and the branch was dead code. + final rhrSeries = [ + for (final r in recent) + if (r['rhr'] is num) (r['rhr'] as num).toDouble(), + ]; + expect(rhrSeries.length, 30); + expect(rhrSeries.length, greaterThanOrEqualTo(10)); + }); + + test('a day with no rhr keeps a null rhr (never a fabricated number)', () { + final days = _synthDays(3); + days[1]['rhr'] = null; + final recent = (buildCrossDayBundle(days, const {})['recent'] as List) + .cast(); + expect(recent[1]['rhr'], isNull); + expect(recent[0]['rhr'], isA()); + }); + + // ── CTL/ATL/TSB needs a DENSE per-day series ───────────────────────────── + // + // ctlAtlTsb is an EWMA over ONE SAMPLE PER DAY. Filtering to only the days + // that carry a TRIMP handed it a compressed calendar, so load never decayed + // across rest gaps and TSB was systematically wrong for anyone who trains + // sporadically. + test('rest days are 0-load impulses, not omitted from the load EWMA', () { + // 90 days, but only every 9th day carries a TRIMP (10 loaded days). + final days = >[]; + var dt = DateTime(2024, 1, 1); + for (var i = 0; i < 90; i++) { + days.add({ + 'date': '${dt.year}-${dt.month.toString().padLeft(2, '0')}' + '-${dt.day.toString().padLeft(2, '0')}', + 'rhr': 55.0, + 'rmssd': 45.0, + if (i % 9 == 0) 'trimp': 150.0, + }); + dt = DateTime(dt.year, dt.month, dt.day + 1); + } + final load = ((buildCrossDayBundle(days, const {})['load'] as Map)['value'] + as Map) + .cast(); + final ctl = (load['ctl'] as num).toDouble(); + final atl = (load['atl'] as num).toDouble(); + + // Sparse (old) behaviour handed ctlAtlTsb ten CONSECUTIVE 150s, which + // converges both EWMAs to 150 with no decay between them. Dense (fixed) + // behaviour decays across the 8 rest days after each session, so the + // 7-day ATL in particular must sit far below the session load. + expect(atl, lessThan(100.0), + reason: 'fatigue must decay across 8 consecutive rest days'); + expect(ctl, lessThan(150.0)); + // TSB = ctl - atl must be a real (non-degenerate) form number. + expect((load['tsb'] as num).toDouble(), closeTo(ctl - atl, 1e-6)); + }); + + test('an every-day-trained series is unchanged by densification', () { + // No calendar gaps and a TRIMP on every day → the dense series IS the + // per-row series, so this pins that the fix is a no-op for that case. + final out = buildCrossDayBundle(_synthDays(30), const {}); + final load = ((out['load'] as Map)['value'] as Map).cast(); + expect(load['ctl'], isA()); + expect((load['atl'] as num).toDouble(), greaterThan(50.0)); + }); }); } diff --git a/test/day_window_dst_test.dart b/test/day_window_dst_test.dart new file mode 100644 index 00000000..f4cff983 --- /dev/null +++ b/test/day_window_dst_test.dart @@ -0,0 +1,198 @@ +// LOCAL DAY WINDOWS ACROSS A DST TRANSITION. +// +// `_localDayStartSec(dayId) + 86400` treats every local calendar day as exactly +// 24 h. It isn't: a spring-forward day is 23 h local and a fall-back day is +// 25 h. So on those two days a year the day window either overran into the NEXT +// day (deleteDays silently deleted the following day's first hour of +// decoded_onehz / sessions / band_* / events, and exportDaysDb copied it) or +// fell an hour short (fall-back left the last hour of the day behind). +// +// The host running these tests is very unlikely to sit in a DST zone, so we +// move the PROCESS timezone with libc setenv("TZ")+tzset() before asserting. +// Dart's DateTime reads the C library's local time on every call (it does not +// cache a zone), so this genuinely re-homes the local calendar. POSIX only — +// the test self-skips on Windows. + +import 'dart:ffi'; +import 'dart:io'; + +import 'package:ffi/ffi.dart'; +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; +import 'package:openstrap_edge/data/day_label.dart'; +import 'package:openstrap_edge/data/db.dart'; +import 'package:openstrap_edge/data/models.dart'; + +typedef _SetenvNative = Int32 Function(Pointer, Pointer, Int32); +typedef _SetenvDart = int Function(Pointer, Pointer, int); +typedef _UnsetenvNative = Int32 Function(Pointer); +typedef _UnsetenvDart = int Function(Pointer); +typedef _TzsetNative = Void Function(); +typedef _TzsetDart = void Function(); + +void _setProcessTz(String? tz) { + final lib = DynamicLibrary.process(); + final key = 'TZ'.toNativeUtf8(); + try { + if (tz == null) { + lib.lookupFunction<_UnsetenvNative, _UnsetenvDart>('unsetenv')(key); + } else { + final value = tz.toNativeUtf8(); + lib.lookupFunction<_SetenvNative, _SetenvDart>('setenv')(key, value, 1); + calloc.free(value); + } + lib.lookupFunction<_TzsetNative, _TzsetDart>('tzset')(); + } finally { + calloc.free(key); + } +} + +/// America/New_York: 2026-03-08 springs forward (23 h), 2026-11-01 falls back +/// (25 h). Both are ordinary 24 h days everywhere the flat +86400 was "right". +const _springForward = '2026-03-08'; +const _fallBack = '2026-11-01'; + +Sample _sample(int ts, int counter) => Sample( + tsEpoch: ts, + counter: counter, + hr: 70, + rrIntervalsMs: const [800], + ax: 0, + ay: 0, + az: 0, + spo2RedRaw: 0, + spo2IrRaw: 0, + skinTempRaw: 0, +); + +RawRecord _raw(int ts, int counter) => RawRecord( + counter: counter, + packetType: 47, + hex: 'dst$counter', + capturedAt: ts * 1000, + recTs: ts, +); + +void main() { + final originalTz = Platform.environment['TZ']; + + setUpAll(() async { + _setProcessTz('America/New_York'); + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + LocalDb.dbName = 'openstrap_dst_test.db'; + await databaseFactory.deleteDatabase( + p.join(await databaseFactory.getDatabasesPath(), LocalDb.dbName), + ); + }); + + tearDownAll(() async { + await LocalDb.close(); + await databaseFactory.deleteDatabase( + p.join(await databaseFactory.getDatabasesPath(), LocalDb.dbName), + ); + _setProcessTz(originalTz); + }); + + test('the DST fixture timezone actually applied (guards the whole file)', () { + expect( + DateTime(2026, 3, 8).timeZoneOffset, + const Duration(hours: -5), + reason: 'setenv(TZ)+tzset() did not re-home the local calendar; the ' + 'assertions below would be vacuous', + ); + }, skip: Platform.isWindows ? 'POSIX setenv/tzset only' : null); + + test('localDayEndSec is the next local midnight, not start + 86400', () { + // Spring forward: 23 h. + expect(localDayLengthSec(_springForward), 23 * 3600); + expect( + localDayEndSec(_springForward), + localDayStartSec('2026-03-09'), + reason: 'a day ends exactly where the next one starts', + ); + // Fall back: 25 h. + expect(localDayLengthSec(_fallBack), 25 * 3600); + expect(localDayEndSec(_fallBack), localDayStartSec('2026-11-02')); + // An ordinary day is still 24 h. + expect(localDayLengthSec('2026-06-15'), 86400); + // Month and year rollover still work. + expect(localDayEndSec('2026-01-31'), localDayStartSec('2026-02-01')); + expect(localDayEndSec('2026-12-31'), localDayStartSec('2027-01-01')); + // Malformed labels degrade to null rather than epoch 0. + expect(localDayStartSec('not-a-date'), isNull); + expect(localDayEndSec('2026-06'), isNull); + }, skip: Platform.isWindows ? 'POSIX setenv/tzset only' : null); + + test( + 'deleteDays on a spring-forward day must not eat the NEXT day\'s first hour', + () async { + final springStart = localDayStartSec(_springForward)!; + final nextStart = localDayStartSec('2026-03-09')!; + // With the 23 h day, start + 86400 lands one hour INTO 2026-03-09. + expect(springStart + 86400, nextStart + 3600); + + // A record 30 min into 2026-03-09 — inside the buggy window, outside the + // real one. + final victimTs = nextStart + 1800; + await LocalDb.insertRecord(_raw(victimTs, 5001), _sample(victimTs, 5001)); + // A record safely inside the spring-forward day itself. + final doomedTs = springStart + 3600 * 12; + await LocalDb.insertRecord(_raw(doomedTs, 5002), _sample(doomedTs, 5002)); + + // A session in each, likewise. + await LocalDb.putSession({ + 'id': 'sess-next-day', + 'start_ts': victimTs, + 'end_ts': victimTs + 600, + 'type': 'run', + 'status': 'done', + 'source': 'manual', + 'created_at': victimTs * 1000, + }); + + await LocalDb.deleteDays({_springForward}); + + final db = await LocalDb.instance; + expect( + await db.query('decoded_onehz', where: 'counter = ?', whereArgs: [5002]), + isEmpty, + reason: 'the selected day itself must be deleted', + ); + expect( + await db.query('decoded_onehz', where: 'counter = ?', whereArgs: [5001]), + isNotEmpty, + reason: 'the NEXT local day is not selected and must survive', + ); + expect( + await db.query('sessions', where: 'id = ?', whereArgs: ['sess-next-day']), + isNotEmpty, + ); + }, + skip: Platform.isWindows ? 'POSIX setenv/tzset only' : null, + ); + + test( + 'deleteDays on a fall-back day must not leave the last hour behind', + () async { + final fallStart = localDayStartSec(_fallBack)!; + final fallEnd = localDayEndSec(_fallBack)!; + // With the 25 h day, start + 86400 stops an hour SHORT of local midnight. + expect(fallStart + 86400, fallEnd - 3600); + + final lateTs = fallEnd - 1800; // in the 25th hour + await LocalDb.insertRecord(_raw(lateTs, 5003), _sample(lateTs, 5003)); + + await LocalDb.deleteDays({_fallBack}); + + final db = await LocalDb.instance; + expect( + await db.query('decoded_onehz', where: 'counter = ?', whereArgs: [5003]), + isEmpty, + reason: 'the last local hour of the day belongs to that day', + ); + }, + skip: Platform.isWindows ? 'POSIX setenv/tzset only' : null, + ); +} diff --git a/test/db_migration_ladder_test.dart b/test/db_migration_ladder_test.dart new file mode 100644 index 00000000..7e53f2ab --- /dev/null +++ b/test/db_migration_ladder_test.dart @@ -0,0 +1,296 @@ +// END-TO-END migration-ladder regressions, run against the REAL LocalDb over +// sqflite_ffi. Each test hand-builds a database at an OLD schema version, then +// opens it through LocalDb so sqflite runs the whole onUpgrade ladder. +// +// Why this file exists: `onUpgrade` runs inside ONE exclusive transaction, so a +// single throwing step rolls the whole ladder back and `openDatabase` rethrows. +// The app then has NO recoverable state — it is stuck on the loading screen on +// every launch, permanently. Two steps used a bare `ALTER TABLE … ADD COLUMN` +// against a table that a LATER-numbered `_create*` helper had already created +// with the CURRENT (column-bearing) DDL: +// +// oldV <= 2 : step 3 re-creates raw_records WITH rec_ts, step 6 re-adds it. +// oldV <= 6 : step 7 creates sessions WITH steps, step 11 re-adds it. +// +// Plus the legacy-table migrations (sync_cursor / sync_ledger / sync_quarantine) +// which renamed → created → copied → dropped OUTSIDE any transaction, so a crash +// mid-copy orphaned `
_legacy` forever and silently lost the resumable-sync +// cursor (strap_trim / counter_hw / rec_ts_hw). + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; +import 'package:openstrap_edge/data/db.dart'; + +/// The pre-v3 raw_records shape: keyed by frame hex, NO rec_ts column. +const _legacyRawDdl = ''' + CREATE TABLE raw_records ( + hex TEXT PRIMARY KEY, + counter INTEGER, + packet_type INTEGER, + captured_at INTEGER NOT NULL, + uploaded INTEGER NOT NULL DEFAULT 0 + ) +'''; + +/// The v6-era raw_records shape: hex PK, rec_ts already present. +const _v6RawDdl = ''' + CREATE TABLE raw_records ( + hex TEXT PRIMARY KEY, + counter INTEGER, + packet_type INTEGER, + captured_at INTEGER NOT NULL, + rec_ts INTEGER NOT NULL DEFAULT 0, + uploaded INTEGER NOT NULL DEFAULT 0 + ) +'''; + +/// The v5-era derived tables, so step 9's derived_day → day_result copy is real. +const _v5DerivedDdl = [ + ''' + CREATE TABLE derived_day ( + date TEXT PRIMARY KEY, + payload_json TEXT NOT NULL, + version INTEGER NOT NULL, + last_raw_ts INTEGER NOT NULL, + computed_at INTEGER NOT NULL, + rhr REAL, rmssd REAL, readiness REAL + ) +''', + ''' + CREATE TABLE baselines ( + key TEXT PRIMARY KEY, + payload_json TEXT NOT NULL, + updated_at INTEGER NOT NULL + ) +''', + ''' + CREATE TABLE metric_series ( + date TEXT NOT NULL, key TEXT NOT NULL, value REAL, + PRIMARY KEY (date, key) + ) +''', +]; + +Future _dbPath(String name) async => + p.join(await databaseFactory.getDatabasesPath(), name); + +/// Build a database file at [version] with [ddl] applied, then close it. +Future _seedOldDb( + String name, + int version, + List ddl, { + Future Function(Database db)? seedRows, +}) async { + final path = await _dbPath(name); + await databaseFactory.deleteDatabase(path); + final db = await databaseFactory.openDatabase( + path, + options: OpenDatabaseOptions( + version: version, + onCreate: (db, _) async { + for (final s in ddl) { + await db.execute(s); + } + }, + ), + ); + if (seedRows != null) await seedRows(db); + await db.close(); +} + +/// Open [name] through LocalDb (running the real ladder) and hand back the +/// resulting user_version. +Future _openThroughLocalDb(String name) async { + await LocalDb.close(); + LocalDb.dbName = name; + final db = await LocalDb.instance; + final rows = await db.rawQuery('PRAGMA user_version'); + return (rows.first.values.first as num?)?.toInt() ?? -1; +} + +void main() { + final created = []; + + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + }); + + tearDownAll(() async { + await LocalDb.close(); + for (final n in created) { + await databaseFactory.deleteDatabase(await _dbPath(n)); + } + }); + + test( + 'upgrade from v2 completes — step 3 recreates raw_records WITH rec_ts, ' + 'so step 6 must not re-add it (duplicate column bricked every launch)', + () async { + const name = 'migrate_from_v2_test.db'; + created.add(name); + await _seedOldDb(name, 2, [ + _legacyRawDdl, + 'CREATE TABLE samples (counter INTEGER PRIMARY KEY, ts INTEGER NOT NULL, hr INTEGER)', + '''CREATE TABLE events ( + hex TEXT PRIMARY KEY, event_id INTEGER, ts INTEGER, + captured_at INTEGER NOT NULL)''', + ]); + + // Before the guard this threw + // DatabaseException(duplicate column name: rec_ts) out of openDatabase, + // rolling the whole ladder back — forever, on every launch. + final version = await _openThroughLocalDb(name); + expect(version, LocalDb.schemaVersion); + + final health = await LocalDb.schemaHealth(); + expect(health['ok'], isTrue, reason: '$health'); + }, + ); + + test( + 'upgrade from v6 completes — step 7 creates sessions WITH steps, ' + 'so step 11 must not re-add it', + () async { + const name = 'migrate_from_v6_test.db'; + created.add(name); + await _seedOldDb( + name, + 6, + [ + _v6RawDdl, + 'CREATE TABLE samples (counter INTEGER PRIMARY KEY, ts INTEGER NOT NULL, hr INTEGER)', + '''CREATE TABLE events ( + hex TEXT PRIMARY KEY, event_id INTEGER, ts INTEGER, + captured_at INTEGER NOT NULL)''', + ..._v5DerivedDdl, + ], + seedRows: (db) async { + await db.insert('raw_records', { + 'hex': 'deadbeef', + 'counter': 42, + 'packet_type': 47, + 'captured_at': 1780000000 * 1000, + 'rec_ts': 0, + }); + await db.insert('derived_day', { + 'date': '2026-05-05', + 'payload_json': '{"legacy": true}', + 'version': 1, + 'last_raw_ts': 1780000000, + 'computed_at': 1, + 'rhr': 55.0, + }); + }, + ); + + // Before the guard: DatabaseException(duplicate column name: steps). + final version = await _openThroughLocalDb(name); + expect(version, LocalDb.schemaVersion); + + final health = await LocalDb.schemaHealth(); + expect(health['ok'], isTrue, reason: '$health'); + + // The ladder's real work still happened: derived_day carried across. + final migrated = await LocalDb.dayResult('2026-05-05'); + expect(migrated, isNotNull); + expect(migrated!['payload_json'], '{"legacy": true}'); + + // …and `sessions` has exactly ONE `steps` column, not two. + final db = await LocalDb.instance; + final cols = await db.rawQuery('PRAGMA table_info(sessions)'); + expect(cols.where((c) => c['name'] == 'steps').length, 1); + expect(cols.where((c) => c['name'] == 'hrr_bpm').length, 1); + }, + ); + + test( + 'an orphaned sync_cursor_legacy is RESUMED, not abandoned — a crash ' + 'mid-copy must not silently lose the resumable-sync cursor', + () async { + const name = 'migrate_legacy_resume_test.db'; + created.add(name); + // Exactly the state an interrupted legacy migration leaves behind: the + // NEW-shaped sync_cursor already exists (so the old code's "already + // current" early-return fired and never looked at the orphan), while + // sync_cursor_legacy still holds the rows that were never copied. + await _seedOldDb( + name, + LocalDb.schemaVersion, + [ + '''CREATE TABLE sync_cursor ( + name TEXT PRIMARY KEY, value TEXT, updated_at INTEGER NOT NULL)''', + '''CREATE TABLE sync_cursor_legacy ( + name TEXT PRIMARY KEY, value TEXT, note TEXT)''', + ], + seedRows: (db) async { + // Say the copy died after one of three rows. + await db.insert('sync_cursor', { + 'name': 'strap_trim', + 'value': 'aabbccdd', + 'updated_at': 1, + }); + for (final r in const [ + ['strap_trim', 'aabbccdd'], + ['counter_hw', '1200000'], + ['rec_ts_hw', '1780000000'], + ]) { + await db.insert('sync_cursor_legacy', { + 'name': r[0], + 'value': r[1], + }); + } + }, + ); + + await _openThroughLocalDb(name); + + // Every legacy row is now in the live table… + expect(await LocalDb.getCursor('strap_trim'), 'aabbccdd'); + expect(await LocalDb.getCursorInt('counter_hw'), 1200000); + expect(await LocalDb.getCursorInt('rec_ts_hw'), 1780000000); + // …and the orphan is gone, so this can never run again. + final db = await LocalDb.instance; + final left = await db.rawQuery( + "SELECT name FROM sqlite_master WHERE type='table' " + "AND name='sync_cursor_legacy'", + ); + expect(left, isEmpty); + }, + ); + + test( + 'a legacy migration interrupted BEFORE the CREATE (table missing, legacy ' + 'present) also resumes cleanly', + () async { + const name = 'migrate_legacy_resume2_test.db'; + created.add(name); + await _seedOldDb( + name, + LocalDb.schemaVersion, + [ + '''CREATE TABLE sync_cursor_legacy ( + name TEXT PRIMARY KEY, value TEXT, note TEXT)''', + ], + seedRows: (db) async { + await db.insert('sync_cursor_legacy', { + 'name': 'strap_trim', + 'value': 'feedface', + }); + }, + ); + + await _openThroughLocalDb(name); + expect(await LocalDb.getCursor('strap_trim'), 'feedface'); + final db = await LocalDb.instance; + expect( + await db.rawQuery( + "SELECT name FROM sqlite_master WHERE type='table' " + "AND name='sync_cursor_legacy'", + ), + isEmpty, + ); + }, + ); +} diff --git a/test/db_p0_fixes_test.dart b/test/db_p0_fixes_test.dart new file mode 100644 index 00000000..1fa07cd5 --- /dev/null +++ b/test/db_p0_fixes_test.dart @@ -0,0 +1,471 @@ +// P0 regressions in the LocalDb data layer, run against the REAL LocalDb over +// sqflite_ffi. Each test fails against the pre-fix behaviour. +// +// 1. exportDaysDb had NEVER worked — openDatabase(onCreate:) with no version: +// throws ArgumentError before opening anything; and the decoded_rr copy +// built one `IN (?, …)` per counter (86 400 a day, past +// SQLITE_MAX_VARIABLE_NUMBER). +// 3. the decoded_rr orphan guard covered only the UNIQUE(rec_ts) eviction, not +// the `counter` PRIMARY KEY eviction the strap's reboot counter-reset causes. +// 4. decodedRrByCounterRange assumed counters rise with rec_ts, so a page +// spanning a reboot queried `counter >= high AND counter <= low` → zero rows +// and the whole page's RR beats vanished silently (no RMSSD/HRV). +// 5. deleteDays never cascaded workout_route (deleteSession does), so a deleted +// GPS run left every lat/lng point on disk forever. +// 8. importFromDbFile replayed decoded_onehz through a plain batch.insert, +// bypassing the orphan guard entirely. +// 10. getRecords' day/night counts now come from SQL, never from payloads. +// 11. the timeline's event query was the OLDEST 2000 rows globally, so recent +// days' markers vanished once `events` grew past that. + +import 'dart:io'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:path_provider_platform_interface/path_provider_platform_interface.dart'; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; +import 'package:openstrap_edge/data/db.dart'; +import 'package:openstrap_edge/data/models.dart'; + +/// path_provider has no plugin in a unit test; exportDaysDb needs a temp dir. +class _FakePathProvider extends PathProviderPlatform { + _FakePathProvider(this.root); + final String root; + @override + Future getTemporaryPath() async => root; + @override + Future getApplicationSupportPath() async => root; + @override + Future getApplicationDocumentsPath() async => root; + @override + Future getApplicationCachePath() async => root; + @override + Future getLibraryPath() async => root; + @override + Future getDownloadsPath() async => root; +} + +Sample _sample(int ts, int counter, List rr) => Sample( + tsEpoch: ts, + counter: counter, + hr: 70, + rrIntervalsMs: rr, + ax: 0, + ay: 0, + az: 0, + spo2RedRaw: 0, + spo2IrRaw: 0, + skinTempRaw: 0, +); + +RawRecord _raw(int ts, int counter) => RawRecord( + counter: counter, + packetType: 47, + hex: 'p0fix$counter', + capturedAt: ts * 1000, + recTs: ts, +); + +void main() { + late Directory tmp; + + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + tmp = await Directory.systemTemp.createTemp('openstrap_p0_'); + PathProviderPlatform.instance = _FakePathProvider(tmp.path); + LocalDb.dbName = 'openstrap_p0_fixes_test.db'; + await databaseFactory.deleteDatabase( + p.join(await databaseFactory.getDatabasesPath(), LocalDb.dbName), + ); + }); + + tearDownAll(() async { + await LocalDb.close(); + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + await databaseFactory.deleteDatabase(p.join(dir, 'p0_foreign_export.db')); + if (await tmp.exists()) await tmp.delete(recursive: true); + }); + + // ── fix 3 ──────────────────────────────────────────────────────────────── + test( + 'a REUSED counter (reboot reset) leaves no stale-timestamped decoded_rr ' + 'beats behind — the counter-PK eviction is guarded too', + () async { + const older = 1785000000; + const newer = 1785000600; // a DIFFERENT second, same counter + const counter = 777; + + await LocalDb.insertRecord( + _raw(older, counter), + _sample(older, counter, [800, 810, 820]), // THREE beats + ); + // Post-reboot the counter is handed out again, now for a later second. + // The INSERT-OR-REPLACE evicts the older second's decoded_onehz row via + // the `counter` PRIMARY KEY; only beat_index 0 and 1 are overwritten, so + // beat_index 2 used to SURVIVE still stamped with `older` — invisible to + // both prune paths, and it polluted every later RR read of that counter. + await LocalDb.insertRecord( + _raw(newer, counter), + _sample(newer, counter, [900, 910]), // TWO beats + ); + + final db = await LocalDb.instance; + final beats = await db.query( + 'decoded_rr', + where: 'counter = ?', + whereArgs: [counter], + orderBy: 'beat_index ASC', + ); + expect(beats, hasLength(2), reason: 'the third beat must not survive'); + expect( + beats.every((b) => b['rr_ts_ms'] == newer * 1000), + isTrue, + reason: 'no beat may carry the evicted second\'s timestamp: $beats', + ); + expect([for (final b in beats) b['rr_ms']], [900, 910]); + + // And globally: no orphans, no cross-second contamination. + final stale = await db.rawQuery( + 'SELECT COUNT(*) c FROM decoded_rr rr ' + 'JOIN decoded_onehz d ON d.counter = rr.counter ' + 'WHERE rr.rr_ts_ms != d.rec_ts * 1000', + ); + expect(stale.first['c'], 0); + }, + ); + + // ── fix 4 ──────────────────────────────────────────────────────────────── + test( + 'decodedRrByCounterRange returns a page spanning a counter RESET — the ' + 'endpoints are page bounds, not a monotonic counter span', + () async { + const t0 = 1785100000; + // Pre-reboot: high counter. Post-reboot: the counter restarts near zero, + // for the NEXT second — exactly what a page ordered by (rec_ts, counter) + // straddles. + await LocalDb.insertRecord( + _raw(t0, 1200000), + _sample(t0, 1200000, [800, 805]), + ); + await LocalDb.insertRecord( + _raw(t0 + 1, 5), + _sample(t0 + 1, 5, [900, 905]), + ); + + // Read the page exactly as derivation_engine does. + final page = await LocalDb.decodedOneHzBatchByRecTsRange( + limit: 100, + fromRecTs: t0, + toRecTs: t0 + 1, + ); + expect(page, hasLength(2)); + final first = (page.first['counter'] as num).toInt(); + final last = (page.last['counter'] as num).toInt(); + expect(first, 1200000); + expect(last, 5, reason: 'the page really does end on a LOWER counter'); + + final rr = await LocalDb.decodedRrByCounterRange( + fromCounter: first, + toCounter: last, + ); + // `counter >= 1200000 AND counter <= 5` used to match nothing at all — + // the window silently produced no RR beats, so no RMSSD/HRV, no error. + expect(rr, hasLength(4)); + expect([for (final r in rr) r['rr_ms']], [800, 805, 900, 905]); + }, + ); + + // ── fix 5 ──────────────────────────────────────────────────────────────── + test('deleteDays cascades workout_route with its session', () async { + const dayId = '2026-04-10'; + final startSec = + DateTime(2026, 4, 10, 9).millisecondsSinceEpoch ~/ 1000; + await LocalDb.putSession({ + 'id': 'run-with-route', + 'start_ts': startSec, + 'end_ts': startSec + 1800, + 'type': 'run', + 'status': 'done', + 'source': 'manual', + 'created_at': startSec * 1000, + }); + await LocalDb.appendRoutePoints('run-with-route', [ + for (var i = 0; i < 5; i++) + { + 'session_id': 'run-with-route', + 'seq': i, + 'ts_ms': (startSec + i) * 1000, + 'lat': 12.97 + i * 0.001, + 'lng': 77.59 + i * 0.001, + }, + ]); + expect(await LocalDb.sessionHasRoute('run-with-route'), isTrue); + + await LocalDb.deleteDays({dayId}); + + final db = await LocalDb.instance; + expect( + await db.query('sessions', where: 'id = ?', whereArgs: ['run-with-route']), + isEmpty, + ); + expect( + await db.query( + 'workout_route', + where: 'session_id = ?', + whereArgs: ['run-with-route'], + ), + isEmpty, + reason: 'every lat/lng point of a deleted day must go with it', + ); + }); + + // ── fix 11 ─────────────────────────────────────────────────────────────── + test( + 'eventsInRange is bounded BY THE DAY — a day past the oldest-2000 page is ' + 'still reachable', + () async { + final db = await LocalDb.instance; + await db.delete('events'); + final base = DateTime(2026, 5, 1).millisecondsSinceEpoch ~/ 1000; + // 2400 old events, then 3 on a much later day. + final batch = db.batch(); + for (var i = 0; i < 2400; i++) { + batch.insert('events', { + 'hex': 'old$i', + 'event_id': 1, + 'ts': base + i, + 'captured_at': (base + i) * 1000, + }); + } + final lateDayStart = + DateTime(2026, 5, 20).millisecondsSinceEpoch ~/ 1000; + for (var i = 0; i < 3; i++) { + batch.insert('events', { + 'hex': 'new$i', + 'event_id': 2, + 'ts': lateDayStart + 3600 * (i + 1), + 'captured_at': (lateDayStart + 3600 * (i + 1)) * 1000, + }); + } + await batch.commit(noResult: true); + + // The OLD path: oldest-2000 globally, then filtered to the day → nothing. + final oldest = await LocalDb.unuploadedEvents(limit: 2000); + final viaOldPath = oldest.where( + (e) => + (e['ts'] as num) >= lateDayStart && + (e['ts'] as num) < lateDayStart + 86400, + ); + expect( + viaOldPath, + isEmpty, + reason: 'documents the bug the day-bounded query replaces', + ); + + final viaNewPath = await LocalDb.eventsInRange( + lateDayStart, + lateDayStart + 86400, + ); + expect(viaNewPath, hasLength(3)); + expect(viaNewPath.every((e) => e['event_id'] == 2), isTrue); + }, + ); + + // ── fix 10 ─────────────────────────────────────────────────────────────── + test( + 'dayResultDayIdsDesc / daysWithSleepTst answer in SQL, latest version only', + () async { + final db = await LocalDb.instance; + await db.delete('day_result'); + await LocalDb.putDayResult( + dayId: '2026-02-01', + algoVersion: 40, + payloadJson: '{"sleep":{"accounting":{"value":{"tst_sec":21600}}}}', + windowJson: '{}', + ); + // A LATER version of the same day that lost its sleep block — the latest + // version is the one that counts. + await LocalDb.putDayResult( + dayId: '2026-02-01', + algoVersion: 41, + payloadJson: '{"sleep":{"accounting":{"value":{"tst_sec":25200}}}}', + windowJson: '{}', + ); + await LocalDb.putDayResult( + dayId: '2026-02-02', + algoVersion: 41, + payloadJson: '{"sleep":{"accounting":{"value":{}}}}', + windowJson: '{}', + ); + // A corrupt payload must degrade to "no sleep", never take the query out. + await LocalDb.putDayResult( + dayId: '2026-02-03', + algoVersion: 41, + payloadJson: 'not json at all', + windowJson: '{}', + ); + + expect(await LocalDb.dayResultDayIdsDesc(), [ + '2026-02-03', + '2026-02-02', + '2026-02-01', + ]); + expect(await LocalDb.daysWithSleepTst(), {'2026-02-01'}); + }, + ); + + // ── fix 8 ──────────────────────────────────────────────────────────────── + test( + 'importFromDbFile routes decoded_onehz through the orphan guard', + () async { + final db = await LocalDb.instance; + await db.delete('decoded_onehz'); + await db.delete('decoded_rr'); + + const collideTs = 1786000000; // rec_ts collision, different counter + const reuseCounter = 8003; // counter collision, different rec_ts + const localReuseTs = 1786000500; + const foreignReuseTs = 1786009999; + + await LocalDb.insertRecord( + _raw(collideTs, 8002), + _sample(collideTs, 8002, [700, 710, 720]), + ); + await LocalDb.insertRecord( + _raw(localReuseTs, reuseCounter), + _sample(localReuseTs, reuseCounter, [600, 610, 620]), + ); + + // A foreign export that collides both ways. + final dir = await databaseFactory.getDatabasesPath(); + final srcPath = p.join(dir, 'p0_foreign_export.db'); + await databaseFactory.deleteDatabase(srcPath); + final src = await databaseFactory.openDatabase(srcPath); + await src.execute(''' + CREATE TABLE decoded_onehz ( + counter INTEGER PRIMARY KEY, rec_ts INTEGER NOT NULL, + hr INTEGER NOT NULL, ax REAL NOT NULL, ay REAL NOT NULL, + az REAL NOT NULL, spo2_red_raw INTEGER NOT NULL, + spo2_ir_raw INTEGER NOT NULL, skin_temp_raw INTEGER NOT NULL) + '''); + await src.execute(''' + CREATE TABLE decoded_rr ( + counter INTEGER NOT NULL, beat_index INTEGER NOT NULL, + rr_ts_ms INTEGER NOT NULL, rr_ms INTEGER NOT NULL, + PRIMARY KEY (counter, beat_index)) + '''); + Future foreign(int counter, int recTs, List rr) async { + await src.insert('decoded_onehz', { + 'counter': counter, + 'rec_ts': recTs, + 'hr': 61, + 'ax': 0.0, + 'ay': 0.0, + 'az': 0.0, + 'spo2_red_raw': 0, + 'spo2_ir_raw': 0, + 'skin_temp_raw': 0, + }); + for (var i = 0; i < rr.length; i++) { + await src.insert('decoded_rr', { + 'counter': counter, + 'beat_index': i, + 'rr_ts_ms': recTs * 1000, + 'rr_ms': rr[i], + }); + } + } + + await foreign(8001, collideTs, [500]); // same second, other counter + await foreign(reuseCounter, foreignReuseTs, [400]); // same counter, other second + await src.close(); + + await LocalDb.importFromDbFile(srcPath); + + // (a) UNIQUE(rec_ts) eviction: the local counter's beats went with it. + expect( + await db.query('decoded_rr', where: 'counter = ?', whereArgs: [8002]), + isEmpty, + reason: 'the evicted counter\'s beats must not be stranded', + ); + // (b) counter-PK eviction: no beat under the reused counter still carries + // the local second's timestamp. + final reused = await db.query( + 'decoded_rr', + where: 'counter = ?', + whereArgs: [reuseCounter], + ); + expect(reused, hasLength(1)); + expect(reused.first['rr_ts_ms'], foreignReuseTs * 1000); + + // Nothing orphaned, nothing cross-stamped, anywhere. + final orphans = await db.rawQuery( + 'SELECT COUNT(*) c FROM decoded_rr ' + 'WHERE counter NOT IN (SELECT counter FROM decoded_onehz)', + ); + expect(orphans.first['c'], 0); + final stale = await db.rawQuery( + 'SELECT COUNT(*) c FROM decoded_rr rr ' + 'JOIN decoded_onehz d ON d.counter = rr.counter ' + 'WHERE rr.rr_ts_ms != d.rec_ts * 1000', + ); + expect(stale.first['c'], 0); + }, + ); + + // ── fix 1 ──────────────────────────────────────────────────────────────── + test( + 'exportDaysDb actually produces a database, and copies well past ' + 'SQLITE_MAX_VARIABLE_NUMBER counters of RR', + () async { + final db = await LocalDb.instance; + await db.delete('decoded_onehz'); + await db.delete('decoded_rr'); + + const dayId = '2026-03-20'; + final dayStart = DateTime(2026, 3, 20).millisecondsSinceEpoch ~/ 1000; + const n = 1200; // > SQLITE_MAX_VARIABLE_NUMBER's 999 floor + await LocalDb.commitSyncBatch( + [for (var i = 0; i < n; i++) _raw(dayStart + i, 90000 + i)], + [for (var i = 0; i < n; i++) _sample(dayStart + i, 90000 + i, [800])], + ); + await LocalDb.putDayResult( + dayId: dayId, + algoVersion: 41, + payloadJson: '{"exported":true}', + windowJson: '{}', + ); + + // Before the fix this threw ArgumentError('onCreate must be null if no + // version is specified') — the export had never once produced a file. + final path = await LocalDb.exportDaysDb({dayId}); + expect(await File(path).exists(), isTrue); + + final out = await databaseFactory.openDatabase( + path, + options: OpenDatabaseOptions(readOnly: true, singleInstance: false), + ); + try { + int count(List> r) => + (r.first.values.first as num).toInt(); + expect( + count(await out.rawQuery('SELECT COUNT(*) FROM decoded_onehz')), + n, + ); + expect( + count(await out.rawQuery('SELECT COUNT(*) FROM decoded_rr')), + n, + reason: 'the chunked IN() must not drop any counter', + ); + expect( + count(await out.rawQuery('SELECT COUNT(*) FROM day_result')), + 1, + ); + } finally { + await out.close(); + } + }, + ); +} diff --git a/test/derive_day_window_test.dart b/test/derive_day_window_test.dart new file mode 100644 index 00000000..9b568d25 --- /dev/null +++ b/test/derive_day_window_test.dart @@ -0,0 +1,234 @@ +// Regression tests for three quieter derivation defects: +// +// * the deliberate widening of the nocturnal search window was a NO-OP, +// because the coordinator only ever LOADED substrate back to the previous +// 18:00 while the day model searches from the previous NOON — so +// `searchStart = max(dataStart, …)` clipped it straight back and any sleep +// onset before 18:00 was truncated to the slice start; +// * the habitual-midsleep prior converted HISTORICAL sleep blocks using the +// CURRENT UTC offset, so re-deriving days from the other side of a DST +// transition (or a trip) shifted them by an hour and could change which +// candidate sleep was selected; +// * `_buildWakeDayFeatures` substituted age 30 / 70 kg / sex 'm' / RHR 60 for +// a user who never entered a profile and then PERSISTED strain / calories / +// calories_total as real scalars — fabricated numbers wearing real numbers' +// clothes, against the never-impute contract the rest of the layer keeps. + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; +import 'package:openstrap_edge/compute/derivation_engine.dart'; +import 'package:openstrap_edge/compute/profile.dart'; +import 'package:openstrap_edge/compute/substrate.dart'; +import 'package:openstrap_edge/data/db.dart'; + +/// A flat 1 Hz substrate over [durSec] starting at [startSec], HR [hr]. +Substrate _synthDay(int startSec, int durSec, {int hr = 82}) { + final ts = []; + final hrs = []; + final ax = [], ay = [], az = []; + for (var i = 0; i < durSec; i++) { + ts.add(startSec + i); + hrs.add(hr + (i % 7)); // gentle deterministic variation, never 0 + // Enough orientation change to produce real motion minutes. + ax.add(0.05 * ((i % 60) / 60.0)); + ay.add(0.05 * ((i % 30) / 30.0)); + az.add(0.98); + } + final n = ts.length; + return Substrate( + tsSec: ts, + hr: hrs, + rrTsMs: const [], + rrMs: const [], + ax: ax, + ay: ay, + az: az, + spo2Red: List.filled(n, 0), + spo2Ir: List.filled(n, 0), + skinTemp: List.filled(n, 3000), + skinContact: List.filled(n, 0), + ); +} + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + LocalDb.dbName = 'openstrap_day_window_test.db'; + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + tearDownAll(() async { + await LocalDb.close(); + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + // ── the loaded window must cover the whole documented search window ──────── + + group('target-day substrate window', () { + test('reaches back to the previous local NOON, not 18:00', () { + const dayId = '2026-04-10'; + final dayStart = DateTime(2026, 4, 10).millisecondsSinceEpoch ~/ 1000; + final dayEnd = DateTime(2026, 4, 11).millisecondsSinceEpoch ~/ 1000; + + final (from, to) = DerivationEngine().debugTargetDayWindow(dayId); + + expect(dayStart - from, kNocturnalSearchLookbackSec, + reason: 'calendarDays searches from dayStart − ' + 'kNocturnalSearchLookbackSec; loading less means ' + '`searchStart = max(dataStart, …)` silently clips it back'); + expect(dayStart - from, 12 * 3600, reason: 'the previous local NOON'); + expect(from, lessThan(dayStart - 6 * 3600), + reason: 'strictly wider than the old prev-18:00 window, which made ' + 'the documented widening a no-op'); + expect(to, dayEnd - 1); + }); + + test('the constant the loader uses is the one the day model searches with', + () { + // Two call sites, one constant — they cannot drift apart again. + const dayId = '2026-10-25'; // a European DST-transition date + final dayStart = DateTime(2026, 10, 25).millisecondsSinceEpoch ~/ 1000; + final (from, _) = DerivationEngine().debugTargetDayWindow(dayId); + expect(from, dayStart - kNocturnalSearchLookbackSec); + }); + }); + + // ── the habitual-midsleep prior is resolved AT THE DAY, not at "now" ─────── + + group('historical timezone offset', () { + test('tzOffsetSecondsAt resolves per instant, not once for today', () { + final jan = DateTime(2020, 1, 15, 3, 0).millisecondsSinceEpoch ~/ 1000; + final jul = DateTime(2020, 7, 15, 3, 0).millisecondsSinceEpoch ~/ 1000; + for (final t in [jan, jul]) { + expect( + tzOffsetSecondsAt(t), + DateTime.fromMillisecondsSinceEpoch(t * 1000, isUtc: false) + .timeZoneOffset + .inSeconds, + reason: 'must ask the platform for the offset that applied THEN', + ); + } + // In a DST-observing zone the two instants disagree; a constant + // "today's offset" cannot equal both. + final janOff = tzOffsetSecondsAt(jan); + final julOff = tzOffsetSecondsAt(jul); + if (janOff != julOff) { + final nowOff = DateTime.now().timeZoneOffset.inSeconds; + expect(janOff == nowOff && julOff == nowOff, isFalse); + } + }); + + test('calendarDays resolves the offset AT the day being segmented', () { + // Zone-independent: inject the resolver and inspect what it was asked + // for. The old code never asked at all — it read + // `DateTime.now().timeZoneOffset`, a constant applied to every + // historical day regardless of the offset actually in effect then. + final dayStart = DateTime(2020, 1, 15).millisecondsSinceEpoch ~/ 1000; + final sub = _synthDay(dayStart + 3600, 900); + final asked = []; + final days = calendarDays( + sub, + // An override forces the segmentation branch (and therefore the + // habitual-midsleep prior) to run on a short synthetic capture. + override: SleepWindowOverride( + dayId: '2020-01-15', + onsetSec: dayStart + 3700, + offsetSec: dayStart + 4300, + source: 'manual', + ), + tzOffsetAt: (t) { + asked.add(t); + return DateTime.fromMillisecondsSinceEpoch(t * 1000).timeZoneOffset + .inSeconds; + }, + ); + + expect(days, isNotEmpty); + expect(asked, isNotEmpty, + reason: 'the offset must be RESOLVED per day, not read off ' + 'DateTime.now() once'); + final nowSec = DateTime.now().millisecondsSinceEpoch ~/ 1000; + for (final t in asked) { + expect(t, dayStart, + reason: 'resolved at the local midnight of the day being ' + 'segmented'); + expect((t - nowSec).abs(), greaterThan(86400), + reason: 'a historical instant, emphatically not "now"'); + } + }); + }); + + // ── never impute a profile ──────────────────────────────────────────────── + + group('absent profile abstains instead of imputing', () { + // 2 h of daytime 1 Hz data on one calendar day — no sleep, so every minute + // is wake and the wake-day feature block runs in full. + final dayStart = DateTime(2026, 4, 10).millisecondsSinceEpoch ~/ 1000; + + Future> deriveWith( + Profile profile, + String dayLabel, + int localDayStart, + ) async { + final sub = _synthDay(localDayStart + 9 * 3600, 2 * 3600); + await DerivationEngine() + .deriveImportedDays(sub, profile, {dayLabel}); + final out = {}; + for (final key in const [ + 'strain', + 'trimp', + 'calories', + 'calories_total', + 'steps', + ]) { + out[key] = await LocalDb.metricValueOn(dayLabel, key); + } + return out; + } + + test('no profile → no strain, no calories, no TDEE', () async { + final got = await deriveWith(const Profile(), '2026-04-10', dayStart); + expect(got['strain'], isNull, + reason: 'Banister TRIMP needs a real resting HR, HRmax and sex — ' + 'age 30 / RHR 60 / sex m were fabricated'); + expect(got['calories'], isNull, + reason: 'Keytel needs real age, weight and sex'); + expect(got['calories_total'], isNull, + reason: 'Mifflin BMR needs real anthropometrics'); + }); + + test('steps still compute without a profile (data-derived, not imputed)', + () async { + // `dailyStepEstimate` falls back to the day's own 10th-percentile HR when + // no resting HR is known — that is derived from the data, so abstaining + // would be over-correction. + final got = await deriveWith(const Profile(), '2026-04-11', + DateTime(2026, 4, 11).millisecondsSinceEpoch ~/ 1000); + expect(got['steps'], isNotNull); + }); + + test('a real profile still produces strain and calories', () async { + final got = await deriveWith( + const Profile( + ageYears: 34, + weightKg: 72, + heightCm: 178, + sex: 'm', + restingHrManual: 55, + ), + '2026-04-12', + DateTime(2026, 4, 12).millisecondsSinceEpoch ~/ 1000, + ); + expect(got['strain'], isNotNull, + reason: 'the abstention must be about MISSING inputs only'); + expect(got['calories'], isNotNull); + expect(got['calories_total'], isNotNull); + }); + }); +} diff --git a/test/derive_isolate_lifecycle_test.dart b/test/derive_isolate_lifecycle_test.dart new file mode 100644 index 00000000..ab544a3b --- /dev/null +++ b/test/derive_isolate_lifecycle_test.dart @@ -0,0 +1,237 @@ +// Regression tests for the derivation engine's ISOLATE LIFECYCLE. +// +// Every one of these failures had the same shape: a worker isolate dies or +// hangs, the main side awaits a Completer that can never complete, `_running` +// stays true, and `DeriveScheduler._drain` never returns — ALL derivation is +// dead until the app is restarted. The engine never sees an error, so it never +// even logs one. +// +// * `_loadSubstrateRange` spawned the prepare worker with NO onError/onExit +// port and put NO timeout on the result, while the worker itself only +// reported errors from its 'finish' branch — so any throw in its 'page' +// handler (an unguarded numeric read over a SQLite row) killed it silently. +// * `Isolate.run(...).timeout(...)` only stops the CALLER waiting; the isolate +// keeps burning a core behind the bounded worker pool's back. And the +// sleep-staging site had no timeout at all. + +import 'dart:async'; +import 'dart:isolate'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_edge/compute/derivation_engine.dart'; +import 'package:openstrap_edge/compute/derive_prepare.dart'; + +/// Drive the real prepare worker over a caller-supplied page sequence and +/// return whichever came first: a result payload, or an error. Mirrors +/// `_loadSubstrateRange`'s wiring (onError + onExit + a bounded wait) so the +/// worker's own failure contract is what's under test. +Future _drivePrepareWorker( + List> messages, { + Duration timeout = const Duration(seconds: 10), +}) async { + final port = ReceivePort(); + final isolate = await Isolate.spawn( + derivationPrepareWorker, + port.sendPort, + onError: port.sendPort, + onExit: port.sendPort, + ); + final ready = Completer(); + final done = Completer(); + late final StreamSubscription sub; + void finish(Object? v) { + if (!done.isCompleted) done.complete(v); + } + + sub = port.listen((message) { + if (message is SendPort) { + if (!ready.isCompleted) ready.complete(message); + return; + } + if (message is Map && message['type'] == 'result') { + finish(message['payload']); + return; + } + if (message is Map && message['type'] == 'error') { + finish(StateError('worker error: ${message['error']}')); + return; + } + if (message is List) { + finish(StateError('worker crashed: ${message.first}')); + return; + } + if (message == null) finish(StateError('worker exited without a result')); + }); + try { + final worker = await ready.future.timeout(timeout); + for (final m in messages) { + worker.send(m); + } + return await done.future.timeout(timeout); + } finally { + await sub.cancel(); + port.close(); + isolate.kill(priority: Isolate.immediate); + } +} + +void main() { + group('prepare worker never dies silently', () { + test('a malformed decoded row is reported, not swallowed into a hang', + () async { + // `hr` arrives as a String. SQLite storage classes are per-VALUE, so a row + // written by an older/importing path really can do this — and the old + // `row['hr'] as num?` threw inside the 'page' handler, whose only error + // reporting lived in the (never-reached) 'finish' branch. + final out = await _drivePrepareWorker([ + const {'type': 'config', 'mode': 'substrate'}, + { + 'type': 'page', + 'frames': [ + { + 'rec_ts': 1780000000, + 'hr': 'not-a-number', + 'ax': 0.0, + 'ay': 0.0, + 'az': 1.0, + 'counter': 1, + } + ], + 'rr': const [], + }, + const {'type': 'finish'}, + ]); + // Either it is now tolerated (guarded numeric read) or it is REPORTED — + // the one unacceptable outcome is the wait never ending, which the + // enclosing timeout would surface as a TimeoutException. + expect(out, isNot(isA())); + expect(out, isA(), + reason: 'the guarded read treats a non-numeric cell as absent rather ' + 'than killing the worker'); + }); + + test('a page that throws in the worker fails the wait instead of hanging', + () async { + // `frames` non-empty but a frame is not a Map at all -> whereType filters + // it, so this exercises the ordinary path; the load-bearing assertion is + // simply that the call always TERMINATES. + final out = await _drivePrepareWorker([ + const {'type': 'config', 'mode': 'substrate'}, + const { + 'type': 'page', + 'frames': [42, 'nonsense'], + 'rr': [], + }, + const {'type': 'finish'}, + ]); + expect(out, isA()); + }); + + test('a well-formed page still decodes to a real substrate', () async { + final out = await _drivePrepareWorker([ + const {'type': 'config', 'mode': 'substrate'}, + { + 'type': 'page', + 'frames': [ + for (var i = 0; i < 5; i++) + { + 'rec_ts': 1780000000 + i, + 'hr': 60 + i, + 'ax': 0.0, + 'ay': 0.0, + 'az': 1.0, + 'spo2_red_raw': 10, + 'spo2_ir_raw': 20, + 'skin_temp_raw': 3000, + 'counter': i, + } + ], + 'rr': const [], + }, + const {'type': 'finish'}, + ]); + final payload = (out as Map).cast(); + expect((payload['ts_sec'] as List).length, 5); + expect((payload['hr'] as List).first, 60); + }); + }); + + group('runCancellableIsolate', () { + test('returns the computed value', () async { + final v = await runCancellableIsolate( + () => 6 * 7, + const Duration(seconds: 10), + ); + expect(v, 42); + }); + + test('a result that is itself a List is not mistaken for an error', () { + // The uncaught-error wire format is a 2-element List and `onExit` sends + // null — a naive protocol would misread either as failure. + expect( + runCancellableIsolate>( + () => ['boom', 'stack'], + const Duration(seconds: 10), + ), + completion(equals(['boom', 'stack'])), + ); + }); + + test('a null result is not mistaken for a silent exit', () { + expect( + runCancellableIsolate(() => null, const Duration(seconds: 10)), + completion(isNull), + ); + }); + + test('a throw inside the isolate surfaces as an error, not a hang', () { + expect( + runCancellableIsolate( + () => throw StateError('kaboom'), + const Duration(seconds: 10), + ), + throwsA(isA()), + ); + }); + + test('an isolate that ends without answering fails fast, never hangs', () { + // Nothing left on its event loop -> the VM tears the isolate down. With + // no `onExit` port wired (the old `_loadSubstrateRange`) that produced + // total silence and an eternal await; now it is a hard error. + expect( + runCancellableIsolate( + () => Completer().future, // no pending events -> isolate exits + const Duration(seconds: 10), + label: 'dead', + ), + throwsA(isA()), + ); + }); + + test('a wedged-but-alive computation TIMES OUT (and is killed)', () async { + // The defect this pins: `Isolate.run` with no timeout at all — the + // sleep-staging site — left the caller awaiting forever with + // `_running == true`, so DeriveScheduler._drain never returned again. + final sw = Stopwatch()..start(); + await expectLater( + runCancellableIsolate( + () async { + // An open ReceivePort keeps the isolate's event loop alive, so it + // stays running (exactly like a real hung compute) rather than + // exiting and tripping the onExit path above. + final keepAlive = ReceivePort(); + await Completer().future; + keepAlive.close(); + return 0; + }, + const Duration(milliseconds: 400), + label: 'wedged', + ), + throwsA(isA()), + ); + sw.stop(); + expect(sw.elapsed, lessThan(const Duration(seconds: 8)), + reason: 'the wait is bounded by the timeout, not by the isolate'); + }); + }); +} diff --git a/test/derive_result_protection_test.dart b/test/derive_result_protection_test.dart new file mode 100644 index 00000000..fc644444 --- /dev/null +++ b/test/derive_result_protection_test.dart @@ -0,0 +1,304 @@ +// Regression tests for the two ways a derivation pass could DESTROY a good +// day_result — both of which are permanent, because `putDayResult` is +// ConflictAlgorithm.replace on BOTH `day_result` AND `metric_series` (so every +// scalar for the date is NULLed), raw is pruned after 3 days (so there is +// nothing left to re-derive from), and a finalized row is never revisited. +// +// 1. "Re-analyze" over a day older than raw retention. `LocalDb.dataHistoryDays` +// lists derived days with `raw_count == 0`; Advanced data → Select all → +// Re-analyze runs `runDays(force: true)` over ALL of them. Such a day +// prepares an EMPTY substrate, derives an all-absent bundle, and — because +// an empty bundle's `endSec` was 0, making `endSec + 48 h < dataNowSec` +// unconditionally true — wrote that blank FINALIZED over the good row. +// Only `run()` had a pruned-raw guard, and only for user-override days. +// +// 2. A skip marker. One `_perDayTimeout` (90 s) overrun on a loaded phone +// during a backlog sweep replaced the day's whole result with +// `{'skipped': true}` and, once >48 h behind the data edge, finalized it — +// including TODAY (a good 08:00 result blanked by a transient 09:00 +// timeout). `rescanRecent` explicitly refuses to do this for exactly this +// reason; `run()` did it anyway. + +import 'dart:convert'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; +import 'package:openstrap_edge/compute/derivation_engine.dart'; +import 'package:openstrap_edge/compute/derive_prepare.dart'; +import 'package:openstrap_edge/compute/profile.dart'; +import 'package:openstrap_edge/compute/substrate.dart'; +import 'package:openstrap_edge/data/db.dart'; +import 'package:openstrap_edge/data/models.dart'; + +void main() { + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + LocalDb.dbName = 'openstrap_result_protection_test.db'; + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + tearDownAll(() async { + await LocalDb.close(); + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + /// A GOOD, finished day: real headline scalars in `day_result` + the full + /// baseline `metric_series` fan-out. Exactly what months-old history looks + /// like after its raw has been pruned. + Future seedGoodDay(String dayId, {bool finalized = true}) async { + await LocalDb.putDayResult( + dayId: dayId, + algoVersion: kAlgoVersion, + payloadJson: jsonEncode({ + 'scalars': { + 'rhr': 52.0, + 'rmssd': 61.0, + 'readiness': 74.0, + 'ln_rmssd': 4.11, + 'resp_rate': 14.2, + 'skin_temp_adc': 3011.0, + 'strain': 9.4, + 'tst_min': 431.0, + }, + 'sleep': {'accounting': {'value': {'tst_sec': 25860}}}, + }), + windowJson: '{"onset_ms": 1, "offset_ms": 2}', + finalized: finalized, + rhr: 52, + rmssd: 61, + readiness: 74, + series: { + 'rhr': 52.0, + 'rmssd': 61.0, + 'readiness': 74.0, + 'ln_rmssd': 4.11, + 'resp_rate': 14.2, + 'skin_temp_adc': 3011.0, + 'strain': 9.4, + 'tst_min': 431.0, + }, + ); + } + + Future> readScalars(String dayId) async { + final out = {}; + for (final key in const [ + 'rhr', + 'rmssd', + 'readiness', + 'strain', + 'tst_min', + ]) { + out[key] = await LocalDb.metricValueOn(dayId, key); + } + return out; + } + + // ── 1. an empty (raw-pruned) re-derive must not blank the day ───────────── + + test('re-analyzing a day whose raw is long gone keeps the existing result', + () async { + const oldDay = '2026-01-05'; + await seedGoodDay(oldDay); + final before = await readScalars(oldDay); + expect(before['readiness'], 74.0, reason: 'precondition'); + + // A data edge exists (decoded rows for a MUCH later day), but the target + // day has no decoded rows at all — the post-retention state. + final edgeSec = DateTime(2026, 3, 20, 9, 0).millisecondsSinceEpoch ~/ 1000; + await LocalDb.insertRecord( + RawRecord( + counter: 900001, + packetType: 47, + hex: 'edge', + capturedAt: edgeSec * 1000, + recTs: edgeSec, + ), + Sample( + tsEpoch: edgeSec, + counter: 900001, + hr: 58, + rrIntervalsMs: const [1000], + ax: 0, + ay: 0, + az: 1, + spo2RedRaw: 1, + spo2IrRaw: 1, + skinTempRaw: 3000, + ), + ); + expect(await LocalDb.lastDecodedRecTs(), edgeSec); + + // THE reachable path: Advanced data → Select all → Re-analyze. + await DerivationEngine().runDays(const Profile(), {oldDay}, force: true); + + final after = await readScalars(oldDay); + expect(after, equals(before), + reason: 'an empty derive must never REPLACE the persisted scalars — ' + 'putDayResult nulls every metric_series row for the date'); + final row = await LocalDb.dayResult(oldDay); + expect(row, isNotNull); + expect(row!['skipped'], 0); + expect((row['readiness'] as num?)?.toDouble(), 74.0); + final payload = + (jsonDecode(row['payload_json'] as String) as Map)['scalars'] as Map; + expect(payload['strain'], 9.4, + reason: 'the full bundle survives, not just the indexed columns'); + }); + + test('an empty result for a day with NO prior result is written unfinalized', + () async { + // Nothing to protect here, so the row IS written — but it must stay + // recomputable. Locking an all-absent row is what made the damage permanent. + const freshDay = '2026-01-06'; + expect(await LocalDb.dayResult(freshDay), isNull, reason: 'precondition'); + + await DerivationEngine().runDays(const Profile(), {freshDay}, force: true); + + final row = await LocalDb.dayResult(freshDay); + if (row != null) { + expect(row['finalized'], 0, + reason: 'a result with nothing in it must never lock — a later pass ' + '(or restored substrate) has to be able to fill the day in'); + } + }); + + // ── the endSec that made the blank FINALIZE ─────────────────────────────── + + test('an empty substrate yields the day\'s real calendar end, not 0', () { + const dayId = '2026-01-05'; + final prepared = SleepSessionCandidate.absent(dayId).toPreparedDay( + daySub: Substrate.empty, + sleepSub: Substrate.empty, + ); + // endSec == 0 makes the finalization test `endSec + 48 h < dataNowSec` + // unconditionally true, so the blank locked immediately. + expect(prepared.endSec, isNot(0)); + expect(prepared.endSec, localNextMidnightSecForDayLabel(dayId)); + expect(prepared.endSec, + DateTime(2026, 1, 6).millisecondsSinceEpoch ~/ 1000); + }); + + test('a non-empty substrate still ends at its last record + 1', () { + final ts = DateTime(2026, 1, 5, 22, 0).millisecondsSinceEpoch ~/ 1000; + final sub = Substrate( + tsSec: [ts - 1, ts], + hr: const [60, 61], + rrTsMs: const [], + rrMs: const [], + ax: const [0, 0], + ay: const [0, 0], + az: const [1, 1], + spo2Red: const [0, 0], + spo2Ir: const [0, 0], + skinTemp: const [0, 0], + skinContact: const [0, 0], + ); + final prepared = SleepSessionCandidate.absent('2026-01-05') + .toPreparedDay(daySub: sub, sleepSub: Substrate.empty); + expect(prepared.endSec, ts + 1); + }); + + // ── 2. a skip marker must never overwrite a real result ─────────────────── + + test('a transient timeout never blanks a good day', () async { + const day = '2026-02-10'; + await seedGoodDay(day, finalized: false); + final before = await readScalars(day); + + // The day sits far behind the data edge — the exact condition under which + // the old code wrote the marker FINALIZED. + final dayEndSec = DateTime(2026, 2, 11).millisecondsSinceEpoch ~/ 1000; + final dataNowSec = dayEndSec + 10 * 86400; + await DerivationEngine().debugMarkDaySkipped( + day, + dayEndSec, + dataNowSec, + reason: 'timeout', + ); + + final row = await LocalDb.dayResult(day); + expect(row!['skipped'], 0, reason: 'the good row is untouched'); + expect((row['readiness'] as num?)?.toDouble(), 74.0); + expect(await readScalars(day), equals(before), + reason: 'metric_series survives — putDayResult would have nulled it'); + }); + + test('a good TODAY is not blanked by one transient failure', () async { + // The daily-life case: a good 08:00 result, then a 09:00 pass times out. + const day = '2026-02-11'; + await seedGoodDay(day, finalized: false); + final dayEndSec = DateTime(2026, 2, 12).millisecondsSinceEpoch ~/ 1000; + await DerivationEngine().debugMarkDaySkipped( + day, + dayEndSec, + dayEndSec - 3600, // data edge still inside the day + reason: 'error', + ); + final row = await LocalDb.dayResult(day); + expect(row!['skipped'], 0); + expect((row['rhr'] as num?)?.toDouble(), 52.0); + }); + + test('a skip marker IS written when there is no good row to lose', () async { + const day = '2026-02-12'; + expect(await LocalDb.dayResult(day), isNull, reason: 'precondition'); + final dayEndSec = DateTime(2026, 2, 13).millisecondsSinceEpoch ~/ 1000; + await DerivationEngine().debugMarkDaySkipped( + day, + dayEndSec, + dayEndSec + 10 * 86400, + reason: 'day_prepare_budget_exceeded', + ); + final row = await LocalDb.dayResult(day); + expect(row, isNotNull); + expect(row!['skipped'], 1); + // A STRUCTURAL failure still finalizes once aged out, so a pathological day + // isn't retried forever. + expect(row['finalized'], 1); + }); + + test('a TRANSIENT skip is never finalized, so the day gets another chance', + () async { + const day = '2026-02-13'; + final dayEndSec = DateTime(2026, 2, 14).millisecondsSinceEpoch ~/ 1000; + await DerivationEngine().debugMarkDaySkipped( + day, + dayEndSec, + dayEndSec + 10 * 86400, // aged well past finalization + reason: 'timeout', + ); + final row = await LocalDb.dayResult(day); + expect(row!['skipped'], 1); + expect(row['finalized'], 0, + reason: 'finalizing a 90 s timeout locks the day out of every future ' + 'pass at this algo version — permanently blank'); + expect( + (await LocalDb.finalizedDayIds(kAlgoVersion)).contains(day), + isFalse, + ); + }); + + test('a skip marker does not overwrite an existing skip marker\'s reason ' + 'with a worse one — but is allowed to replace it', () async { + const day = '2026-02-14'; + final dayEndSec = DateTime(2026, 2, 15).millisecondsSinceEpoch ~/ 1000; + await DerivationEngine() + .debugMarkDaySkipped(day, dayEndSec, dayEndSec, reason: 'timeout'); + await DerivationEngine().debugMarkDaySkipped( + day, + dayEndSec, + dayEndSec + 10 * 86400, + reason: 'day_prepare_budget_exceeded', + ); + final row = await LocalDb.dayResult(day); + final payload = jsonDecode(row!['payload_json'] as String) as Map; + expect(payload['reason'], 'day_prepare_budget_exceeded', + reason: 'a skip marker carries no user data, so replacing one with ' + 'another is fine — only REAL results are protected'); + }); +} diff --git a/test/headless_gate_test.dart b/test/headless_gate_test.dart index 2265659e..012e664c 100644 --- a/test/headless_gate_test.dart +++ b/test/headless_gate_test.dart @@ -1,15 +1,21 @@ -// HeadlessSyncGate: mutual exclusion across the three iOS headless wake -// sources (BLE-restore, BGProcessingTask, BGAppRefreshTask) + the skip-streak -// telemetry that makes repeated wake-source collisions observable instead of -// a single easy-to-miss debugPrint line. +// HeadlessSyncGate: mutual exclusion across EVERY headless wake source — the +// three iOS ones (BLE-restore, BGProcessingTask, BGAppRefreshTask) and the +// Android post-boot wake — plus the skip-streak telemetry that makes repeated +// wake-source collisions observable instead of a single easy-to-miss +// debugPrint line. import 'dart:async'; import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_edge/sync/band_ownership.dart'; +import 'package:openstrap_edge/sync/headless_boot.dart'; import 'package:openstrap_edge/sync/headless_gate.dart'; void main() { - setUp(() => HeadlessSyncGate.resetForTest()); + setUp(() { + HeadlessSyncGate.resetForTest(); + BandOwnership.resetForTest(); + }); test('a solo run is never a skip and leaves no streak', () async { final result = await HeadlessSyncGate.tryRun('owner_a', () async => 1); @@ -94,4 +100,66 @@ void main() { await run; expect(HeadlessSyncGate.busy, isFalse); }); + + group('P2 — the Android boot wake goes through the gate too', () { + test('the gate is BUSY for the whole duration of a boot drain', () async { + final lease = BandOwnership.tryAcquireHeadless()!; + final started = Completer(); + final finish = Completer(); + + final run = runBootSyncThroughGate( + lease, + runner: (l) async { + started.complete(); + await finish.future; + return true; + }, + ); + + await started.future; + // OLD BEHAVIOUR: the boot path called runHeadlessSync(lease: lease) + // directly and fire-and-forget, so `busy` read false for the entire + // boot drain and any other wake source would have run concurrently. + expect(HeadlessSyncGate.busy, isTrue); + expect( + await HeadlessSyncGate.tryRun('ble_restore_wake', () async => 7), + isNull, + reason: 'another wake source must SKIP while the boot drain holds it', + ); + + finish.complete(); + expect(await run, isTrue); + expect(HeadlessSyncGate.busy, isFalse); + }); + + test('a boot wake that loses the race skips AND releases its band lease', + () async { + final lease = BandOwnership.tryAcquireHeadless()!; + expect(BandOwnership.owner, BandOwnerKind.headless); + + final finish = Completer(); + final holder = HeadlessSyncGate.tryRun('ios_bg_task', () async { + await finish.future; + }); + + var ran = false; + final result = await runBootSyncThroughGate( + lease, + runner: (l) async { + ran = true; + return true; + }, + ); + + expect(result, isNull); + expect(ran, isFalse); + // The lease was acquired before the gate was consulted; a skipped cycle + // must hand it back or the band stays owned by a run that never happened. + expect(BandOwnership.owner, isNull); + expect(HeadlessSyncGate.consecutiveSkipsFor(kBootWakeGateOwner), 1); + + finish.complete(); + await holder; + }); + }); } diff --git a/test/import_data_safety_test.dart b/test/import_data_safety_test.dart new file mode 100644 index 00000000..c87a64af --- /dev/null +++ b/test/import_data_safety_test.dart @@ -0,0 +1,320 @@ +// Data-integrity regressions for the three import paths. +// +// • WHOOP CSV must NEVER overwrite a day the device derived from real 1 Hz +// (putDayResult is INSERT-OR-REPLACE on day_result AND metric_series, and +// the importer used to pass finalized:true, which additionally locked the +// day out of DerivationEngine forever — months of band data, gone, from a +// button reachable in onboarding AND in Profile). +// • Energy is converted from the COLUMN'S declared unit, never guessed from +// the value's magnitude (a real 4,500 kcal ultra day was being rewritten as +// 1,076 kcal and then exported to Apple Health / Health Connect). +// • A cloud session with no start_ts is skipped, not filed on 1970-01-01. +// • The raw-CSV importer's high-water date only moves forward, so an +// out-of-order row can't discard a whole buffered day. +import 'dart:convert'; +import 'dart:io'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; +import 'package:openstrap_edge/cloud/cloud_import.dart'; +import 'package:openstrap_edge/compute/derivation_engine.dart' show kAlgoVersion; +import 'package:openstrap_edge/compute/substrate.dart' show localDateLabel; +import 'package:openstrap_edge/data/db.dart'; +import 'package:openstrap_edge/import/noop_import.dart'; +import 'package:openstrap_edge/import/whoop_import.dart'; + +/// Local-time wall clock → the day label the importer will file it under. +const _wake = '2026-03-05 08:30:00'; +final _wakeSec = DateTime.parse(_wake).millisecondsSinceEpoch ~/ 1000; +final _day = localDateLabel(_wakeSec); + +String _csv(Directory dir, String name, String energyHeader, String energy) { + final f = File(p.join(dir.path, name)); + f.writeAsStringSync( + 'Cycle start time,Wake onset,Recovery score %,Resting heart rate (bpm),' + 'Heart rate variability (ms),Day Strain,$energyHeader,' + 'Asleep duration (min)\n' + '$_wake,$_wake,42,70,19,7.5,$energy,300\n', + ); + return f.path; +} + +Future?> _row(String day) => LocalDb.dayResult(day); + +Future _metric(String day, String key) async { + final db = await LocalDb.instance; + final rows = await db.query('metric_series', + where: 'date = ? AND key = ?', whereArgs: [day, key]); + if (rows.isEmpty) return null; + return (rows.first['value'] as num?)?.toDouble(); +} + +void main() { + late Directory tmp; + + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + LocalDb.dbName = 'openstrap_import_safety_test.db'; + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + tmp = Directory.systemTemp.createTempSync('openstrap_import_test'); + }); + + tearDownAll(() async { + await LocalDb.close(); + try { + tmp.deleteSync(recursive: true); + } catch (_) {} + }); + + group('WhoopImporter never destroys a real derived day', () { + test('skips a date that already holds a 1 Hz-derived day', () async { + // A REAL derived day: no `imported` marker, real scalars. + await LocalDb.putDayResult( + dayId: _day, + algoVersion: kAlgoVersion, + payloadJson: jsonEncode({'date': _day, 'source': 'onehz', 'real': true}), + windowJson: '{}', + rhr: 48.0, + rmssd: 88.0, + readiness: 91.0, + series: {'rhr': 48.0, 'rmssd': 88.0, 'readiness': 91.0}, + ); + + final res = await WhoopImporter.importFiles( + [_csv(tmp, 'day_real.csv', 'Energy burned (cal)', '2400')]); + + expect(res.days, 0, reason: 'a real derived day must not be replaced'); + expect(res.skippedExistingDays, 1); + + final row = await _row(_day); + final payload = jsonDecode(row!['payload_json'] as String) as Map; + expect(payload['real'], isTrue, reason: 'payload was overwritten'); + expect(payload['source'], 'onehz'); + // The scalars the user cares about survive untouched. + expect(await _metric(_day, 'rhr'), 48.0); + expect(await _metric(_day, 'rmssd'), 88.0); + expect(await _metric(_day, 'readiness'), 91.0); + // …and the day was NOT force-finalized out of the derivation engine. + expect((row['finalized'] as num).toInt(), 0); + }); + + test('writes into a genuinely empty day', () async { + const otherWake = '2026-03-09 07:15:00'; + final otherDay = localDateLabel( + DateTime.parse(otherWake).millisecondsSinceEpoch ~/ 1000); + final f = File(p.join(tmp.path, 'day_empty.csv')); + f.writeAsStringSync( + 'Cycle start time,Wake onset,Recovery score %,Resting heart rate (bpm),' + 'Heart rate variability (ms),Day Strain,Energy burned (cal),' + 'Asleep duration (min)\n' + '$otherWake,$otherWake,55,60,70,9.1,2200,420\n', + ); + final res = await WhoopImporter.importFiles([f.path]); + expect(res.days, 1); + expect(res.skippedExistingDays, 0); + final payload = + jsonDecode((await _row(otherDay))!['payload_json'] as String) as Map; + expect(payload['source'], 'whoop_export'); + expect(await _metric(otherDay, 'rhr'), 60.0); + // No raw for this date, so nothing could ever re-derive it — finalizing + // is correct here. + expect(((await _row(otherDay))!['finalized'] as num).toInt(), 1); + }); + + test('does not finalize a day that still has raw to re-derive from', + () async { + const wake = '2026-05-04 07:00:00'; + final wakeSec = DateTime.parse(wake).millisecondsSinceEpoch ~/ 1000; + final day = localDateLabel(wakeSec); + final db = await LocalDb.instance; + await db.insert('decoded_onehz', { + 'counter': 900001, + 'rec_ts': wakeSec, + 'hr': 62, + 'ax': 0.0, + 'ay': 0.0, + 'az': 1.0, + 'spo2_red_raw': 0, + 'spo2_ir_raw': 0, + 'skin_temp_raw': 0, + }); + final f = File(p.join(tmp.path, 'day_with_raw.csv')); + f.writeAsStringSync( + 'Cycle start time,Wake onset,Recovery score %,Resting heart rate (bpm),' + 'Heart rate variability (ms),Day Strain,Energy burned (cal),' + 'Asleep duration (min)\n' + '$wake,$wake,50,60,60,5,2000,400\n', + ); + expect((await WhoopImporter.importFiles([f.path])).days, 1); + // finalized:true would lock DerivationEngine out of this day FOREVER — + // the vendor snapshot would permanently outrank the real 1 Hz signal. + expect(((await _row(day))!['finalized'] as num).toInt(), 0); + }); + + test('re-importing over a PREVIOUS import is allowed', () async { + const wake = '2026-03-11 06:00:00'; + final day = + localDateLabel(DateTime.parse(wake).millisecondsSinceEpoch ~/ 1000); + String write(String rhr) { + final f = File(p.join(tmp.path, 'day_reimport.csv')); + f.writeAsStringSync( + 'Cycle start time,Wake onset,Recovery score %,' + 'Resting heart rate (bpm),Heart rate variability (ms),Day Strain,' + 'Energy burned (cal),Asleep duration (min)\n' + '$wake,$wake,50,$rhr,65,6.0,2000,400\n', + ); + return f.path; + } + + expect((await WhoopImporter.importFiles([write('58')])).days, 1); + expect(await _metric(day, 'rhr'), 58.0); + final again = await WhoopImporter.importFiles([write('61')]); + expect(again.days, 1, reason: 'vendor snapshots may replace each other'); + expect(await _metric(day, 'rhr'), 61.0); + }); + }); + + group('WhoopImporter energy units come from the header, not the value', () { + Future importEnergy( + String wake, String header, String value) async { + final day = + localDateLabel(DateTime.parse(wake).millisecondsSinceEpoch ~/ 1000); + final f = File(p.join(tmp.path, 'energy_${header.hashCode}.csv')); + f.writeAsStringSync( + 'Cycle start time,Wake onset,Recovery score %,Resting heart rate (bpm),' + 'Heart rate variability (ms),Day Strain,$header,Asleep duration (min)\n' + '$wake,$wake,50,60,60,5,$value,400\n', + ); + await WhoopImporter.importFiles([f.path]); + return _metric(day, 'calories'); + } + + test('a real 4,500 kcal ultra day is NOT rewritten as kJ', () async { + // The old heuristic (`v > 4000 ? v / 4.184 : v`) turned this into 1,076. + expect(await importEnergy('2026-04-01 07:00:00', 'Energy burned (cal)', + '4500'), + 4500.0); + }); + + test('a kJ column IS converted', () async { + final v = await importEnergy( + '2026-04-02 07:00:00', 'Energy burned (kJ)', '8000'); + expect(v, closeTo(8000 / 4.184, 0.01)); + }); + + test('an ambiguous unit-less column is dropped, not guessed', () async { + expect( + await importEnergy('2026-04-03 07:00:00', 'Energy burned', '5000'), + isNull); + }); + }); + + group('CloudImporter session rows', () { + test('skips a session with no start_ts instead of filing it at epoch 0', + () async { + final wrote = await CloudImporter.debugWriteSession({ + 'id': 'malformed-1', + 'end_ts': 1780000000, + 'type': 'run', + }); + expect(wrote, isFalse); + final db = await LocalDb.instance; + final rows = await db + .query('sessions', where: 'start_ts <= ?', whereArgs: [0]); + expect(rows, isEmpty, reason: 'a 1970-01-01 phantom workout was written'); + final byId = + await db.query('sessions', where: 'id = ?', whereArgs: ['malformed-1']); + expect(byId, isEmpty); + }); + + test('writes a well-formed session', () async { + final wrote = await CloudImporter.debugWriteSession({ + 'id': 'good-1', + 'start_ts': 1780000000, + 'end_ts': 1780003600, + 'type': 'run', + }); + expect(wrote, isTrue); + final db = await LocalDb.instance; + final rows = + await db.query('sessions', where: 'id = ?', whereArgs: ['good-1']); + expect(rows.length, 1); + expect((rows.first['start_ts'] as num).toInt(), 1780000000); + expect((rows.first['duration_min'] as num).toInt(), 60); + }); + }); + + group('raw-CSV importer row ordering (high-water date)', () { + test('the first row starts the window', () { + expect(NoopImporter.decideRow('2026-01-02', null, {}), RowOrder.advance); + }); + + test('a later date closes out the previous one', () { + expect(NoopImporter.decideRow('2026-01-03', '2026-01-02', {}), + RowOrder.advance); + }); + + test('same-date rows just buffer', () { + expect(NoopImporter.decideRow('2026-01-02', '2026-01-02', {}), + RowOrder.buffer); + }); + + test('an out-of-order row NEVER rewinds the high-water date', () { + // THE bug: this used to set curDate back to 2026-01-01, so the next + // 2026-01-02 row called deriveAndPrune('2026-01-01') and dropped every + // buffered sample of 2026-01-02. + expect(NoopImporter.decideRow('2026-01-01', '2026-01-02', {}), + RowOrder.buffer); + expect(NoopImporter.decideRow('2026-01-01', '2026-01-02', {}), + isNot(RowOrder.advance)); + }); + + test('a row for an already-derived day is reported late, not silently lost', + () { + expect( + NoopImporter.decideRow('2026-01-01', '2026-01-02', {'2026-01-01'}), + RowOrder.late, + ); + }); + + test('a whole out-of-order sequence keeps every day exactly once', () { + // Interleaved input: D1, D2, D1(late-ish), D2, D3, D2(late), D3 + const rows = [ + '2026-01-01', + '2026-01-02', + '2026-01-01', + '2026-01-02', + '2026-01-03', + '2026-01-02', + '2026-01-03', + ]; + String? cur; + final derived = {}; + final advanced = []; + var late = 0; + for (final d in rows) { + switch (NoopImporter.decideRow(d, cur, derived)) { + case RowOrder.advance: + if (cur != null) { + advanced.add(cur); + derived.add(cur); + } + cur = d; + case RowOrder.buffer: + break; + case RowOrder.late: + late++; + } + } + // Each day is derived exactly once and always in order. + expect(advanced, ['2026-01-01', '2026-01-02']); + expect(cur, '2026-01-03'); + // Both genuinely unusable rows (a D1 after D1 was derived, and a D2 + // after D2 was derived) are counted rather than silently dropped. + expect(late, 2); + }); + }); +} diff --git a/test/local_repository_p0_test.dart b/test/local_repository_p0_test.dart new file mode 100644 index 00000000..745016bc --- /dev/null +++ b/test/local_repository_p0_test.dart @@ -0,0 +1,135 @@ +// Repository-layer P0 regressions, against the REAL LocalRepositoryImpl + +// LocalDb over sqflite_ffi. +// +// 7. getCycle() crashed the whole cycle screen whenever the mean cycle length +// came out below the 10-day ovulation floor: `(mean - 14).round().clamp(10, +// mean.round())` THROWS ArgumentError when lowerLimit > upperLimit. Two +// logged `start` markers 8 days apart is enough (a correction the user +// made, or a genuinely short cycle). +// 10. getRecords() no longer reads day_result payloads at all — its day/night +// counts come from SQL. It used to `recentDayResults(3650)` (SELECT r.*, +// hr_curve + hypnogram + HRV series for TEN YEARS) and jsonDecode every one +// on the main isolate for what is only a scalar-extremes screen. + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; +import 'package:openstrap_edge/data/db.dart'; +import 'package:openstrap_edge/data/local_repository_impl.dart'; + +void main() { + late LocalRepositoryImpl repo; + + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + LocalDb.dbName = 'openstrap_repo_p0_test.db'; + await databaseFactory.deleteDatabase( + p.join(await databaseFactory.getDatabasesPath(), LocalDb.dbName), + ); + repo = LocalRepositoryImpl(getProfileMap: () => {'track_cycle': true}); + }); + + tearDownAll(() async { + await LocalDb.close(); + await databaseFactory.deleteDatabase( + p.join(await databaseFactory.getDatabasesPath(), LocalDb.dbName), + ); + }); + + setUp(() async { + final db = await LocalDb.instance; + await db.delete('cycle_log'); + await db.delete('day_result'); + await db.delete('metric_series'); + await db.delete('sessions'); + }); + + // ── fix 7 ──────────────────────────────────────────────────────────────── + test( + 'getCycle degrades to phase "unknown" on a sub-10-day mean cycle instead ' + 'of throwing ArgumentError out of clamp()', + () async { + // Two starts 8 days apart → meanLength 8 → clamp(10, 8) used to throw. + await LocalDb.putCycleLog('2026-06-01', 'start'); + await LocalDb.putCycleLog('2026-06-09', 'start'); + + final cycle = await repo.getCycle(); + + expect(cycle['enabled'], isTrue); + expect(cycle['phase'], 'unknown', reason: 'honest, not invented'); + expect(cycle['fertile_start'], isNull); + expect(cycle['fertile_end'], isNull); + // Everything that IS knowable still comes back. + expect(cycle['mean_length'], 8); + expect(cycle['predicted_next'], isNotNull); + expect(cycle['cycle_day'], isNotNull); + }, + ); + + test('a normal-length cycle still gets a real phase + fertile window', + () async { + await LocalDb.putCycleLog('2026-06-01', 'start'); + await LocalDb.putCycleLog('2026-06-29', 'start'); // 28 days + + final cycle = await repo.getCycle(); + expect(cycle['mean_length'], 28); + expect(cycle['phase'], isNot('unknown')); + expect(cycle['fertile_start'], isNotNull); + expect(cycle['fertile_end'], isNotNull); + }); + + test('exactly 10 days — the clamp boundary — does not throw', () async { + await LocalDb.putCycleLog('2026-06-01', 'start'); + await LocalDb.putCycleLog('2026-06-11', 'start'); + final cycle = await repo.getCycle(); + expect(cycle['mean_length'], 10); + expect(cycle['phase'], isNotNull); + }); + + // ── fix 10 ─────────────────────────────────────────────────────────────── + test( + 'getRecords counts days/nights from SQL, with no payload decode — and a ' + 'corrupt bundle degrades instead of breaking the screen', + () async { + String bundle(int? tstSec) => tstSec == null + ? '{"scalars":{}}' + : '{"sleep":{"accounting":{"value":{"tst_sec":$tstSec}}}}'; + + await LocalDb.putDayResult( + dayId: '2026-01-01', + algoVersion: 41, + payloadJson: bundle(21600), + windowJson: '{}', + series: const {'rhr': 52.0}, + ); + await LocalDb.putDayResult( + dayId: '2026-01-02', + algoVersion: 41, + payloadJson: bundle(25200), + windowJson: '{}', + ); + await LocalDb.putDayResult( + dayId: '2026-01-03', + algoVersion: 41, + payloadJson: bundle(null), // wore it, never slept in it + windowJson: '{}', + ); + await LocalDb.putDayResult( + dayId: '2026-01-04', + algoVersion: 41, + payloadJson: '<>', // not JSON at all + windowJson: '{}', + ); + + final records = await repo.getRecords(); + expect(records['days_tracked'], 4); + expect( + records['nights_tracked'], + 2, + reason: 'only days whose bundle records a real tst_sec', + ); + expect(records['workouts_tracked'], 0); + }, + ); +} diff --git a/test/metric_trend_redesign_test.dart b/test/metric_trend_redesign_test.dart index b62686b6..355bb277 100644 --- a/test/metric_trend_redesign_test.dart +++ b/test/metric_trend_redesign_test.dart @@ -235,10 +235,13 @@ void main() { expect(t.takeException(), isNull); }); - test('trendBarLabel maps scales to weekday / week-index / month', () { + test('trendBarLabel maps scales to weekday / window-end / month', () { final ts = DateTime.utc(2026, 7, 6).millisecondsSinceEpoch ~/ 1000; // Mon expect(trendBarLabel('week', 0, {'t_start': ts}), 'Mon'); - expect(trendBarLabel('month', 2, {'t_start': ts}), 'W3'); + // 'month' buckets are ROLLING 7-day windows ending at the anchor, not + // calendar weeks — 'W1…W4' claimed a calendar structure they don't have. + // See absent_not_zero_test.dart. + expect(trendBarLabel('month', 2, {'t_start': ts}), 'Jul 12'); expect(trendBarLabel('quarter', 0, {'t_start': ts}), 'Jul'); }); }); diff --git a/test/notification_center_test.dart b/test/notification_center_test.dart index 0fb9a8a3..4b511197 100644 --- a/test/notification_center_test.dart +++ b/test/notification_center_test.dart @@ -3,7 +3,10 @@ // we construct NotificationPrefs/NotificationEvent directly. import 'package:flutter_test/flutter_test.dart'; +import 'package:shared_preferences/shared_preferences.dart'; + import 'package:openstrap_edge/notify/notification_event.dart'; +import 'package:openstrap_edge/notify/notification_ids.dart'; import 'package:openstrap_edge/notify/notification_prefs.dart'; NotificationEvent _ev(NotifCategory c, NotifPriority p) => NotificationEvent( @@ -67,15 +70,24 @@ void main() { }); group('osId partitioning', () { - test('categories land in disjoint bands', () { - final health = _ev(NotifCategory.health, NotifPriority.critical).osId; - final recovery = _ev(NotifCategory.recovery, NotifPriority.normal).osId; - final reminders = _ev(NotifCategory.reminders, NotifPriority.low).osId; + setUp(() { + SharedPreferences.setMockInitialValues({}); + NotificationIds.instance.resetForTest(); + }); + + test('categories land in disjoint bands', () async { + final ids = NotificationIds.instance; + final health = + await ids.idFor(_ev(NotifCategory.health, NotifPriority.critical)); + final recovery = + await ids.idFor(_ev(NotifCategory.recovery, NotifPriority.normal)); + final reminders = + await ids.idFor(_ev(NotifCategory.reminders, NotifPriority.low)); expect(health ~/ 100000, equals(3)); expect(recovery ~/ 100000, equals(2)); expect(reminders ~/ 100000, equals(4)); }); - test('same logical event yields a stable id (replace, not stack)', () { + test('same logical event yields a stable id (replace, not stack)', () async { final a = NotificationEvent( dedupeKey: '2026-06-27:illness', category: NotifCategory.health, @@ -88,7 +100,8 @@ void main() { title: 'different title', body: 'different body', date: '2026-06-27'); - expect(a.osId, equals(b.osId)); + expect(await NotificationIds.instance.idFor(a), + equals(await NotificationIds.instance.idFor(b))); }); }); } diff --git a/test/notification_day_guard_test.dart b/test/notification_day_guard_test.dart new file mode 100644 index 00000000..4babbf43 --- /dev/null +++ b/test/notification_day_guard_test.dart @@ -0,0 +1,201 @@ +// Regression tests for the once-per-day notification guard. +// +// AppState's "your recovery is ready" (_kLastRecoveryNotifDay) and "step goal +// reached" (_kLastStepGoalDay) used to write their persisted day-guard BEFORE +// calling NotificationCenter.emit. emit DROPS the event outright when +// NotificationPrefs.shouldFireOs says no — and the DEFAULT quiet window is +// 22:00–07:00, which a band syncing at 06:40 (the heavy finalize that computes +// the new day's recovery) sits squarely inside. So the guard was burned on a +// notification that never reached the user, and it then blocked every retry for +// the rest of the day: "Your recovery is ready" simply never fired. +// +// NotificationCenter.emitOncePerDay is the fixed sequencing (claim the guard +// only on a real present) and emit now REPORTS whether it presented. +// +// NOTE — no sqlite factory is registered here, so FiredKeyStore runs in its +// degraded shared_preferences mode. That's fine: this suite is about the DAY +// guard, not the cross-isolate claim (see notification_claim_atomic_test.dart). + +import 'package:flutter_test/flutter_test.dart'; +import 'package:shared_preferences/shared_preferences.dart'; + +import 'package:openstrap_edge/data/day_label.dart'; +import 'package:openstrap_edge/notify/notification_center.dart'; +import 'package:openstrap_edge/notify/notification_event.dart'; +import 'package:openstrap_edge/notify/notification_ids.dart'; +import 'package:openstrap_edge/notify/notification_service.dart'; + +const String kGuardKey = 'last_recovery_notif_day'; +final String kDay = todayLabel(); +final String kTomorrow = dayLabelOf(DateTime.now().add(const Duration(days: 1))); + +NotificationEvent _recoveryReady({String? day}) { + final d = day ?? kDay; + return NotificationEvent( + dedupeKey: '$d:recovery_ready', + category: NotifCategory.recovery, + priority: NotifPriority.normal, + title: 'Your recovery is ready', + body: 'Recovery 71. Tap to see today.', + date: d, + route: '/today', + ); +} + +class _Sink { + final List shown = []; + bool grant; + _Sink({this.grant = true}); + Future call(NotificationEvent e, + {bool allowPermissionPrompt = true}) async { + if (!grant) return false; + shown.add(e.dedupeKey); + return true; + } +} + +/// Prefs values that make the quiet window cover the entire 24 h, so a +/// non-critical event is deterministically suppressed regardless of the +/// wall-clock the test happens to run at. +Map _quietAllDay() => { + 'notif_quiet_enabled': true, + 'notif_quiet_start': 0, + 'notif_quiet_end': 1440, + }; + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + + final original = NotificationCenter.instance.presentSink; + tearDown(() => NotificationCenter.instance.presentSink = original); + + setUp(() { + NotificationIds.instance.resetForTest(); + }); + + group('emit reports whether the event actually reached the OS', () { + test('returns false when quiet hours suppress it', () async { + SharedPreferences.setMockInitialValues(_quietAllDay()); + final sink = _Sink(); + NotificationCenter.instance.presentSink = sink.call; + expect(await NotificationCenter.instance.emit(_recoveryReady()), isFalse); + expect(sink.shown, isEmpty); + }); + + test('returns false when the OS present is refused (permission denied)', + () async { + SharedPreferences.setMockInitialValues({}); + final sink = _Sink(grant: false); + NotificationCenter.instance.presentSink = sink.call; + expect(await NotificationCenter.instance.emit(_recoveryReady()), isFalse); + }); + + test('returns true on a real present', () async { + SharedPreferences.setMockInitialValues({}); + final sink = _Sink(); + NotificationCenter.instance.presentSink = sink.call; + expect(await NotificationCenter.instance.emit(_recoveryReady()), isTrue); + expect(sink.shown, ['$kDay:recovery_ready']); + }); + }); + + group('emitOncePerDay consumes the guard only on a real present', () { + test('a quiet-hours suppression does NOT burn the day guard, and the ' + 'retry once quiet hours end still fires', () async { + SharedPreferences.setMockInitialValues(_quietAllDay()); + final sink = _Sink(); + NotificationCenter.instance.presentSink = sink.call; + + // 06:40 sync inside the quiet window → suppressed. + final firstTry = await NotificationCenter.instance.emitOncePerDay( + prefsKey: kGuardKey, + dayId: kDay, + e: _recoveryReady(), + ); + expect(firstTry, isFalse); + final prefs = await SharedPreferences.getInstance(); + expect(prefs.getString(kGuardKey), isNull, + reason: 'the guard must not be consumed by an event that never ' + 'reached the user'); + + // Quiet hours over — the next derive pass must still be able to fire. + await prefs.setBool('notif_quiet_enabled', false); + final retry = await NotificationCenter.instance.emitOncePerDay( + prefsKey: kGuardKey, + dayId: kDay, + e: _recoveryReady(), + ); + expect(retry, isTrue); + expect(sink.shown, ['$kDay:recovery_ready']); + expect(prefs.getString(kGuardKey), kDay); + }); + + test('a permission-denied no-op does NOT burn the day guard either', + () async { + SharedPreferences.setMockInitialValues({}); + final sink = _Sink(grant: false); + NotificationCenter.instance.presentSink = sink.call; + + expect( + await NotificationCenter.instance.emitOncePerDay( + prefsKey: kGuardKey, dayId: kDay, e: _recoveryReady()), + isFalse, + ); + final prefs = await SharedPreferences.getInstance(); + expect(prefs.getString(kGuardKey), isNull); + + sink.grant = true; + expect( + await NotificationCenter.instance.emitOncePerDay( + prefsKey: kGuardKey, dayId: kDay, e: _recoveryReady()), + isTrue, + ); + expect(prefs.getString(kGuardKey), kDay); + }); + + test('a real present consumes the guard, and the same day never re-fires', + () async { + SharedPreferences.setMockInitialValues({}); + final sink = _Sink(); + NotificationCenter.instance.presentSink = sink.call; + + expect( + await NotificationCenter.instance.emitOncePerDay( + prefsKey: kGuardKey, dayId: kDay, e: _recoveryReady()), + isTrue, + ); + expect( + await NotificationCenter.instance.emitOncePerDay( + prefsKey: kGuardKey, dayId: kDay, e: _recoveryReady()), + isFalse, + ); + expect(sink.shown.length, 1); + }); + + test('a NEW day is a fresh guard', () async { + SharedPreferences.setMockInitialValues({kGuardKey: kDay}); + final sink = _Sink(); + NotificationCenter.instance.presentSink = sink.call; + + expect( + await NotificationCenter.instance.emitOncePerDay( + prefsKey: kGuardKey, dayId: kDay, e: _recoveryReady()), + isFalse, + ); + expect( + await NotificationCenter.instance.emitOncePerDay( + prefsKey: kGuardKey, + dayId: kTomorrow, + e: _recoveryReady(day: kTomorrow)), + isTrue, + ); + final prefs = await SharedPreferences.getInstance(); + expect(prefs.getString(kGuardKey), kTomorrow); + }); + }); + + // Sanity: the service singleton's permission cache must not leak between the + // suites in this file (emitOncePerDay never touches it, but presentSink does + // in production). + tearDownAll(() => NotificationService.instance.invalidatePermissionCache()); +} diff --git a/test/notification_ids_test.dart b/test/notification_ids_test.dart new file mode 100644 index 00000000..f3597b88 --- /dev/null +++ b/test/notification_ids_test.dart @@ -0,0 +1,120 @@ +// Regression tests for the OS notification id ALLOCATOR (notification_ids.dart). +// +// Ids used to be DERIVED as `categoryBase + dedupeKey.hashCode.abs() % 100000`. +// That is a hash modulo: two distinct dedupeKeys in the same category whose +// hashes agree mod 100000 produced the SAME id, and `FlutterLocalNotifications +// .show` REPLACES a post with the same id rather than stacking beside it — so +// one of the two notifications vanished with no trace. (The "partitioned so a +// health alert can never overwrite a reminder" comment only ever covered +// CROSS-category collisions.) +// +// The first test below FINDS a real collision under the old formula and proves +// the allocator keeps those two keys apart. + +import 'package:flutter_test/flutter_test.dart'; +import 'package:shared_preferences/shared_preferences.dart'; + +import 'package:openstrap_edge/data/day_label.dart'; +import 'package:openstrap_edge/notify/notification_event.dart'; +import 'package:openstrap_edge/notify/notification_ids.dart'; + +/// Dated keys, anchored to TODAY so the retention prune (14 days) can never +/// make this suite time-dependent. +final String kToday = todayLabel(); + +NotificationEvent _ev(String key, + [NotifCategory c = NotifCategory.reminders]) => + NotificationEvent( + dedupeKey: key, + category: c, + title: 't', + body: 'b', + date: kToday, + ); + +/// The pre-fix id formula, kept here purely as the thing we regress against. +int _legacySlot(String dedupeKey) => dedupeKey.hashCode.abs() % 100000; + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + + setUp(() { + SharedPreferences.setMockInitialValues({}); + NotificationIds.instance.resetForTest(); + }); + + test('two same-category keys that COLLIDE under the old hash get distinct ids', + () async { + // Search for a genuine collision (birthday bound: ~450 keys over 100k slots). + String? a, b; + final bySlot = {}; + for (var i = 0; i < 200000 && b == null; i++) { + final k = '$kToday:probe$i'; + final prior = bySlot[_legacySlot(k)]; + if (prior != null) { + a = prior; + b = k; + } else { + bySlot[_legacySlot(k)] = k; + } + } + expect(b, isNotNull, + reason: 'expected a hashCode%100000 collision within the probe budget'); + + // Precondition: the OLD scheme really did hand these two the same id. + expect(_legacySlot(a!), equals(_legacySlot(b!))); + + final ids = NotificationIds.instance; + final idA = await ids.idFor(_ev(a)); + final idB = await ids.idFor(_ev(b)); + expect(idA, isNot(equals(idB)), + reason: 'colliding keys must not share an OS id — one would ' + 'silently REPLACE the other in the shade'); + // Both still inside the reminders band. + expect(idA ~/ NotificationIds.bandSize, equals(4)); + expect(idB ~/ NotificationIds.bandSize, equals(4)); + }); + + test('a batch of distinct keys gets a fully distinct set of ids', () async { + final ids = NotificationIds.instance; + final out = {}; + for (var i = 0; i < 500; i++) { + out.add(await ids.idFor(_ev('$kToday:k$i'))); + } + expect(out.length, 500); + }); + + test('the same dedupeKey keeps its id — a re-post replaces in place', + () async { + final ids = NotificationIds.instance; + final first = await ids.idFor(_ev('$kToday:recovery_ready')); + final again = await ids.idFor(_ev('$kToday:recovery_ready')); + expect(again, equals(first)); + }); + + test('an allocation survives a process restart (persisted, not memoized)', + () async { + final first = + await NotificationIds.instance.idFor(_ev('$kToday:illness')); + // Simulate a fresh process: in-memory maps gone, shared_preferences intact. + NotificationIds.instance.resetForTest(); + final afterRestart = + await NotificationIds.instance.idFor(_ev('$kToday:illness')); + expect(afterRestart, equals(first)); + }); + + test('categories stay in disjoint bands', () async { + final ids = NotificationIds.instance; + expect(await ids.idFor(_ev('k', NotifCategory.device)) ~/ 100000, 1); + expect(await ids.idFor(_ev('k', NotifCategory.recovery)) ~/ 100000, 2); + expect(await ids.idFor(_ev('k', NotifCategory.health)) ~/ 100000, 3); + expect(await ids.idFor(_ev('k', NotifCategory.reminders)) ~/ 100000, 4); + }); + + test('the SAME key in two categories gets two ids, one per band', () async { + final ids = NotificationIds.instance; + final health = await ids.idFor(_ev('$kToday:x', NotifCategory.health)); + final rem = await ids.idFor(_ev('$kToday:x', NotifCategory.reminders)); + expect(health, isNot(equals(rem))); + }); +} diff --git a/test/notification_permission_test.dart b/test/notification_permission_test.dart new file mode 100644 index 00000000..93b7ae5a --- /dev/null +++ b/test/notification_permission_test.dart @@ -0,0 +1,127 @@ +// Regression tests for NotificationService's permission cache. +// +// `_granted` used to latch for the whole process the first time +// ensurePermission() ran, and NOTHING ever reset it. The prompt fires from +// AppState._persistPaired (right after pairing), so a user who tapped "Don't +// Allow" there, then went to OS Settings and enabled notifications, and came +// back, still got nothing: every presentEvent / scheduleDaily / scheduleWeekly / +// scheduleOnce early-returned on the stale `false`. Zero notifications and zero +// scheduled reminders until a full app restart. +// +// Fixed two ways, both covered below: +// • only a GRANT is cached — a denial is re-read from the live (non-prompting) +// OS state on the next call; +// • invalidatePermissionCache() drops a cached grant too, so a REVOCATION is +// noticed. app.dart calls it on every foreground resume. + +import 'package:flutter_test/flutter_test.dart'; + +import 'package:openstrap_edge/notify/notification_service.dart'; + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + + final svc = NotificationService.instance; + + setUp(() { + svc.invalidatePermissionCache(); + svc.debugRequestPermission = null; + svc.debugProbePermission = null; + }); + + tearDown(() { + svc.invalidatePermissionCache(); + svc.debugRequestPermission = null; + svc.debugProbePermission = null; + }); + + test('a denial is NOT cached forever — enabling in OS Settings takes effect ' + 'without an app restart', () async { + var osEnabled = false; + var prompts = 0; + svc.debugProbePermission = () async => osEnabled; + svc.debugRequestPermission = () async { + prompts++; + return osEnabled; // the user taps "Don't Allow" + }; + + expect(await svc.ensurePermission(), isFalse); + expect(prompts, 1); + + // User goes to Settings and switches notifications ON. + osEnabled = true; + + expect(await svc.ensurePermission(), isTrue, + reason: 'a stale cached denial must not survive the user enabling ' + 'notifications in OS Settings'); + // And we did NOT re-prompt to discover it (the OS no-ops a second request + // after a denial anyway — Settings is the only real path back). + expect(prompts, 1); + }); + + test('a denial that is still a denial keeps returning false (and does not ' + 're-prompt)', () async { + var prompts = 0; + svc.debugProbePermission = () async => false; + svc.debugRequestPermission = () async { + prompts++; + return false; + }; + expect(await svc.ensurePermission(), isFalse); + expect(await svc.ensurePermission(), isFalse); + expect(await svc.ensurePermission(), isFalse); + expect(prompts, 1); + }); + + test('a GRANT is cached — no repeated platform round-trips', () async { + var probes = 0, prompts = 0; + svc.debugProbePermission = () async { + probes++; + return true; + }; + svc.debugRequestPermission = () async { + prompts++; + return true; + }; + expect(await svc.ensurePermission(), isTrue); + expect(await svc.ensurePermission(), isTrue); + expect(prompts, 1); + expect(probes, 0); + }); + + test('invalidatePermissionCache re-reads a REVOKED grant', () async { + var osEnabled = true; + svc.debugProbePermission = () async => osEnabled; + svc.debugRequestPermission = () async => osEnabled; + + expect(await svc.ensurePermission(), isTrue); + + // Revoked in Settings while we were backgrounded. + osEnabled = false; + expect(await svc.ensurePermission(), isTrue, + reason: 'still the cached grant until the resume hook runs'); + + svc.invalidatePermissionCache(); // what app.dart does on resume + expect(await svc.ensurePermission(), isFalse); + }); + + test('allowPrompt:false never prompts, and a not-yet-decided state is not ' + 'cached as denied', () async { + var prompts = 0; + var osEnabled = false; + svc.debugProbePermission = () async => osEnabled; + svc.debugRequestPermission = () async { + prompts++; + return true; + }; + + // Headless caller (background_sync) checks, never requests. + expect(await svc.ensurePermission(allowPrompt: false), isFalse); + expect(prompts, 0); + + // A later foreground, contextual call still gets to ask. + osEnabled = true; + expect(await svc.ensurePermission(), isTrue); + expect(prompts, 1); + }); +} diff --git a/test/notification_schedule_dst_test.dart b/test/notification_schedule_dst_test.dart new file mode 100644 index 00000000..9ffad4f2 --- /dev/null +++ b/test/notification_schedule_dst_test.dart @@ -0,0 +1,139 @@ +// Regression tests for the scheduled-reminder wall-clock arithmetic. +// +// NotificationService._nextInstanceOf used to advance with absolute Durations: +// d = d.add(const Duration(days: 1)); // weekday walk +// d = d.add(Duration(days: weekday != null ? 7 : 1)); // roll forward +// A Duration is ELAPSED time, not a calendar day. Across a DST transition the +// wall-clock time therefore drifts by an hour: the Sunday-18:00 weekly recap +// computed over a spring-forward landed at 19:00 (and at 17:00 over a +// fall-back), and the bedtime / hydration dailies drifted the same way. +// +// nextInstanceOf now rebuilds the TZDateTime from its calendar fields, so the +// tz database resolves whatever UTC offset that day carries. + +import 'package:flutter_test/flutter_test.dart'; +import 'package:timezone/data/latest_all.dart' as tzdata; +import 'package:timezone/timezone.dart' as tz; + +import 'package:openstrap_edge/notify/notification_service.dart'; + +/// The pre-fix implementation, kept verbatim so each test can show the drift +/// it produced. +tz.TZDateTime _legacyNextInstanceOf(tz.TZDateTime now, int hour, int minute, + {int? weekday}) { + var d = tz.TZDateTime( + now.location, now.year, now.month, now.day, hour, minute); + if (weekday != null) { + while (d.weekday != weekday) { + d = d.add(const Duration(days: 1)); + } + } + if (!d.isAfter(now)) { + d = d.add(Duration(days: weekday != null ? 7 : 1)); + } + return d; +} + +void main() { + late tz.Location ny; + + setUpAll(() { + tzdata.initializeTimeZones(); + ny = tz.getLocation('America/New_York'); + }); + + group('weekly recap across a spring-forward', () { + // US DST 2026 starts Sunday 8 March. From Sunday 1 March 19:00 the next + // Sunday-18:00 instance is 8 March 18:00 EDT. + test('keeps the 18:00 wall-clock, not +168h of elapsed time', () { + final now = tz.TZDateTime(ny, 2026, 3, 1, 19, 0); + final next = + nextInstanceOf(now, 18, 0, weekday: DateTime.sunday); + expect(next.month, 3); + expect(next.day, 8); + expect(next.hour, 18); + expect(next.minute, 0); + expect(next.weekday, DateTime.sunday); + + // The old arithmetic drifted an hour late. + final legacy = + _legacyNextInstanceOf(now, 18, 0, weekday: DateTime.sunday); + expect(legacy.hour, 19, reason: 'guards the regression being fixed'); + }); + }); + + group('weekly recap across a fall-back', () { + // US DST 2026 ends Sunday 1 November. From Sunday 25 Oct 19:00 the next + // Sunday-18:00 instance is 1 Nov 18:00 EST. + test('keeps the 18:00 wall-clock, not −1h', () { + final now = tz.TZDateTime(ny, 2026, 10, 25, 19, 0); + final next = nextInstanceOf(now, 18, 0, weekday: DateTime.sunday); + expect(next.month, 11); + expect(next.day, 1); + expect(next.hour, 18); + + final legacy = + _legacyNextInstanceOf(now, 18, 0, weekday: DateTime.sunday); + expect(legacy.hour, 17, reason: 'guards the regression being fixed'); + }); + }); + + group('daily nudge across a DST boundary', () { + test('a 22:00 bedtime the evening before spring-forward stays 22:00', () { + // Saturday 7 March 2026 23:00 → next 22:00 is Sunday 8 March (DST day). + final now = tz.TZDateTime(ny, 2026, 3, 7, 23, 0); + final next = nextInstanceOf(now, 22, 0); + expect(next.day, 8); + expect(next.hour, 22); + expect(_legacyNextInstanceOf(now, 22, 0).hour, 23, + reason: 'guards the regression being fixed'); + }); + + test('nextCalendarDay (the skipToday roll) is DST-safe too', () { + final d = tz.TZDateTime(ny, 2026, 3, 7, 22, 0); + final rolled = nextCalendarDay(d); + expect(rolled.day, 8); + expect(rolled.hour, 22); + expect(d.add(const Duration(days: 1)).hour, 23, + reason: 'guards the regression being fixed'); + }); + }); + + group('ordinary (non-DST) behaviour is unchanged', () { + test('a time later today is today', () { + final now = tz.TZDateTime(ny, 2026, 6, 10, 9, 0); + final next = nextInstanceOf(now, 21, 30); + expect(next.day, 10); + expect(next.hour, 21); + expect(next.minute, 30); + }); + + test('a time already past rolls to tomorrow', () { + final now = tz.TZDateTime(ny, 2026, 6, 10, 22, 0); + final next = nextInstanceOf(now, 21, 30); + expect(next.day, 11); + expect(next.hour, 21); + }); + + test('month/year rollover is handled by calendar normalisation', () { + final now = tz.TZDateTime(ny, 2026, 12, 31, 23, 59); + final next = nextInstanceOf(now, 8, 0); + expect(next.year, 2027); + expect(next.month, 1); + expect(next.day, 1); + expect(next.hour, 8); + }); + + test('the weekday walk always lands on the requested weekday', () { + for (var day = 1; day <= 28; day++) { + final now = tz.TZDateTime(ny, 2026, 4, day, 12, 0); + for (var wd = DateTime.monday; wd <= DateTime.sunday; wd++) { + final next = nextInstanceOf(now, 18, 0, weekday: wd); + expect(next.weekday, wd); + expect(next.hour, 18); + expect(next.isAfter(now), isTrue); + } + } + }); + }); +} diff --git a/test/readiness_baseline_pollution_test.dart b/test/readiness_baseline_pollution_test.dart index cde94920..b87f3c51 100644 --- a/test/readiness_baseline_pollution_test.dart +++ b/test/readiness_baseline_pollution_test.dart @@ -170,4 +170,83 @@ void main() { reason: 'clean baseline → readiness computes on the FIRST derive, ' 'with no refresh required to self-heal'); }); + + // ── The SWEEP path: the "frozen" snapshot must actually stay frozen ──────── + // + // The load path above was fixed, but the SWEEP re-introduced the same + // pollution one level up: `run()`/`rescanRecent()` loaded the snapshot ONCE + // — already containing the persisted values of the up-to-21 days it was + // about to re-derive — and then appended each finished day's scalars back + // into that shared snapshot, evicting a real old day to stay at 28. Days + // 2..N of the sweep therefore read a window holding duplicate copies of the + // recent days, in DESCENDING date order (the sweep runs newest-first), and + // median/MAD collapsed toward the repeated values exactly as before. + + Future seedSweepDays(String month, int n) async { + await (await LocalDb.instance).delete('metric_series'); + for (var i = 1; i <= n; i++) { + // Distinct readiness AND distinct rhr per day, so "did this day's own + // value leak into its own window" is decidable by value. + await seedDay('2026-$month-${i.toString().padLeft(2, '0')}', 40.0 + i); + } + } + + test('a sweep never mutates the frozen snapshot — every day gets the same ' + 'window a fresh load would give', () async { + await seedSweepDays('09', 28); + // run()/rescanRecent dispatch NEWEST-FIRST over the recent window. + final orderedDays = [ + for (var i = 28; i >= 8; i--) '2026-09-${i.toString().padLeft(2, '0')}', + ]; + + final first = await debugSweepBaselineWindows('readiness', orderedDays); + final second = await debugSweepBaselineWindows('readiness', orderedDays); + expect(first, equals(second), + reason: 'the sweep snapshot is immutable — deriving days cannot change ' + 'what a later day in the same sweep reads'); + + for (var i = 0; i < orderedDays.length; i++) { + final window = first[i]; + expect(window.toSet().length, window.length, + reason: '${orderedDays[i]}: no value may appear twice — the old ' + 'append path stacked each finished day back in'); + final sorted = [...window]..sort(); + expect(window, equals(sorted), + reason: '${orderedDays[i]}: the window is a real trailing series in ' + 'date order, not history with recent days appended out of order'); + } + }); + + test('no day is ever inside its own baseline window', () async { + await seedSweepDays('10', 28); + final today = '2026-10-28'; + final ownValue = 40.0 + 28; + + // The unfiltered window (what backs the persisted artifact + the rescan + // signature) legitimately contains today... + final unfiltered = await debugBaselineWindow('readiness'); + expect(unfiltered, contains(ownValue)); + + // ...but the window the day is DERIVED against must not. + final own = (await debugSweepBaselineWindows('readiness', [today])).single; + expect(own, isNot(contains(ownValue)), + reason: "z-scoring a day against a baseline containing itself pulls " + 'the baseline toward the value under test — the exact ' + 'self-inclusion analytics v38 fixed one layer down'); + expect(own.length, 27, reason: 'strictly the 27 prior days'); + expect(own.every((v) => v < ownValue), isTrue, + reason: 'strictly EARLIER days — a backfill sweep must not leak later ' + "days into an older day's baseline (that would also make the " + 'result depend on sweep order)'); + }); + + test('a mid-history backfill day sees only days before it', () async { + await seedSweepDays('11', 28); + // Re-derive a day in the MIDDLE of history (what "Re-analyze" does). + final mid = '2026-11-10'; + final window = (await debugSweepBaselineWindows('readiness', [mid])).single; + expect(window.length, 9, reason: 'the 9 days 2026-11-01..09'); + expect(window.last, 40.0 + 9); + expect(window.every((v) => v < 40.0 + 10), isTrue); + }); } diff --git a/test/route_math_test.dart b/test/route_math_test.dart index ad9a8bbe..a6a9dda2 100644 --- a/test/route_math_test.dart +++ b/test/route_math_test.dart @@ -218,6 +218,57 @@ void main() { isEmpty, ); }); + + // REGRESSION: computeSplits walked raw haversine over every consecutive + // pair while totalDistanceMeters skipped implausible segments, so one GPS + // teleport across a tunnel gap made the route detail screen contradict its + // own headline — "5 km" above a list of ~60 splits, most of them phantom. + group('implausible segments (recording gaps)', () { + /// 5 km of real running, then a 55 km teleport, then more running. + List withTeleport() { + final pts = _line(count: 51, stepMeters: 100, stepSec: 30); // 5 000 m + final last = pts.last; + return [ + ...pts, + // 55 km in 30 s — far past kMaxPlausibleSpeedMps × gap. + RoutePoint( + seq: 51, + tsMs: last.tsMs + 30000, + lat: 0, + lng: last.lng + 55000 / _mPerDegLngAtEq, + ), + RoutePoint( + seq: 52, + tsMs: last.tsMs + 60000, + lat: 0, + lng: last.lng + (55000 + 100) / _mPerDegLngAtEq, + ), + ]; + } + + test('splits agree with totalDistanceMeters across a teleport', () { + final pts = withTeleport(); + final total = totalDistanceMeters(pts); + expect(total, closeTo(5100, 30)); // teleport contributes nothing + + final splits = computeSplits(pts, const [], unitMeters: 1000); + final splitSum = splits.fold(0, (a, s) => a + s.meters); + expect(splitSum, closeTo(total, 1), + reason: 'headline distance and the splits list must not disagree'); + // 5 full km + a short trailing partial — NOT ~60 phantom splits. + expect(splits.length, 6); + expect(splits.last.meters, closeTo(100, 30)); + }); + + test('a plausible fast segment is still counted', () { + // 300 m in 30 s (10 m/s) is fast but real — must not be filtered. + final pts = _line(count: 11, stepMeters: 300, stepSec: 30); // 3 000 m + final splits = computeSplits(pts, const [], unitMeters: 1000); + expect(splits.fold(0, (a, s) => a + s.meters), + closeTo(totalDistanceMeters(pts), 1)); + expect(splits.length, 3); + }); + }); }); group('emaSpeed', () { diff --git a/test/route_tracker_test.dart b/test/route_tracker_test.dart index 33db2278..faaa3fb1 100644 --- a/test/route_tracker_test.dart +++ b/test/route_tracker_test.dart @@ -486,4 +486,82 @@ void main() { unawaited(ctrl.close()); }); }); + + // REGRESSION: dispose() cancelled only the watchdog. The GPS StreamSubscription + // stayed live and `_stopped` stayed false, so a dispose() without a stop() + // left the location stream (and the platform GPS session behind it) running + // for the life of the process — and nothing ever called dispose() at all, so + // the six ValueNotifiers and their listeners leaked once per route workout. + group('lifecycle: dispose', () { + test('dispose() cancels the GPS subscription', () async { + var cancels = 0; + final ctrl = StreamController(onCancel: () => cancels++); + final t = RouteTracker(sink: (_) async {}, batchSize: 100); + t.start(ctrl.stream); + ctrl.add(_fix(0)); + await pumpEventQueue(); + expect(t.isRunning, isTrue); + + t.dispose(); + await pumpEventQueue(); + + expect(cancels, 1, reason: 'the location stream stayed subscribed'); + expect(t.isRunning, isFalse); + await ctrl.close(); + }); + + test('dispose() without stop() ignores any in-flight sample', () async { + final ctrl = StreamController(); + final seen = >[]; + final t = RouteTracker(sink: (b) async => seen.add(b), batchSize: 1); + t.start(ctrl.stream); + t.dispose(); + ctrl.add(_fix(0)); + ctrl.addError(StateError('location service died')); + await pumpEventQueue(); // must not touch a disposed ValueNotifier + expect(seen, isEmpty); + await ctrl.close(); + }); + + test('stop() disposes, so a tracker is fully released after one workout', + () async { + var cancels = 0; + final ctrl = StreamController(onCancel: () => cancels++); + final flushed = []; + final t = RouteTracker( + sink: (b) async => flushed.addAll(b), + batchSize: 100, // nothing flushes until stop() + ); + t.start(ctrl.stream); + ctrl.add(_fix(0)); + ctrl.add(_fix(1)); + await pumpEventQueue(); + + await t.stop(); + + expect(cancels, 1); + expect(flushed.length, 2, reason: 'the tail must still be persisted'); + // The notifiers are gone: writing to a disposed ValueNotifier throws. + expect(() => t.path.value = const [], throwsA(isA())); + await ctrl.close(); + }); + + test('dispose() and stop() are both idempotent, in either order', () async { + final ctrl = StreamController(); + final t = RouteTracker(sink: (_) async {}, batchSize: 100); + t.start(ctrl.stream); + await t.stop(); + await t.stop(); // no double-dispose crash + t.dispose(); + t.dispose(); + + final ctrl2 = StreamController(); + final t2 = RouteTracker(sink: (_) async {}, batchSize: 100); + t2.start(ctrl2.stream); + t2.dispose(); + await t2.stop(); // stop after dispose must not throw + await ctrl.close(); + await ctrl2.close(); + }); + }); } diff --git a/test/telemetry_consent_default_test.dart b/test/telemetry_consent_default_test.dart new file mode 100644 index 00000000..26f0a6cb --- /dev/null +++ b/test/telemetry_consent_default_test.dart @@ -0,0 +1,149 @@ +// ZERO COLLECTION BEFORE CONSENT. +// +// OpenStrap's store builds collect nothing, ever. The Firebase SDKs, however, +// auto-collect from process start unless the PLATFORM config says otherwise: +// Analytics logs first_open/session_start, Performance opens app-start and +// network traces, Crashlytics uploads any startup crash. All of that happens +// before Dart has read the user's stored telemetry consent (an async +// SharedPreferences load that is deliberately off the startup critical path), +// so the runtime setters alone were not a gate at all. +// +// Two things are asserted here: +// 1. the native config in BOTH platforms disables auto-collection, and does +// it with the runtime-overridable "…_enabled = false" keys rather than the +// permanent "…deactivated" ones (which consent could never re-enable); +// 2. the Dart seam only ever switches collection ON from loaded consent. +import 'dart:io'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_edge/telemetry/telemetry_service.dart'; + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + + group('platform config: Firebase auto-collection is off by default', () { + final plist = File('ios/Runner/Info.plist').readAsStringSync(); + final manifest = + File('android/app/src/main/AndroidManifest.xml').readAsStringSync(); + + /// The Info.plist value immediately following [key]. + String? plistValueAfter(String key) { + final i = plist.indexOf('$key'); + if (i < 0) return null; + final rest = plist.substring(i + '$key'.length); + final m = RegExp(r'<(true|false|string|integer)\s*/?>').firstMatch(rest); + return m?.group(1); + } + + for (final key in const [ + 'FIREBASE_ANALYTICS_COLLECTION_ENABLED', + 'FirebaseCrashlyticsCollectionEnabled', + 'firebase_performance_collection_enabled', + ]) { + test('Info.plist sets $key to false', () { + expect(plistValueAfter(key), 'false', + reason: 'iOS would auto-collect before consent exists'); + }); + } + + test('Info.plist does not use the PERMANENT deactivation keys', () { + // Those cannot be re-enabled at runtime, which would break the opt-in + // path entirely rather than gate it. + expect(plist, isNot(contains('FIREBASE_ANALYTICS_COLLECTION_DEACTIVATED'))); + expect(plist, + isNot(contains('firebase_performance_collection_deactivated'))); + }); + + for (final key in const [ + 'firebase_analytics_collection_enabled', + 'firebase_crashlytics_collection_enabled', + 'firebase_performance_collection_enabled', + ]) { + test('AndroidManifest sets $key to false', () { + final m = RegExp( + '[]; + + setUp(() { + applied.clear(); + TelemetryService.debugCollectionSink = applied.add; + TelemetryService.instance.debugResetConsent(); // fresh-install state + }); + tearDown(() { + TelemetryService.debugCollectionSink = null; + TelemetryService.instance.debugResetConsent(); + }); + + test('a fresh install starts disabled with consent unresolved', () { + final t = TelemetryService.instance; + expect(t.enabled, isFalse); + expect(t.consentResolved, isFalse); + }); + + test('enforceCollectionOffUntilConsent pushes false to every SDK', () { + final t = TelemetryService.instance; + t.enforceCollectionOffUntilConsent(); // what main() calls at startup + expect(applied, [false]); + expect(t.enabled, isFalse); + expect(t.consentResolved, isFalse); // still unresolved — it is not consent + }); + + test('enforceCollectionOffUntilConsent never revokes a loaded opt-in', () { + final t = TelemetryService.instance; + t.applyConsent(true); + applied.clear(); + t.enforceCollectionOffUntilConsent(); + expect(applied, isEmpty); + expect(t.enabled, isTrue); + }); + + test('applyConsent(false) keeps every SDK off', () { + final t = TelemetryService.instance; + applied.clear(); + t.applyConsent(false); + expect(applied, [false]); + expect(t.enabled, isFalse); + expect(t.consentResolved, isTrue); + }); + + test('applyConsent(true) is the ONLY thing that enables collection', () { + final t = TelemetryService.instance; + applied.clear(); + t.applyConsent(true); + expect(applied, [true]); + expect(t.enabled, isTrue); + // …and it can be revoked again. + applied.clear(); + t.applyConsent(false); + expect(applied, [false]); + expect(t.enabled, isFalse); + }); + + test('a consent-less session never transmits: flush() is a no-op', () async { + final t = TelemetryService.instance; + t.applyConsent(false); + t.deviceId = 'test-device'; + t.record(kind: 'event', level: 'info', message: 'unit_test_event'); + // No network client is configured in tests; flush must return without + // attempting anything rather than throwing. + await expectLater(t.flush(), completes); + expect(t.enabled, isFalse); + }); + }); +} diff --git a/test/widget_service_sentinels_test.dart b/test/widget_service_sentinels_test.dart new file mode 100644 index 00000000..976f4812 --- /dev/null +++ b/test/widget_service_sentinels_test.dart @@ -0,0 +1,70 @@ +// The home-widget / watch snapshot is a WRITE-ONLY surface: whatever Dart puts +// in the App Group is what the user sees on their lock screen, with no chance +// to notice it was invented. Every int key uses -1 for "no data" and the native +// readers gate on it — so a nullable metric must never be written as a plausible +// default instead. + +import 'package:flutter/services.dart'; +import 'package:flutter_test/flutter_test.dart'; + +import 'package:openstrap_edge/models/payloads.dart'; +import 'package:openstrap_edge/widget/widget_service.dart'; + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + + late Map written; + + setUp(() { + written = {}; + final messenger = + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger; + messenger.setMockMethodCallHandler(const MethodChannel('home_widget'), + (call) async { + if (call.method == 'saveWidgetData') { + final args = (call.arguments as Map).cast(); + written[args['id'] as String] = args['data']; + } + return true; + }); + messenger.setMockMethodCallHandler( + const MethodChannel('openstrap/ios_config'), + (call) async => call.method == 'appGroupIdentifier' ? 'group.test' : null); + }); + + tearDown(() { + final messenger = + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger; + messenger.setMockMethodCallHandler( + const MethodChannel('home_widget'), null); + messenger.setMockMethodCallHandler( + const MethodChannel('openstrap/ios_config'), null); + }); + + test('an underived sleep need is written as the -1 sentinel, not a fabricated ' + '8h00m', () async { + await WidgetService.push(TodayData.fromJson({ + 'daily': { + 'readiness': {'value': 74}, + }, + // duration_min is known; need_min is NOT derived yet. + 'sleep': {'duration_min': 437}, + })); + + expect(written['sleep_min'], 437); + // 480 here put a hard 8h00m need on the home widget AND the watch, and the + // sleep ring was drawn as a fraction of that invented denominator. + expect(written['sleep_need_min'], -1); + // The convention it now matches. + expect(written['rhr'], -1); + expect(written['hrv'], -1); + }); + + test('a real sleep need is still written through', () async { + await WidgetService.push(TodayData.fromJson({ + 'daily': const {}, + 'sleep': {'duration_min': 437, 'need_min': 462}, + })); + expect(written['sleep_need_min'], 462); + }); +} From 23aa39b4bcb0c3d01624d4fe180dcb5de8cd7ce6 Mon Sep 17 00:00:00 2001 From: abdulsaheel Date: Sun, 26 Jul 2026 18:32:59 +0530 Subject: [PATCH 2/5] fix: green CI, and stop the coach history stranding a tool message MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI failures (both real, neither caught locally): - `_denseDailyTrimp` turned "no observed load at all" into a synthetic all-zero series, so a user with no workouts got a confident CTL/ATL/TSB of 0 instead of an absent load model — the exact fabrication this sweep removes elsewhere. It surfaced only on CI because CI resolves the PINNED analytics (where ctlAtlTsb has no minimum-history gate) while local dev resolves the working copy through pubspec_overrides. Now abstains, which is correct against BOTH the pinned and the updated analytics, so the test no longer depends on which one is resolved. - The tsb/ctl-atl equality assertion used a 1e-6 tolerance. The three values round-trip through JSON independently, so the reconstructed difference can differ from the stored tsb by a ULP, and which way it lands is architecture-dependent: it passed on arm64 macOS and failed on x64 Linux by exactly 1e-6. Loosened to 1e-4, which still pins the relationship without asserting bit-identical floats across platforms. Review feedback (PR Agent, edge#158): - `_trimHistory` promised in its docstring that a `tool` message never outlives the assistant turn whose `tool_calls` it answers, but enforced it only as a side effect of the loop bounds. Both loops stop at `length > 1`, so a single turn larger than the whole byte budget walked the history down to one element: [assistant(tool_calls), tool] dropped the assistant and left the orphan. Providers 400 on that, and because the history is persisted the rejection repeats every turn — the session is bricked, not just one reply. Now enforced unconditionally. --- lib/coach/coach_engine.dart | 19 ++++++ lib/compute/crossday_pipeline.dart | 6 ++ test/coach_history_trim_test.dart | 106 +++++++++++++++++++++++++++++ test/crossday_pipeline_test.dart | 10 ++- 4 files changed, 139 insertions(+), 2 deletions(-) create mode 100644 test/coach_history_trim_test.dart diff --git a/lib/coach/coach_engine.dart b/lib/coach/coach_engine.dart index d60fe120..40a2d529 100644 --- a/lib/coach/coach_engine.dart +++ b/lib/coach/coach_engine.dart @@ -14,6 +14,7 @@ import 'dart:convert'; import 'dart:io'; import 'dart:math'; +import 'package:flutter/foundation.dart'; import 'package:http/http.dart' as http; import 'package:path_provider/path_provider.dart'; @@ -170,6 +171,15 @@ class CoachEngine { // Serializable display transcript (text bubbles + charts) shown in the UI. final List transcript = []; + /// Test seams for the history-trimming invariant. `_history` is private and + /// the trim only runs deep inside the tool-calling loop, so there is no other + /// way to pin the orphaned-`tool` edge case without a live provider. + @visibleForTesting + List> get debugHistory => _history; + + @visibleForTesting + void debugTrimHistory() => _trimHistory(); + // Current session identity (sessions are persisted per-user, many per user). String _sessionId = ''; String _title = ''; @@ -234,6 +244,15 @@ class CoachEngine { _history.removeAt(0); } } + // Both loops stop at `length > 1`, so one turn larger than the whole budget + // can strand a lone `tool` at the head: [assistant(tool_calls), tool] drops + // the assistant and then has nothing left to pair with. That orphan is the + // exact shape this method exists to prevent, and providers 400 on it, so + // enforce the invariant unconditionally rather than as a side effect of the + // loop bounds. + while (_history.isNotEmpty && _history.first['role'] == 'tool') { + _history.removeAt(0); + } } void reset() { _history.clear(); transcript.clear(); } diff --git a/lib/compute/crossday_pipeline.dart b/lib/compute/crossday_pipeline.dart index 55a94bad..c4b08c93 100644 --- a/lib/compute/crossday_pipeline.dart +++ b/lib/compute/crossday_pipeline.dart @@ -391,6 +391,12 @@ const int _maxDenseTrimpDays = 400; /// PURE: `DateTime.parse` / `DateTime(y, m, d + 1)` are deterministic functions /// of the input strings, and the day-step normalizes DST-length days correctly. List _denseDailyTrimp(List dates, List trimps) { + // NO observed load at all is not "a run of zero-load days" — it is no load + // history. Densifying it would hand the EWMA a synthetic all-zero series and + // turn "we have never seen a workout" into a confident CTL/ATL/TSB of 0, + // which is the fabrication this whole pass exists to remove. Abstain by + // handing back an empty series and let the load model decline. + if (!trimps.any((t) => t != null)) return const []; final byDate = {}; String? first, last; for (var i = 0; i < dates.length && i < trimps.length; i++) { diff --git a/test/coach_history_trim_test.dart b/test/coach_history_trim_test.dart new file mode 100644 index 00000000..1f18fd99 --- /dev/null +++ b/test/coach_history_trim_test.dart @@ -0,0 +1,106 @@ +// CoachEngine._trimHistory — the resent-context size bound. +// +// The trim drops WHOLE turns from the oldest end specifically so that a `tool` +// message never outlives the assistant turn whose `tool_calls` it answers: +// OpenAI-compatible providers reject an orphaned tool message with a 400, and +// once the history is persisted that rejection repeats on every subsequent turn +// of the session — the conversation is bricked, not just one reply. +// +// That invariant used to be a side effect of the loop bounds rather than +// something the code enforced. Both loops stop at `length > 1`, so a single +// turn larger than the whole byte budget walked the history down to exactly one +// element and left it there — and if that survivor was the `tool` half of a +// pair, the very orphan the method exists to prevent was what got sent. +import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_edge/coach/coach_config.dart'; +import 'package:openstrap_edge/coach/coach_engine.dart'; +import 'package:openstrap_edge/data/local_repository.dart'; + +class _FakeRepo extends LocalRepository {} + +/// A message whose encoded size comfortably exceeds the whole history budget, +/// so the trim is forced to walk all the way down. +Map _huge(String role) => { + 'role': role, + 'content': 'x' * (CoachEngine.kMaxHistoryChars + 1000), + }; + +void main() { + late CoachEngine engine; + + setUp(() { + engine = CoachEngine(config: CoachConfig(), api: _FakeRepo()); + }); + + group('CoachEngine history trimming never strands a tool message', () { + test('an oversized assistant turn does not leave its tool reply orphaned', + () { + // The exact reachable shape: one assistant turn bigger than the entire + // budget, followed by the tool result answering its tool_calls. Dropping + // the assistant strands the tool with nothing to pair against. + engine.debugHistory.addAll([ + _huge('assistant'), + {'role': 'tool', 'tool_call_id': 'call_1', 'content': 'sleep data'}, + ]); + + engine.debugTrimHistory(); + + // Pre-fix this asserted-out: the history was exactly [tool]. + expect( + engine.debugHistory.any((m) => m['role'] == 'tool'), + isFalse, + reason: 'a tool message must never survive without its assistant turn', + ); + }); + + test('the surviving history never BEGINS with a tool message', () { + engine.debugHistory.addAll([ + {'role': 'user', 'content': 'how did I sleep?'}, + _huge('assistant'), + {'role': 'tool', 'tool_call_id': 'call_1', 'content': 'sleep data'}, + {'role': 'assistant', 'content': 'You slept well.'}, + ]); + + engine.debugTrimHistory(); + + if (engine.debugHistory.isNotEmpty) { + expect(engine.debugHistory.first['role'], isNot('tool')); + } + }); + + test('a history already under the ceiling is left completely alone', () { + // Guards against over-correction: the orphan sweep must not eat a + // legitimate, correctly-paired turn that was never over budget. + final intact = >[ + {'role': 'user', 'content': 'how did I sleep?'}, + {'role': 'assistant', 'content': null, 'tool_calls': const []}, + {'role': 'tool', 'tool_call_id': 'call_1', 'content': 'sleep data'}, + {'role': 'assistant', 'content': 'You slept well.'}, + ]; + engine.debugHistory.addAll(intact); + + engine.debugTrimHistory(); + + expect(engine.debugHistory, hasLength(intact.length)); + expect(engine.debugHistory.first['role'], 'user'); + expect(engine.debugHistory.any((m) => m['role'] == 'tool'), isTrue); + }); + + test('trimming a long well-formed history keeps a user message at the head', + () { + for (var i = 0; i < 40; i++) { + engine.debugHistory.addAll([ + {'role': 'user', 'content': 'q$i ${'x' * 4000}'}, + {'role': 'assistant', 'content': null, 'tool_calls': const []}, + {'role': 'tool', 'tool_call_id': 'c$i', 'content': 'r$i'}, + {'role': 'assistant', 'content': 'a$i'}, + ]); + } + + engine.debugTrimHistory(); + + expect(engine.debugHistory, isNotEmpty); + expect(engine.debugHistory.first['role'], 'user'); + }); + }); +} diff --git a/test/crossday_pipeline_test.dart b/test/crossday_pipeline_test.dart index 676bc163..22834cba 100644 --- a/test/crossday_pipeline_test.dart +++ b/test/crossday_pipeline_test.dart @@ -287,8 +287,14 @@ void main() { expect(atl, lessThan(100.0), reason: 'fatigue must decay across 8 consecutive rest days'); expect(ctl, lessThan(150.0)); - // TSB = ctl - atl must be a real (non-degenerate) form number. - expect((load['tsb'] as num).toDouble(), closeTo(ctl - atl, 1e-6)); + // TSB = ctl - atl must be a real (non-degenerate) form number. The + // tolerance is deliberately looser than 1e-6: ctl/atl/tsb round-trip + // through JSON independently, so the reconstructed difference can differ + // from the stored tsb by a ULP, and which way it lands is + // platform-dependent (this passed on arm64 macOS and failed on x64 Linux + // CI by exactly 1e-6). 1e-4 still pins the relationship without + // asserting bit-level float reproducibility across architectures. + expect((load['tsb'] as num).toDouble(), closeTo(ctl - atl, 1e-4)); }); test('an every-day-trained series is unchanged by densification', () { From b5c3352beadedde1fc42eb79aa26916afdb8926c Mon Sep 17 00:00:00 2001 From: abdulsaheel Date: Sun, 26 Jul 2026 19:13:27 +0530 Subject: [PATCH 3/5] fix: anchor the 1 Hz activity estimate on a personal baseline, and record real live-pedometer coverage MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Diagnosed against a real user database, where the app reported 39,384 steps for a day whose true value was ~2,000 (yesterday: 46). The 1 Hz estimator keyed off ENMO, `mean(max(0, |a| - gRef))`, with gRef auto-calibrated per day from the stillest samples — which are the long sleep block, where the wrist sits in a different orientation than it does awake. That day gRef came out at 0.9797 against ~1.032 on every other day (hourly median |a| was 0.94-0.99 asleep vs 1.03-1.06 awake). ENMO subtracts gRef from every sample, so a reference 0.05 g low adds 0.05 g to every minute of the day — exactly the 0.05 g walking floor. Ordinary sitting cleared the gate for hours: 368 "ambulatory" minutes on a partial day. Sweeping gRef over the identical samples gave 42,155 steps at 0.97 and 0 at 1.02. The signal and the calibration error are the same size, so no threshold could have fixed it; the analytics package now decides activity from a per-axis high-passed dynamic amplitude, where any per-axis offset or gain error cancels exactly. This is the edge half. PERSONAL FLOOR. The new feature needs to know what "moving" means for this wearer, and analytics is pure so it cannot read history. Each day now persists `dyn_p90` — its own high quantile of the dynamic amplitude — and the next day's derive anchors on the MEDIAN across trailing days, with the same strict self-exclusion every other baseline uses. Validated on the real database: per-day values sit at 0.40-0.47 across ten days and the resulting floor moves only between 0.4493 and 0.4575, absorbing a day whose own value fell to 0.3148. A same-day floor was measured to fail in the opposite direction (it collapses on a quiet day and passes everything), which is why the anchor is multi-day and a median rather than a pooled quantile. Below the minimum history there is NO floor and the estimator abstains, so a new user sees the real 100 Hz count and no invented remainder. That matters here: on this database Tier B contributed exactly 0 on every normal day and 39,368 on the one drifted day — the stored total equalled the live count exactly (46=46, 119+18+12=149, 105+201+103=409). It has never once produced a useful number. ACTIVE MINUTES. Persisted as a first-class series and surfaced as the primary quantity. Gait is 1.4-2.5 Hz and 120 spm aliases to DC at 1 Hz, so steps are not resolvable from this substrate; minutes are, and minutes of moderate activity is the unit public activity guidance uses. Steps are now reported as a range over the free-living cadence band. LIVE COVERAGE. `live_coverage` windows were being written with end_ts == start_ts — 22 of 24 rows in the real database, including one claiming 1,657 real steps over 0 seconds. The window was derived purely from the band record timestamp carried on live frames, which does not advance across most sessions. Two consequences: the no-double-count exclusion excluded ~1 minute instead of the streamed period, and there was no way to align real 100 Hz counts to 1 Hz minutes — destroying the only ground truth available to calibrate the estimator. The window is now derived by a pure policy: anchored in the band record-time base (what coverageWindowsOverlapping compares against), with the duration taken from the 100 Hz sample count and the ingest wall hull, and raised to the time the claimed steps could physically span. It can never be zero. The persistence guard repairs a degenerate window rather than rejecting it, so a real count is never lost. Historical rows are left alone — their true durations were never recorded and inventing them would swap one wrong extent for another. kAlgoVersion 48 -> 49. The sibling analytics fixes are still NOT pinned; pubspec.yaml must be repinned and this bumped again once that PR merges. --- lib/compute/derivation_engine.dart | 132 ++++++++++- lib/data/db.dart | 28 ++- lib/data/live_coverage_policy.dart | 164 +++++++++++++ lib/state/app_state.dart | 69 +++++- test/live_coverage_window_test.dart | 343 ++++++++++++++++++++++++++++ test/step_personal_floor_test.dart | 124 ++++++++++ 6 files changed, 839 insertions(+), 21 deletions(-) create mode 100644 lib/data/live_coverage_policy.dart create mode 100644 test/live_coverage_window_test.dart create mode 100644 test/step_personal_floor_test.dart diff --git a/lib/compute/derivation_engine.dart b/lib/compute/derivation_engine.dart index 7f59d3f3..62cbfc39 100644 --- a/lib/compute/derivation_engine.dart +++ b/lib/compute/derivation_engine.dart @@ -295,7 +295,41 @@ import 'substrate.dart'; // `git show :`. Citing an analytics change that the pinned SHA never // contained is exactly how the v43 changelog documented a readiness fix that // stayed broken for three releases; when the pin moves, bump again. -const int kAlgoVersion = 48; +// v49: steps/activity rebuilt on a calibration-invariant feature. +// +// Diagnosed on a real user database: the day reported 39,384 steps against a +// true value of ~2,000. ENMO is `mean(max(0, |a| - gRef))` and gRef is +// auto-calibrated per day from the stillest samples — which are the long sleep +// block, where the wrist sits in a different orientation. gRef came out at +// 0.9797 that day vs ~1.032 on every other, and since ENMO subtracts it from +// every sample, a reference 0.05 g low adds 0.05 g to every minute: exactly the +// 0.05 g walking floor. Sweeping gRef over the identical samples gave 42,155 +// steps at 0.97 and 0 at 1.02 — the signal and the calibration error are both +// ~0.05 g, so no threshold could have fixed it. +// +// The analytics package now decides activity from the per-axis high-passed +// dynamic amplitude (gravity is DC in the sensor frame; any per-axis offset or +// gain error cancels exactly), anchored on a PERSONAL floor pooled from +// trailing days rather than an absolute constant or a same-day baseline — both +// of which were measured to fail, in opposite directions. +// +// Edge side of that change: +// - `dyn_p90` joins the baseline series: each day persists its own high +// quantile of the dynamic amplitude, and the next day's derive takes the +// MEDIAN across trailing days (self-excluded, like every other baseline) as +// its floor. Below the minimum history the estimator ABSTAINS — a day with +// no personal baseline now reports the real 100 Hz count only, instead of a +// fabricated 1 Hz number. +// - `active_min` is persisted as a first-class series. Minutes are what 1 Hz +// can resolve; steps are derived from them as a range. +// - The steps bundle carries the range + the floor used, and says plainly +// when the 1 Hz estimate is absent. +// +// This bump also re-derives days whose stored step figure came from the old +// estimator. NOTE: the sibling analytics fixes still are NOT pinned — see the +// v48 note below; pubspec.yaml must be repinned and this bumped again once the +// analytics PR merges. +const int kAlgoVersion = 49; /// Raw is kept this many days past derivation, then pruned (derived stays). const int rawRetentionDays = 3; @@ -410,6 +444,12 @@ class _BaselineHistoryCache { 'resp_rate', 'skin_temp_adc', 'readiness', + // Per-day high quantile of the calibration-invariant dynamic accel + // amplitude. The 1 Hz activity estimator's floor is anchored on the MEDIAN + // of this series across trailing days, never on a same-day value: a + // single-day threshold collapses on a quiet day and passes everything, + // which is the mirror image of the absolute-constant failure it replaced. + 'dyn_p90', ]; /// DATED baseline samples, ascending by date, one entry per day (metric_series @@ -1743,6 +1783,15 @@ class DerivationEngine { final stepCalib = await LocalDb.getStepCalibration(); final savedSessions = await LocalDb.sessionsInRange(dayLo, dayHi); + // PERSONAL ambulatory floor, from days STRICTLY BEFORE this one (the same + // self-exclusion every other baseline uses — a day must not help set the + // threshold it is then scored against). Anchoring on trailing days is the + // whole point: an absolute g constant is destroyed by a few-percent + // gravity-reference excursion, and a same-day floor collapses on a quiet + // day. Below the minimum history this is null and the estimator abstains. + final dynHistory = history.valuesBefore('dyn_p90', day.date); + final dynFloorG = ana.personalDynFloorFromDailySummaries(dynHistory); + // Built on THIS isolate so the Isolate.run closure captures only this plain // sendable object (never `this`, `day`, or `bundle`). final blocksInput = _DayBlocksInput( @@ -1756,6 +1805,8 @@ class DerivationEngine { coverageWindows: coverageWindows, liveStepsReal: liveStepsReal, stepCalib: stepCalib, + dynFloorG: dynFloorG, + dynHistoryDays: dynHistory.length, savedSessions: savedSessions, date: day.date, dayEndSec: day.endSec, @@ -1870,6 +1921,14 @@ class DerivationEngine { // Steps = real 100 Hz count + 1 Hz estimate over uncovered minutes // (computed in _stepsAndEnergy; never double-counted). 'steps': sc('steps'), + // Ambulatory minutes — the quantity 1 Hz can actually resolve, and the + // unit public activity guidance uses. Steps are derived FROM this. + 'active_min': sc('active_min'), + // This day's high quantile of the calibration-invariant dynamic accel + // amplitude. Not a user-facing metric: it is the per-day summary the + // NEXT day's derive pools to anchor its personal ambulatory floor, so + // the threshold never depends on a single day (see _BaselineHistoryCache). + 'dyn_p90': sc('dyn_p90'), 'calories_total': sc('calories_total'), // Daytime nap minutes (principled van Hees + HR-dip) → trend + Sleep Coach. 'nap_min': sc('nap_min'), @@ -2520,6 +2579,8 @@ class DerivationEngine { List> coverageWindows, int liveStepsReal, ana.StepCalibration? stepCalib, + double? dynFloorG, + int dynHistoryDays, ) { try { if (daySub.length < 60) return; @@ -2527,6 +2588,12 @@ class DerivationEngine { if (motion.isEmpty) return; final hrPerMin = _hrPerMinuteAligned(motion, daySub); + // This day's own contribution to the personal floor, persisted to + // metric_series so tomorrow's derive can anchor on it. Null for a day too + // thin to summarise — we store nothing rather than a fabricated level. + final dynSummary = ana.dailyDynSummary(motion); + if (dynSummary != null) scMap?['dyn_p90'] = dynSummary; + // STEPS — hybrid, no double-count. Drop any minute already covered by a // 100 Hz window (real count wins), estimate steps for the rest from 1 Hz. bool covered(double tsMinStartMs) { @@ -2548,29 +2615,52 @@ class DerivationEngine { final rhr = (scMap?['rhr'] as num?)?.toDouble(); final est = ana.dailyStepEstimate( motionUn, + personalDynFloorG: dynFloorG, hrPerMin: hrUn, restingHr: rhr, calib: stepCalib, + pooledMinutesAvailable: dynHistoryDays, ); - final estSteps = est.present ? est.value!.steps : 0; + final v = est.present ? est.value : null; + final estSteps = v?.steps ?? 0; final daySteps = liveStepsReal + estSteps; scMap?['steps'] = daySteps.toDouble(); + // ACTIVE MINUTES is the primary, honest quantity here: 1 Hz cannot count + // steps (gait is 1.4-2.5 Hz and 120 spm aliases to DC at this rate), but + // it can resolve ambulatory MINUTES, which is also the unit public + // activity guidance is written in. The step figures are a RANGE over the + // free-living cadence band, and are absent entirely when the personal + // floor has not been established yet. bundle['steps'] = { 'value': daySteps, 'real_100hz': liveStepsReal, // AN-2554 over live windows (real count) - 'estimated_1hz': - estSteps, // walking-min × cadence for uncovered minutes - 'ambulatory_min': est.present ? est.value!.ambulatoryMinutes : 0, - 'cadence_used_spm': est.present ? est.value!.cadenceUsed : 0, + 'estimated_1hz': estSteps, // midpoint of the 1 Hz range + 'estimated_1hz_low': v?.stepsLow, + 'estimated_1hz_high': v?.stepsHigh, + 'active_min': v?.activeMinutes ?? 0, + 'cadence_low_spm': v?.cadenceLowSpm, + 'cadence_high_spm': v?.cadenceHighSpm, + 'dyn_floor_g': v?.dynFloorG, + 'estimate_present': v != null, 'confidence': liveStepsReal > 0 ? 0.7 : (est.present ? est.confidence : 0.2), 'tier': liveStepsReal > 0 && estSteps == 0 ? 'HIGH' : 'ESTIMATE', - 'inputs_used': const ['live_100hz_pedometer', 'enmo_1hz', 'hr_1hz'], - 'note': - 'real 100 Hz count for streamed time + 1 Hz walking estimate for ' - 'the rest (1 Hz cannot count steps directly)', + 'inputs_used': const [ + 'live_100hz_pedometer', + 'dyn_amp_1hz', + 'hr_1hz', + 'personal_dyn_floor', + ], + 'note': v == null + ? 'real 100 Hz count only — the 1 Hz activity estimate needs a ' + 'personal movement baseline from several days of wear ' + '(${est.note ?? 'need_baseline'})' + : 'real 100 Hz count for streamed time + ${v.activeMinutes} active ' + 'minutes estimated from 1 Hz for the rest (1 Hz cannot count ' + 'steps directly, so the step figure is a range)', }; + if (v != null) scMap?['active_min'] = v.activeMinutes.toDouble(); if (profile.isComplete) { final perMinFull = [ for (final h in hrPerMin) @@ -2627,6 +2717,7 @@ class DerivationEngine { required int sleepOnsetSec, required int sleepOffsetSec, double? restingHr, + double? dynFloorG, }) { final activeMin = _activeMinutes(daySub, sleepOnsetSec, sleepOffsetSec); final wear = _wearBlock(daySub); @@ -2682,8 +2773,15 @@ class DerivationEngine { // Steps do NOT need a profile: `dailyStepEstimate` falls back to the day's // own 10th-percentile HR when `restingHr` is null, which is data-derived, // not imputed. Pass the real value or nothing — never the old 60.0. + // + // This is the EARLY-READ path (what Today shows before the full day result + // exists); `_stepsAndEnergy` recomputes and overwrites it with the hybrid + // real-100 Hz + 1 Hz figure moments later. Without a personal floor the + // estimator abstains and `steps` stays null here, which is correct — the + // early read then shows no step figure rather than a fabricated one. final stepMetric = ana.dailyStepEstimate( motion, + personalDynFloorG: dynFloorG, hrPerMin: hrPerMinAll, restingHr: rhrForTrimp, ); @@ -3428,6 +3526,7 @@ class DerivationEngine { sleepOnsetSec: onset, sleepOffsetSec: offset, restingHr: inp.rhr, + dynFloorG: inp.dynFloorG, ); _applyWakeDayFeatures(bundlePatch, scMap, wake); _stepsAndEnergy( @@ -3438,6 +3537,8 @@ class DerivationEngine { inp.coverageWindows, inp.liveStepsReal, inp.stepCalib, + inp.dynFloorG, + inp.dynHistoryDays, ); // _stepsAndEnergy just corrected `steps`/`calories_total` in bundlePatch + // scMap using the hybrid real-100Hz + 1Hz-estimate count, but `wake` (built @@ -3835,6 +3936,15 @@ class _DayBlocksInput { final List> coverageWindows; final int liveStepsReal; final ana.StepCalibration? stepCalib; + + /// PERSONAL ambulatory floor (g, dynAmp units) from trailing days, or null + /// when there isn't enough history yet — in which case the 1 Hz estimator + /// abstains rather than falling back to a constant. Computed on the main + /// isolate (it needs metric_series) and carried in, like the other history. + final double? dynFloorG; + + /// How many trailing days backed [dynFloorG] — only for the cold-start note. + final int dynHistoryDays; final List> savedSessions; final String date; final int dayEndSec; @@ -3850,6 +3960,8 @@ class _DayBlocksInput { required this.coverageWindows, required this.liveStepsReal, required this.stepCalib, + required this.dynFloorG, + required this.dynHistoryDays, required this.savedSessions, required this.date, required this.dayEndSec, diff --git a/lib/data/db.dart b/lib/data/db.dart index 0e657739..e00b63c1 100644 --- a/lib/data/db.dart +++ b/lib/data/db.dart @@ -18,6 +18,7 @@ import 'package:path/path.dart' as p; import 'package:path_provider/path_provider.dart'; import 'package:sqflite/sqflite.dart'; import 'day_label.dart'; +import 'live_coverage_policy.dart'; import 'models.dart'; class LocalDb { @@ -736,6 +737,17 @@ class LocalDb { // Times are device epoch SECONDS (same clock as raw_records.rec_ts, since the // band's RTC is SET_CLOCK'd to phone time on connect). `day` = local date label // of the window start (for per-day step attribution). + // + // HISTORICAL ROWS. Databases written before the window derivation was fixed + // contain ZERO-WIDTH rows (`end_ts == start_ts`) — the old writer took both + // ends from a band record timestamp that does not advance during a live + // session. They are left as they are: their real durations were never + // recorded, and widening them after the fact would replace one wrong extent + // with another while silently changing already-derived days. Readers must + // tolerate them — `coverageWindowsOverlapping` matches them (`end_ts >= lo`) + // and the derivation's minute test (`s + 60 > start && s < end`) still + // excludes the minute containing the row, so such a row under-excludes but + // never crashes or double-adds its steps. static Future _createLiveCoverage(Database db) async { await db.execute(''' CREATE TABLE IF NOT EXISTS live_coverage ( @@ -752,17 +764,27 @@ class LocalDb { } /// Record a real 100 Hz step window (device-time seconds) + its step count. + /// + /// The window is normalised by [sanitizeCoverageWindow] first: a zero-width + /// window that claims steps is REPAIRED (widened to the duration those steps + /// physically imply) rather than dropped, because dropping it would lose a + /// real 100 Hz count; an inverted window is rejected. See that function for + /// the reasoning. This is a guard, not the derivation — the caller is + /// expected to have measured a real window (see + /// `deriveLiveCoverageWindow`); it exists so an upstream regression cannot + /// silently reintroduce degenerate rows. static Future addLiveCoverage( int startTs, int endTs, int steps, String day, ) async { - if (steps <= 0 || endTs < startTs) return; + final w = sanitizeCoverageWindow(startTs, endTs, steps); + if (w == null) return; final db = await instance; await db.insert('live_coverage', { - 'start_ts': startTs, - 'end_ts': endTs, + 'start_ts': w.startTs, + 'end_ts': w.endTs, 'steps': steps, 'day': day, }); diff --git a/lib/data/live_coverage_policy.dart b/lib/data/live_coverage_policy.dart new file mode 100644 index 00000000..6c90ab34 --- /dev/null +++ b/lib/data/live_coverage_policy.dart @@ -0,0 +1,164 @@ +// live_coverage_policy.dart — pure, I/O-free policy for the 100 Hz step +// COVERAGE WINDOW. Nothing here touches BLE, the DB, Flutter or the clock: +// callers hand in the facts they observed for one live-pedometer session and +// read back the window to persist. Every branch is unit-testable. +// +// WHY THE WINDOW EXISTS +// `live_coverage` rows say "over THIS wall period the live 100 Hz pedometer +// counted real steps". The derivation pass adds those real steps to the day and +// EXCLUDES the minutes the window covers from the 1 Hz estimate, so a minute is +// counted by 100 Hz or estimated by 1 Hz — never both. A window is therefore a +// measurement, not a label: get its extent wrong and either minutes get counted +// twice (window too short) or real minutes lose their estimate (too long). +// +// WHICH FAILURE WE PREFER +// Too short fabricates steps that never happened (the same minute counted by +// both paths). Too long drops an estimate for minutes we may not have covered — +// an UNDER-report of something we genuinely did not measure at 100 Hz. Under +// the honesty contract the under-report wins, so when the evidence is +// ambiguous these rules lean toward the WIDER window. + +import 'dart:math' as math; + +/// Sample rate of the live accel stream the pedometer runs on. +const double kLiveSampleRateHz = 100.0; + +/// Fraction of the session's wall-clock hull that must actually be sampled +/// before we treat the hull as the covered period. +/// +/// The streamed time is a UNION of intervals; a `live_coverage` row can only +/// store its convex HULL. At/above this duty cycle the union dominates its own +/// hull (dropouts are the exception), so the hull is the better model of the +/// counting period and, per the preference above, the safer one. Below it the +/// session was mostly NOT streaming — a link that woke for a few seconds an +/// hour — and claiming the hull would silently delete hours of 1 Hz estimate +/// for a handful of real steps. There we fall back to the only quantity we +/// actually measured: the sampled duration. +const double kCoverageDutyFloor = 0.5; + +/// Cadence ceiling used to derive a LOWER BOUND on elapsed time from a step +/// count. Elite sprint cadence tops out near 220 spm; 240 leaves headroom so +/// the bound is never tighter than physiology allows. This is a bound, never an +/// estimate: N steps cannot have happened in less than N / (240/60) seconds. +const double kMaxPlausibleCadenceSpm = 240.0; + +/// A persisted 100 Hz coverage window, in the SAME epoch-second base as +/// `decoded_onehz.rec_ts` (see [deriveLiveCoverageWindow]). +class LiveCoverageWindow { + const LiveCoverageWindow(this.startTs, this.endTs); + final int startTs; + final int endTs; + int get seconds => endTs - startTs; + + @override + String toString() => 'LiveCoverageWindow($startTs..$endTs, ${seconds}s)'; +} + +/// The shortest elapsed time [steps] could physically span, in whole seconds. +/// 0 for a non-positive count. See [kMaxPlausibleCadenceSpm]. +int minCoverageSecondsForSteps(int steps) => + steps <= 0 ? 0 : (steps * 60 / kMaxPlausibleCadenceSpm).ceil(); + +/// Decide the coverage window for one live-pedometer session, or null when +/// there is nothing defensible to record. +/// +/// TIME BASE. The window is compared against `decoded_onehz.rec_ts` by +/// `LocalDb.coverageWindowsOverlapping`, so it must stay in the BAND's record +/// time base. That is why the window is ANCHORED on [bandStartTs] — the record +/// timestamp carried by the first live frame — whenever the band supplied one, +/// and only falls back to the phone clock ([firstIngestMs]) when it never did. +/// The engine SET_CLOCKs the band to phone time on connect, so the two bases +/// agree to within the drift it already corrects, but we do not rely on that: +/// the ANCHOR comes from the band and only the DURATION comes from measurement, +/// and a duration is the same number in either base. +/// +/// DURATION. Three observations bound the covered period: +/// * [samples100Hz] / [kLiveSampleRateHz] — the time we can PROVE we sampled +/// (each 100 Hz sample is 10 ms of covered signal). A lower bound. +/// * the wall-clock hull [firstIngestMs] … [lastIngestMs] — the outer extent +/// of the counting period, dropouts included. An upper bound. +/// * [bandStartTs] … [bandEndTs] — the same hull in band time, used only when +/// the phone timestamps are missing, because in practice the band repeats +/// one record timestamp for a whole live session (span 0) and cannot be +/// trusted to carry the duration. +/// The duty cycle between the first two picks which one models the session — +/// see [kCoverageDutyFloor]. The result is then raised to +/// [minCoverageSecondsForSteps] if the claimed [steps] could not physically fit +/// in it, and is never zero. +LiveCoverageWindow? deriveLiveCoverageWindow({ + required int steps, + required int samples100Hz, + int? bandStartTs, + int bandEndTs = 0, + int? firstIngestMs, + int? lastIngestMs, +}) { + // No real steps → nothing to exclude from the 1 Hz estimate, nothing to store. + if (steps <= 0) return null; + + final int? anchor = (bandStartTs != null && bandStartTs > 0) + ? bandStartTs + : (firstIngestMs != null && firstIngestMs > 0 + ? firstIngestMs ~/ 1000 + : null); + // Steps with no timestamp at all from either clock: we cannot place the + // window on any timeline, and a misplaced window would exclude the WRONG + // minutes. Drop it rather than invent a position. + if (anchor == null) return null; + + final sampledS = samples100Hz <= 0 ? 0.0 : samples100Hz / kLiveSampleRateHz; + + double hullS = 0; + if (firstIngestMs != null && + lastIngestMs != null && + lastIngestMs > firstIngestMs) { + hullS = (lastIngestMs - firstIngestMs) / 1000.0; + } else if (bandStartTs != null && bandEndTs > bandStartTs) { + hullS = (bandEndTs - bandStartTs).toDouble(); + } + + double coveredS; + if (hullS <= 0) { + coveredS = sampledS; // no hull observed — the sampled time is all we have + } else if (sampledS <= 0) { + coveredS = hullS; // steps without sample accounting — hull is the evidence + } else { + final duty = sampledS / hullS; + coveredS = duty >= kCoverageDutyFloor ? hullS : sampledS; + // The sampled time can exceed the hull (duplicate/backlogged frames). The + // hull is wall truth, so it also acts as the ceiling. + if (coveredS > hullS) coveredS = hullS; + } + + // Physiological floor — a true lower bound, applied last so a broken duration + // measurement can never produce a window the steps could not fit into. + var secs = coveredS.ceil(); + secs = math.max(secs, minCoverageSecondsForSteps(steps)); + // A window claiming steps is never zero-width: that silently disables the + // no-double-count exclusion it exists to drive. + if (secs < 1) secs = 1; + return LiveCoverageWindow(anchor, anchor + secs); +} + +/// Persistence-boundary guard: normalise a window before it reaches the +/// `live_coverage` table, or null when it must not be stored at all. +/// +/// This is defence in depth behind [deriveLiveCoverageWindow] — an upstream +/// regression that stops advancing its clock must not be able to write a +/// zero-width window again without being repaired here. +/// +/// REPAIR, NOT REJECT. Dropping the row would throw away a REAL 100 Hz step +/// count (the one measurement on the day that is not an estimate) and hand +/// those minutes back to the 1 Hz estimator. Widening the window to the +/// physically implied minimum ([minCoverageSecondsForSteps]) keeps the steps, +/// restores a non-degenerate exclusion, and claims only what a bound supports. +/// The one case that IS rejected is an inverted window (`endTs < startTs`): +/// that is incoherent rather than merely imprecise, and there is no defensible +/// way to decide which end was meant. +LiveCoverageWindow? sanitizeCoverageWindow(int startTs, int endTs, int steps) { + if (steps <= 0) return null; + if (endTs < startTs) return null; + final minS = math.max(1, minCoverageSecondsForSteps(steps)); + if (endTs - startTs < minS) return LiveCoverageWindow(startTs, startTs + minS); + return LiveCoverageWindow(startTs, endTs); +} diff --git a/lib/state/app_state.dart b/lib/state/app_state.dart index b0494a59..68e57f98 100644 --- a/lib/state/app_state.dart +++ b/lib/state/app_state.dart @@ -40,6 +40,7 @@ import '../compute/hr_max.dart'; import '../compute/profile.dart'; import '../data/day_label.dart'; import '../data/db.dart'; +import '../data/live_coverage_policy.dart'; import '../data/local_repository.dart'; import '../gps/gps_source.dart'; import '../gps/route_tracker.dart'; @@ -802,6 +803,24 @@ class AppState extends ChangeNotifier { Future debugReconcileOrphanedLiveWorkout() => _reconcileOrphanedLiveWorkout(); + /// Feed one live accel frame through the live-pedometer path exactly as + /// [_onLiveFrame] does, with the ingest wall-clock supplied by the caller. + /// Tests only — lets a test replay a session's frames deterministically. + @visibleForTesting + void debugFeedLiveAccel( + List mags, { + int? recTs, + required int atMs, + }) { + _ingestLiveMagsAt(proto.ImuFrame(recTs ?? 0, 0, mags), atMs); + _trackCoverage(recTs); + } + + /// End the live-pedometer session (persist the coverage window, fold the bout + /// into the cadence calibration) without a BLE disconnect. Tests only. + @visibleForTesting + Future debugFinalizeLivePedometer() => _finalizeLivePedometer(); + /// Feed a strap alarm-lifecycle event (56 set / 57–58 fired / 59 cleared) /// without going through the BLE event path. Tests only. @visibleForTesting @@ -1728,14 +1747,36 @@ class AppState extends ChangeNotifier { int _lastLiveUiNotifyMs = 0; // DEVICE-time window (epoch sec) the live pedometer covered this session — so // the 1 Hz estimate can EXCLUDE these minutes (100 Hz real count wins). + // + // The band's record timestamp is the ANCHOR only: it keeps the window in the + // same base as `decoded_onehz.rec_ts` (what `coverageWindowsOverlapping` + // compares against). It is NOT the duration — in practice every live frame of + // a session repeats the same `recTs`, so start==end and the window covered + // nothing. The duration comes from what we actually ingested (100 Hz sample + // count + the phone-clock hull of the ingest times), combined by + // [deriveLiveCoverageWindow]. See that function for the base reconciliation. int? _liveCoverStartTs; int _liveCoverEndTs = 0; + int? _liveFirstIngestMs; // phone clock at the first ingested live frame + int? _liveLastIngestMs; // …and at the last one void _trackCoverage(int? recTs) { if (recTs == null || recTs <= 0) return; _liveCoverStartTs ??= recTs; if (recTs > _liveCoverEndTs) _liveCoverEndTs = recTs; } + /// The window this session covered, or null when there is nothing defensible + /// to record. Pure decision lives in [deriveLiveCoverageWindow]; this only + /// feeds it the observations. + LiveCoverageWindow? _liveCoverageWindow(int steps) => deriveLiveCoverageWindow( + steps: steps, + samples100Hz: _liveSamples, + bandStartTs: _liveCoverStartTs, + bandEndTs: _liveCoverEndTs, + firstIngestMs: _liveFirstIngestMs, + lastIngestMs: _liveLastIngestMs, + ); + /// Steps counted on the live 100 Hz stream this connected session (real, /// gain-applied). Used for cadence calibration. 0 when not streaming. int get _liveRaw => @@ -1754,7 +1795,12 @@ class AppState extends ChangeNotifier { return raw > 0 ? (raw * ana.StepParams.gain).round() : 0; } - void _ingestLiveMags(proto.ImuFrame f) { + void _ingestLiveMags(proto.ImuFrame f) => + _ingestLiveMagsAt(f, DateTime.now().millisecondsSinceEpoch); + + // `nowMs` is passed in (rather than read here) so the coverage bookkeeping + // this method feeds is drivable from a test without a fake clock. + void _ingestLiveMagsAt(proto.ImuFrame f, int nowMs) { final mags = f.mags; if (mags.isEmpty) return; // Append this frame's |a|(g) samples (gravity INCLUDED — AN-2554's dynamic @@ -1769,7 +1815,13 @@ class AppState extends ChangeNotifier { final e = (magSum / mags.length) - 1.0; _liveEnmoSum += e > 0 ? e : 0.0; _liveEnmoN++; - final nowMs = DateTime.now().millisecondsSinceEpoch; + // Phone-clock extent of the ingested stream — the only observation that + // reports how long this session actually ran (the band's record timestamp + // typically repeats). Used as a DURATION only; see [_liveCoverageWindow]. + _liveFirstIngestMs ??= nowMs; + if (_liveLastIngestMs == null || nowMs > _liveLastIngestMs!) { + _liveLastIngestMs = nowMs; + } // Stamp last real motion (for the inactivity nudge). 0.02 g over baseline is // clearly dynamic movement, not resting jitter. if (e > 0.02) { @@ -1825,6 +1877,8 @@ class AppState extends ChangeNotifier { _imuStreamSeen = false; _liveCoverStartTs = null; _liveCoverEndTs = 0; + _liveFirstIngestMs = null; + _liveLastIngestMs = null; } /// End-of-session: if the bout is credible walking, fold it into the personal @@ -1833,20 +1887,19 @@ class AppState extends ChangeNotifier { final steps = liveSteps; // gain-applied final durS = _liveSamples / 100.0; final enmo = _liveEnmoN > 0 ? _liveEnmoSum / _liveEnmoN : 0.0; - // Capture the device-time coverage window BEFORE resetting. - final coverStart = _liveCoverStartTs; - final coverEnd = _liveCoverEndTs; + // Derive the coverage window BEFORE resetting (it reads session counters). + final window = _liveCoverageWindow(steps); _resetLivePedometer(); // Record the REAL 100 Hz step window (device time). The derivation pass adds // it to the day's steps AND excludes those minutes from the 1 Hz estimate, so // 100 Hz always wins and a minute is never counted twice. - if (steps > 0 && coverStart != null && coverEnd >= coverStart) { - final d = DateTime.fromMillisecondsSinceEpoch(coverStart * 1000); + if (window != null) { + final d = DateTime.fromMillisecondsSinceEpoch(window.startTs * 1000); final day = '${d.year.toString().padLeft(4, '0')}-' '${d.month.toString().padLeft(2, '0')}-' '${d.day.toString().padLeft(2, '0')}'; - unawaited(LocalDb.addLiveCoverage(coverStart, coverEnd, steps, day)); + await LocalDb.addLiveCoverage(window.startTs, window.endTs, steps, day); } if (steps <= 0 || durS < 20) return; final cadence = steps / (durS / 60.0); diff --git a/test/live_coverage_window_test.dart b/test/live_coverage_window_test.dart new file mode 100644 index 00000000..977b0604 --- /dev/null +++ b/test/live_coverage_window_test.dart @@ -0,0 +1,343 @@ +// live_coverage — regression coverage for the ZERO-WIDTH window bug. +// +// `live_coverage` rows record the period the live 100 Hz pedometer actually +// counted, so the derivation pass can exclude those minutes from the 1 Hz +// estimate (real count wins, nothing counted twice). The old writer took BOTH +// ends of that window from the band record timestamp carried on live frames — +// a value that does not advance during a live session — so real databases are +// full of rows claiming hundreds of steps over ZERO seconds. A zero-width +// window excludes ~one minute instead of the streamed period (so the rest gets +// double counted) and destroys the only alignment between real 100 Hz counts +// and 1 Hz minutes. +// +// Three layers are covered: +// 1. deriveLiveCoverageWindow — the pure policy that decides the window. +// 2. AppState — a full session whose recTs never advances must still persist +// a window spanning the streamed period (this is the bug, end to end). +// 3. LocalDb.addLiveCoverage — the persistence guard, so an upstream +// regression cannot silently write a degenerate row again. + +import 'dart:math' as math; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:shared_preferences/shared_preferences.dart'; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; + +import 'package:openstrap_edge/data/db.dart'; +import 'package:openstrap_edge/data/live_coverage_policy.dart'; +import 'package:openstrap_edge/state/app_state.dart'; + +/// One 100 Hz frame of walking-shaped |a|(g): a ~2 Hz gait oscillation riding +/// the 1 g gravity baseline, which is what the AN-2554 counter expects. +List _walkFrame(int frameIndex, int samples) => [ + for (var i = 0; i < samples; i++) + 1.0 + + 0.45 * + math.sin( + 2 * math.pi * 2.0 * ((frameIndex * samples + i) / 100.0), + ), + ]; + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + LocalDb.dbName = 'openstrap_live_coverage_test.db'; + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + tearDownAll(() async { + await LocalDb.close(); + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + setUp(() => SharedPreferences.setMockInitialValues({})); + + // ── 1. the pure policy ───────────────────────────────────────────────────── + group('deriveLiveCoverageWindow', () { + const t0 = 1785000000; // band record timestamp (device epoch sec) + + test('a band recTs that never advances still yields the streamed period', + () { + // THE BUG: every live frame of the session carried the same recTs, so the + // old writer stored start == end == t0. + final w = deriveLiveCoverageWindow( + steps: 1657, + samples100Hz: 100 * 1800, // 30 min of 100 Hz samples + bandStartTs: t0, + bandEndTs: t0, // never advanced + firstIngestMs: 1785000000000, + lastIngestMs: 1785000000000 + 1800 * 1000, + ); + expect(w, isNotNull); + expect(w!.seconds, 1800); + expect(w.startTs, t0); + }); + + test('the window is anchored in the BAND record-time base, not the phone ' + 'clock', () { + // Phone and band clocks deliberately disagree here: the window must be + // placed on the band's timeline (what decoded_onehz.rec_ts uses) while + // taking its DURATION from the phone-clock hull. + final w = deriveLiveCoverageWindow( + steps: 200, + samples100Hz: 100 * 300, + bandStartTs: t0, + bandEndTs: t0, + firstIngestMs: 1000000000000, // a completely different epoch + lastIngestMs: 1000000000000 + 300 * 1000, + ); + expect(w!.startTs, t0); + expect(w.endTs, t0 + 300); + }); + + test('falls back to the phone clock only when the band never reported a ' + 'record timestamp', () { + final w = deriveLiveCoverageWindow( + steps: 200, + samples100Hz: 100 * 120, + bandStartTs: null, + bandEndTs: 0, + firstIngestMs: t0 * 1000, + lastIngestMs: (t0 + 120) * 1000, + ); + expect(w!.startTs, t0); + expect(w.seconds, 120); + }); + + test('a near-continuous stream claims the full wall hull (small dropouts ' + 'stay inside the counted period)', () { + // 570 s sampled inside a 600 s hull — 95 % duty. + final w = deriveLiveCoverageWindow( + steps: 700, + samples100Hz: 100 * 570, + bandStartTs: t0, + bandEndTs: t0, + firstIngestMs: t0 * 1000, + lastIngestMs: (t0 + 600) * 1000, + ); + expect(w!.seconds, 600); + }); + + test('a mostly-absent stream claims only the sampled duration, never the ' + 'hull', () { + // 60 s of samples spread over an hour: claiming the hull would delete an + // hour of 1 Hz estimate for one minute of real counting. + final w = deriveLiveCoverageWindow( + steps: 46, + samples100Hz: 100 * 60, + bandStartTs: t0, + bandEndTs: t0, + firstIngestMs: t0 * 1000, + lastIngestMs: (t0 + 3600) * 1000, + ); + expect(w!.seconds, 60); + }); + + test('the sampled duration can never exceed the wall hull', () { + // Duplicate/backlogged frames inflate the sample count past real time. + final w = deriveLiveCoverageWindow( + steps: 100, + samples100Hz: 100 * 900, + bandStartTs: t0, + bandEndTs: t0, + firstIngestMs: t0 * 1000, + lastIngestMs: (t0 + 300) * 1000, + ); + expect(w!.seconds, 300); + }); + + test('never returns a zero-width window when it claims steps', () { + // No sample accounting and no phone timestamps at all: the only surviving + // evidence is the step count, whose physiological floor still bounds the + // window away from zero. + final w = deriveLiveCoverageWindow( + steps: 1657, + samples100Hz: 0, + bandStartTs: t0, + bandEndTs: t0, + ); + expect(w!.seconds, greaterThan(0)); + expect(w.seconds, minCoverageSecondsForSteps(1657)); + }); + + test('a window is widened to the time its steps could physically span', () { + // 400 steps cannot happen in 10 s at any human cadence. + final w = deriveLiveCoverageWindow( + steps: 400, + samples100Hz: 100 * 10, + bandStartTs: t0, + bandEndTs: t0, + firstIngestMs: t0 * 1000, + lastIngestMs: (t0 + 10) * 1000, + ); + expect(w!.seconds, minCoverageSecondsForSteps(400)); + expect(w.seconds, greaterThan(10)); + }); + + test('no steps → no window, and no timeline → no window', () { + expect( + deriveLiveCoverageWindow( + steps: 0, + samples100Hz: 100 * 600, + bandStartTs: t0, + bandEndTs: t0, + firstIngestMs: t0 * 1000, + lastIngestMs: (t0 + 600) * 1000, + ), + isNull, + ); + // Steps with nothing to place them on: a misplaced window would exclude + // the wrong minutes, so nothing is recorded. + expect( + deriveLiveCoverageWindow(steps: 500, samples100Hz: 100 * 600), + isNull, + ); + }); + }); + + // ── 2. the real AppState session (end-to-end regression) ─────────────────── + group('AppState live session', () { + test('a session whose band recTs NEVER advances still persists a window ' + 'covering the streamed period', () async { + final app = AppState.forTesting(); + addTearDown(app.dispose); + + // 4 minutes of 100 Hz walking, delivered 10 frames/s like the 0x33 IMU + // stream, every single frame carrying the SAME record timestamp (the + // behaviour observed on real hardware). + const recTs = 1785600000; // 2026-08-02T… device epoch sec + const frames = 4 * 60 * 10; + const samplesPerFrame = 10; + const startMs = recTs * 1000; + for (var i = 0; i < frames; i++) { + app.debugFeedLiveAccel( + _walkFrame(i, samplesPerFrame), + recTs: recTs, + atMs: startMs + i * 100, + ); + } + expect(app.liveSteps, greaterThan(0), reason: 'walk must count steps'); + + await app.debugFinalizeLivePedometer(); + + final db = await LocalDb.instance; + final rows = await db.query( + 'live_coverage', + where: 'start_ts >= ? AND start_ts < ?', + whereArgs: [recTs, recTs + 3600], + ); + expect(rows, hasLength(1)); + final start = (rows.first['start_ts'] as num).toInt(); + final end = (rows.first['end_ts'] as num).toInt(); + // Pre-fix this was start == end == recTs — a 0 s window. + expect(end - start, greaterThan(0)); + // The streamed period was ~240 s (the last frame's ingest is 100 ms shy). + expect(end - start, closeTo(240, 2)); + expect(start, recTs, reason: 'window stays in the band record-time base'); + expect((rows.first['steps'] as num).toInt(), greaterThan(0)); + }); + + test('a session that streamed nothing writes no window', () async { + final app = AppState.forTesting(); + addTearDown(app.dispose); + await app.debugFinalizeLivePedometer(); + final db = await LocalDb.instance; + final rows = await db.query( + 'live_coverage', + where: 'day = ?', + whereArgs: ['1970-01-01'], + ); + expect(rows, isEmpty); + }); + }); + + // ── 3. the persistence guard ─────────────────────────────────────────────── + group('LocalDb.addLiveCoverage', () { + test('a zero-duration window that claims steps is never persisted as-is — ' + 'and its steps are not lost', () async { + const start = 1786000000; + await LocalDb.addLiveCoverage(start, start, 1657, '2026-08-06'); + + final db = await LocalDb.instance; + final rows = await db.query( + 'live_coverage', + where: 'day = ?', + whereArgs: ['2026-08-06'], + ); + expect(rows, hasLength(1), reason: 'the real 100 Hz count must survive'); + final end = (rows.first['end_ts'] as num).toInt(); + // Pre-fix the row went in verbatim with end_ts == start_ts. + expect(end, greaterThan(start)); + expect(end - start, minCoverageSecondsForSteps(1657)); + expect((rows.first['steps'] as num).toInt(), 1657); + expect(await LocalDb.liveStepsForDay('2026-08-06'), 1657); + }); + + test('an impossibly short window is widened to what its steps imply', + () async { + const start = 1786100000; + await LocalDb.addLiveCoverage(start, start + 3, 600, '2026-08-07'); + final windows = await LocalDb.coverageWindowsOverlapping( + start, + start + 3600, + ); + expect(windows, hasLength(1)); + expect(windows.first[1] - windows.first[0], + minCoverageSecondsForSteps(600)); + }); + + test('an inverted window is rejected, and a zero-step window is not stored', + () async { + const start = 1786200000; + await LocalDb.addLiveCoverage(start, start - 60, 500, '2026-08-08'); + await LocalDb.addLiveCoverage(start, start + 600, 0, '2026-08-08'); + final db = await LocalDb.instance; + final rows = await db.query( + 'live_coverage', + where: 'day = ?', + whereArgs: ['2026-08-08'], + ); + expect(rows, isEmpty); + }); + + test('an honest window is stored exactly as measured', () async { + const start = 1786300000; + await LocalDb.addLiveCoverage(start, start + 1800, 2000, '2026-08-09'); + final windows = await LocalDb.coverageWindowsOverlapping( + start, + start + 3600, + ); + expect(windows.first, [start, start + 1800]); + }); + }); + + // ── 4. historical degenerate rows stay readable ──────────────────────────── + group('legacy zero-width rows already on disk', () { + test('are tolerated by the coverage readers (left alone, not migrated)', + () async { + // Written the way the old code did, bypassing the guard, to prove the + // readers still behave on the rows real users already have. + const start = 1786400000; + final db = await LocalDb.instance; + await db.insert('live_coverage', { + 'start_ts': start, + 'end_ts': start, // zero width + 'steps': 1230, + 'day': '2026-08-10', + }); + expect(await LocalDb.liveStepsForDay('2026-08-10'), 1230); + final windows = await LocalDb.coverageWindowsOverlapping( + start, + start + 3600, + ); + expect(windows, hasLength(1)); + expect(windows.first, [start, start]); + }); + }); +} diff --git a/test/step_personal_floor_test.dart b/test/step_personal_floor_test.dart new file mode 100644 index 00000000..ea4fd584 --- /dev/null +++ b/test/step_personal_floor_test.dart @@ -0,0 +1,124 @@ +// The 1 Hz activity estimator's PERSONAL floor — the edge half of the fix for +// the "39,384 steps" bug. +// +// The analytics package decides activity from a calibration-invariant dynamic +// amplitude, but it is pure: it cannot read history, so it cannot know what +// "moving" means for this wearer. Edge supplies that. Each day persists its own +// high quantile of the dynamic amplitude (`dyn_p90`), and the next day's derive +// takes the MEDIAN across trailing days as its floor. +// +// Two properties matter here and neither is obvious from the analytics tests: +// +// 1. COLD START ABSTAINS. Below the minimum history there is no floor, and the +// estimator must return absent rather than fall back to a constant. +// Falling back to a constant is precisely the bug: the old estimator's +// absolute 0.05 g floor was the same magnitude as the gravity-reference +// error, so a quiet day's sedentary minutes cleared it for hours. +// +// 2. A SINGLE ANOMALOUS DAY CANNOT MOVE THE FLOOR. That is why the anchor is +// a median across days rather than a same-day quantile — a same-day floor +// collapses on a quiet day and passes everything, the mirror image of the +// absolute-constant failure. +import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_analytics/onehz.dart' as ana; + +/// Per-minute rows carrying DELIBERATELY MISLEADING ENMO: every sedentary +/// minute sits at 0.055 g, just above the absolute 0.05 g floor the old +/// estimator used. If an ENMO-based decision path is ever reintroduced these +/// tests fail loudly instead of shipping tens of thousands of phantom steps. +List rows(List dyn) => [ + for (var i = 0; i < dyn.length; i++) + ana.MotionMinute(i * 60000.0, 60, 0.055, 0.02, 1.055, dyn[i]), + ]; + +void main() { + group('personal ambulatory floor — cold start', () { + test('no trailing history → no floor → the estimator ABSTAINS', () { + // Exactly what a fresh install has on day one. + final floor = ana.personalDynFloorFromDailySummaries(const []); + expect(floor, isNull); + + final est = ana.dailyStepEstimate( + rows(List.filled(600, 0.60)), // plenty of real movement + personalDynFloorG: floor, + ); + expect(est.present, isFalse, + reason: 'without a personal baseline the honest answer is "unknown", ' + 'not a number computed against a guessed threshold'); + }); + + test('a day just under the minimum history still abstains', () { + final tooFew = List.filled(ana.personalDynFloorMinDays - 1, 0.44); + expect(ana.personalDynFloorFromDailySummaries(tooFew), isNull); + }); + + test('once enough days exist the floor appears and the estimate follows', + () { + final enough = List.filled(ana.personalDynFloorMinDays, 0.44); + final floor = ana.personalDynFloorFromDailySummaries(enough); + expect(floor, isNotNull); + + final est = ana.dailyStepEstimate( + rows(List.filled(600, 0.60)), + personalDynFloorG: floor, + ); + expect(est.present, isTrue); + expect(est.value!.activeMinutes, greaterThan(0)); + }); + }); + + group('personal ambulatory floor — stability', () { + test('a sedentary day produces no active minutes against a real floor', () { + // The 39,384 shape: a full day of sitting still. Every minute carries an + // ENMO above the OLD absolute floor, so this is the exact input that used + // to inflate — it must now yield nothing. + final floor = + ana.personalDynFloorFromDailySummaries(List.filled(7, 0.44))!; + final est = ana.dailyStepEstimate( + rows(List.filled(900, 0.02)), // sedentary dynamic amplitude + personalDynFloorG: floor, + ); + expect(est.present, isTrue); + expect(est.value!.activeMinutes, 0); + expect(est.value!.steps, 0); + }); + + test('one anomalous day cannot drag the floor (median across days)', () { + final normal = [0.44, 0.43, 0.45, 0.44, 0.46, 0.43, 0.45]; + final clean = ana.personalDynFloorFromDailySummaries(normal)!; + final polluted = + ana.personalDynFloorFromDailySummaries([...normal, 9.0, 8.0])!; + expect((clean - polluted).abs(), lessThan(0.02)); + }); + + test('the floor a quiet day contributes does not collapse the threshold', + () { + // A same-day threshold would collapse here and pass everything; the + // multi-day median holds the line. + final withQuietDay = + [0.44, 0.43, 0.45, 0.001, 0.44, 0.46, 0.45]; + final floor = ana.personalDynFloorFromDailySummaries(withQuietDay)!; + expect(floor, greaterThan(0.4)); + + final est = ana.dailyStepEstimate( + rows(List.filled(900, 0.02)), + personalDynFloorG: floor, + ); + expect(est.value!.activeMinutes, 0); + }); + }); + + group('what edge persists each day', () { + test('dailyDynSummary produces the per-day value the floor pools', () { + final day = rows([for (var i = 0; i < 600; i++) i / 1000.0]); + final summary = ana.dailyDynSummary(day); + expect(summary, isNotNull); + expect(summary!, greaterThan(0)); + }); + + test('a day too thin to summarise stores nothing, not a zero', () { + // Storing 0 would poison the median for every later day. + expect(ana.dailyDynSummary(rows(List.filled(10, 0.4))), isNull); + }); + }); +} From 9f53dd9113d036329f7a807d85df50e2336d356f Mon Sep 17 00:00:00 2001 From: abdulsaheel Date: Sun, 26 Jul 2026 19:24:22 +0530 Subject: [PATCH 4/5] build: pin the sibling PR commits so CI actually compiles this change MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Edge CI was failing to COMPILE, and the failure is the whole reason the v48 note existed: error • The function 'personalDynFloorFromDailySummaries' isn't defined error • The named parameter 'personalDynFloorG' isn't defined error • The getter 'activeMinutes' isn't defined for DailyStepEstimate lib/compute/ calls analytics API that only exists on the sibling PR, but pubspec.yaml still pointed at the pre-fix SHA. It compiled locally only because the gitignored pubspec_overrides.yaml redirects to ../analytics — so the green local run was green-by-omission, and CI was the only thing telling the truth. This is the same drift class as v43, caught in its loud form: a missing symbol rather than a silently different number. The v43 version of this shipped a readiness fix that the pinned SHA never contained and stayed broken for three releases. Both pins now point at the sibling PR heads — still immutable commit SHAs, never floating refs: protocol 5675b2f (OpenStrap/protocol#19) analytics 8b1aa4e (OpenStrap/analytics#30) Verified, rather than assumed, that those SHAs contain what the changelog claims: `git show 8b1aa4e:lib/src/onehz/motion/steps.dart` carries the new step API, rr_correction.dart has the signed-dRR `seg.add(x[k])`, and `git show 5675b2f:lib/src/live.dart` has kKnownRecordVersions. Also corrected the kAlgoVersion changelog. It still said the sibling fixes were NOT pinned, which stopped being true with this commit — a changelog that misdescribes what is in the build is the exact failure being guarded against, in either direction. pubspec.lock now records the real git provenance (resolved-ref) instead of the local path: entries a previous `pub get` wrote while the overrides file was present. Reproduced CI locally to confirm: with pubspec_overrides.yaml moved aside, `flutter pub get` resolves ref == resolved-ref == 8b1aa4e, `flutter analyze` is clean, and the suite is 914 passing. ON MERGE: move both pins to the resulting main commits and bump kAlgoVersion again — a PR-branch SHA can become unreachable if the branch is deleted after a squash merge. --- lib/compute/derivation_engine.dart | 26 +++++++++++++++----------- pubspec.yaml | 25 ++++++++++++++++++++----- 2 files changed, 35 insertions(+), 16 deletions(-) diff --git a/lib/compute/derivation_engine.dart b/lib/compute/derivation_engine.dart index 62cbfc39..573e2485 100644 --- a/lib/compute/derivation_engine.dart +++ b/lib/compute/derivation_engine.dart @@ -285,16 +285,13 @@ import 'substrate.dart'; // - Historical days resolve their timezone offset at their own timestamp // rather than through today's offset. // -// ⚠️ SIBLING-PACKAGE changes are NOT in this bump yet. The analytics fixes from -// the same sweep (sleep no longer reporting a no-data window as light sleep, +// SIBLING-PACKAGE changes ride along with this bump: the analytics sweep from +// the same review (sleep no longer reporting a no-data window as light sleep, // the Lipponen-Tarvainen threshold on the signed dRR series, abstention on // degenerate dispersion, the reconciled TRIMP stack, circular social jetlag) -// live in an analytics commit that `pubspec.yaml` does NOT yet point at — the -// pin is still the previous SHA. Do NOT extend this changelog to claim them -// until that pin is updated to a merged analytics commit and verified with -// `git show :`. Citing an analytics change that the pinned SHA never -// contained is exactly how the v43 changelog documented a readiness fix that -// stayed broken for three releases; when the pin moves, bump again. +// and the protocol decode fixes. This was NOT true when v48 was first written — +// pubspec.yaml still pointed at the pre-fix SHAs then, and this note said so. +// The pins were moved as part of v49; see the pin-status note above it. // v49: steps/activity rebuilt on a calibration-invariant feature. // // Diagnosed on a real user database: the day reported 39,384 steps against a @@ -326,9 +323,16 @@ import 'substrate.dart'; // when the 1 Hz estimate is absent. // // This bump also re-derives days whose stored step figure came from the old -// estimator. NOTE: the sibling analytics fixes still are NOT pinned — see the -// v48 note below; pubspec.yaml must be repinned and this bumped again once the -// analytics PR merges. +// estimator. +// +// PIN STATUS: as of this bump pubspec.yaml points at the analytics and protocol +// PR-branch commits, so everything described in v48 AND v49 is genuinely in the +// build — the edge code physically cannot compile against the older analytics +// pin, which is how we know. Those pins must move to the merge commits (and +// this must bump again) when the sibling PRs land. Verify any analytics claim +// made here with `git show :` before trusting it; a changelog +// citing a change the pinned SHA never contained is how v43 documented a +// readiness fix that stayed broken for three releases. const int kAlgoVersion = 49; /// Raw is kept this many days past derivation, then pruned (derived stays). diff --git a/pubspec.yaml b/pubspec.yaml index 8e919c0c..a3c704ca 100644 --- a/pubspec.yaml +++ b/pubspec.yaml @@ -29,17 +29,32 @@ dependencies: # still builds against ../analytics and ../protocol via pubspec_overrides.yaml. # NOTE: this branch is intentionally WHOOP-4-only — protocol stays on `main`, # NOT the gen5/multiband branch. + # ⚠️ THESE TWO CURRENTLY POINT AT PR-BRANCH COMMITS, NOT `main`. + # This edge change does not compile against the previous pins — it calls + # analytics API that only exists on OpenStrap/analytics#30 — so pinning the + # PR head is what makes CI meaningful rather than green-by-omission. They are + # still immutable commit SHAs, never floating refs. + # BEFORE RELEASE / ON MERGE: move both to the resulting `main` commits and + # bump kAlgoVersion again. Leaving a PR-branch SHA pinned after merge risks + # the commit being unreachable if the branch is deleted after a squash. openstrap_protocol: git: url: https://github.com/OpenStrap/protocol.git - ref: 02fc8e5f2310ded690a522cd9884bf51b4cdc7e1 # tip of protocol main + # OpenStrap/protocol#19 (fix/issues): bounded R-R counts, NaN accel + # rejection, v7/v9/v12/v18 routed through parseR24 so a historical record + # keeps its OWN timestamp instead of collapsing to capture time, alarm + # form dispatch, bounded battery. + ref: 5675b2fc58ab5aa7455cb7b0f7837fe5391e8291 openstrap_analytics: git: url: https://github.com/OpenStrap/analytics.git - # Tip of analytics main: ANR trig fix (#27) + readinessComposite mean/SD-z - # fallback on degenerate MAD (#26 — the change the v43 changelog documented - # but the previous pin never actually contained; it ships at v46). - ref: e3705bf6fc591560e0d6f9d68343a3b861ef3765 + # OpenStrap/analytics#30 (fix/issues): the abstain-over-fabricate sweep, + # plus the step/activity rebuild on a calibration-invariant feature — + # dailyStepEstimate's new signature, personalDynFloorFromDailySummaries + # and dailyDynSummary, all of which lib/compute/ now calls. + # This is the SHA whose content the v49 changelog describes; verify with + # `git show :lib/src/onehz/motion/steps.dart` before trusting it. + ref: 8b1aa4e99490b8cc15397154bc548ec3fc32c44a # BLE — flutter_blue_plus is the maintained cross-platform GATT client. flutter_blue_plus: ^1.36.8 From b014c529479083f6b3292065e42023562861ae9a Mon Sep 17 00:00:00 2001 From: abdulsaheel Date: Sun, 26 Jul 2026 19:44:07 +0530 Subject: [PATCH 5/5] build: repin the siblings to their merge commits; kAlgoVersion 50 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit OpenStrap/protocol#19 and OpenStrap/analytics#30 are merged, so pubspec.yaml moves off the PR-branch heads v49 pointed at and onto the resulting main commits: protocol 5675b2f -> a98cd70 analytics 8b1aa4e -> f5ccae6 A PR-branch SHA stops being the canonical location of a change once it merges, and can be orphaned outright if the branch is later deleted. kAlgoVersion 49 -> 50 even though no edge source line changed alongside it. The constant identifies the code that PRODUCED a day_result, and that code includes the pinned siblings — a device holding v49 rows built against the PR-branch SHAs must re-derive rather than serve them as equivalent. Treating "same content, different commit" as beneath a bump is the assumption that lets a stale bundle outlive a dependency change. Verified at the merge commits themselves rather than inferred from the PRs being green — the check v43 skipped: analytics f5ccae6 steps.dart has the new step API (9 symbol hits) rr_correction.dart:233 is `seg.add(x[k])` (signed dRR) advanced_stager.dart has maxAccelCarryForwardSec protocol a98cd70 live.dart has kKnownRecordVersions Reproduced CI locally with pubspec_overrides.yaml moved aside: resolved-ref matches both pins exactly, flutter analyze clean, 914 passing. --- lib/compute/derivation_engine.dart | 18 +++++++++++++- pubspec.lock | 16 ++++++++----- pubspec.yaml | 38 ++++++++++++++---------------- 3 files changed, 45 insertions(+), 27 deletions(-) diff --git a/lib/compute/derivation_engine.dart b/lib/compute/derivation_engine.dart index 573e2485..f6acb464 100644 --- a/lib/compute/derivation_engine.dart +++ b/lib/compute/derivation_engine.dart @@ -333,7 +333,23 @@ import 'substrate.dart'; // made here with `git show :` before trusting it; a changelog // citing a change the pinned SHA never contained is how v43 documented a // readiness fix that stayed broken for three releases. -const int kAlgoVersion = 49; + +// v50: the sibling PRs merged; pubspec.yaml now pins the resulting `main` +// commits (analytics f5ccae6, protocol a98cd70) instead of the PR-branch heads +// v49 briefly pointed at. +// +// A version bump is required even though no edge SOURCE line changed with it. +// kAlgoVersion identifies the code that PRODUCED a day_result, and that code +// includes the pinned siblings: a device holding v49 rows built against the +// PR-branch SHAs must re-derive against the merged ones rather than serve them +// as equivalent. Treating "same content, different commit" as not worth a bump +// is the assumption that lets a stale bundle survive a dependency change. +// +// Verified at the merge commits themselves, not inferred from the PRs being +// green: steps.dart carries the new step API, rr_correction.dart has the +// signed-dRR `seg.add(x[k])`, advanced_stager.dart has maxAccelCarryForwardSec, +// live.dart has kKnownRecordVersions. +const int kAlgoVersion = 50; /// Raw is kept this many days past derivation, then pruned (derived stays). const int rawRetentionDays = 3; diff --git a/pubspec.lock b/pubspec.lock index f1caf073..0931c086 100644 --- a/pubspec.lock +++ b/pubspec.lock @@ -971,16 +971,20 @@ packages: openstrap_analytics: dependency: "direct main" description: - path: "../analytics" - relative: true - source: path + path: "." + ref: f5ccae61cbc6a8083425ae0121b665deadcb421d + resolved-ref: f5ccae61cbc6a8083425ae0121b665deadcb421d + url: "https://github.com/OpenStrap/analytics.git" + source: git version: "1.0.0" openstrap_protocol: dependency: "direct main" description: - path: "../protocol" - relative: true - source: path + path: "." + ref: a98cd7061346681db17a81844c300c46da118c02 + resolved-ref: a98cd7061346681db17a81844c300c46da118c02 + url: "https://github.com/OpenStrap/protocol.git" + source: git version: "1.0.0" ota_update: dependency: "direct main" diff --git a/pubspec.yaml b/pubspec.yaml index a3c704ca..75b24a55 100644 --- a/pubspec.yaml +++ b/pubspec.yaml @@ -29,32 +29,30 @@ dependencies: # still builds against ../analytics and ../protocol via pubspec_overrides.yaml. # NOTE: this branch is intentionally WHOOP-4-only — protocol stays on `main`, # NOT the gen5/multiband branch. - # ⚠️ THESE TWO CURRENTLY POINT AT PR-BRANCH COMMITS, NOT `main`. - # This edge change does not compile against the previous pins — it calls - # analytics API that only exists on OpenStrap/analytics#30 — so pinning the - # PR head is what makes CI meaningful rather than green-by-omission. They are - # still immutable commit SHAs, never floating refs. - # BEFORE RELEASE / ON MERGE: move both to the resulting `main` commits and - # bump kAlgoVersion again. Leaving a PR-branch SHA pinned after merge risks - # the commit being unreachable if the branch is deleted after a squash. + # Both are now MERGE COMMITS ON `main`, not PR-branch heads — the PR-branch + # SHAs these briefly pointed at are no longer the canonical location of the + # change, and a branch deletion could orphan them. openstrap_protocol: git: url: https://github.com/OpenStrap/protocol.git - # OpenStrap/protocol#19 (fix/issues): bounded R-R counts, NaN accel - # rejection, v7/v9/v12/v18 routed through parseR24 so a historical record - # keeps its OWN timestamp instead of collapsing to capture time, alarm - # form dispatch, bounded battery. - ref: 5675b2fc58ab5aa7455cb7b0f7837fe5391e8291 + # Tip of protocol main — OpenStrap/protocol#19 merged: bounded R-R counts, + # NaN accel rejection, v7/v9/v12/v18 routed through parseR24 so a + # historical record keeps its OWN timestamp instead of collapsing to + # capture time, alarm form dispatch, bounded battery. + # Verified present: `git show :lib/src/live.dart | grep kKnownRecordVersions`. + ref: a98cd7061346681db17a81844c300c46da118c02 openstrap_analytics: git: url: https://github.com/OpenStrap/analytics.git - # OpenStrap/analytics#30 (fix/issues): the abstain-over-fabricate sweep, - # plus the step/activity rebuild on a calibration-invariant feature — - # dailyStepEstimate's new signature, personalDynFloorFromDailySummaries - # and dailyDynSummary, all of which lib/compute/ now calls. - # This is the SHA whose content the v49 changelog describes; verify with - # `git show :lib/src/onehz/motion/steps.dart` before trusting it. - ref: 8b1aa4e99490b8cc15397154bc548ec3fc32c44a + # Tip of analytics main — OpenStrap/analytics#30 merged: the + # abstain-over-fabricate sweep, plus the step/activity rebuild on a + # calibration-invariant feature (dailyStepEstimate's new signature, + # personalDynFloorFromDailySummaries, dailyDynSummary — all called by + # lib/compute/). + # Verified present at THIS sha, not assumed from the PR being merged: + # steps.dart carries the new API, rr_correction.dart has the signed-dRR + # `seg.add(x[k])`, advanced_stager.dart has maxAccelCarryForwardSec. + ref: f5ccae61cbc6a8083425ae0121b665deadcb421d # BLE — flutter_blue_plus is the maintained cross-platform GATT client. flutter_blue_plus: ^1.36.8