Skip to content

feat(apple): hourly step import — per-hour iPhone steps with 90-day backfill (re-cut of #369) - #1429

Merged
ryanbr merged 1 commit into
mainfrom
feat/apple-hourly-steps-v38
Aug 18, 2026
Merged

feat(apple): hourly step import — per-hour iPhone steps with 90-day backfill (re-cut of #369)#1429
ryanbr merged 1 commit into
mainfrom
feat/apple-hourly-steps-v38

Conversation

@ryanbr

@ryanbr ryanbr commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Supersedes #369 by @vishk23, re-cut against current main. That branch was 385 commits behind and both of its migration slots had been taken since — GRDB v32-apple-step-hour collided with v32-rr-src-channel (main is at v37), and Room 26 collided with MIGRATION_25_26 (main is at 30). Migrations can't be rebased around, so this renumbers to the current tail rather than mutating shipped migrations.

What

The daily Apple Health import flattens iPhone steps to one appleDaily.steps total, so an hour the phone spent dead or sitting on a desk is invisible — the day just reads low. This adds an hourly table beside it:

  • GRDB v38-apple-step-hour + Room MIGRATION_30_31 (@Database(version = 31), SCHEMA_VERSION = 31): additive CREATE TABLE only — appleStepHour(deviceId, ts, steps), PK (deviceId, ts) like every other per-sample table.
  • Store API upsertAppleStepHours / appleStepHours, mirroring OuraRawStore, with an idempotent ON CONFLICT(deviceId, ts) upsert.
  • HKStatisticsCollectionQuery at hour interval over the same sync window — same anchor (local midnight) and error handling as the existing daily collect() — plus a one-time 90-day backfill, flagged only after rows actually land (HealthKit returns EMPTY rather than an error when step access is denied, so gating on no-throw alone would burn the one-time widen on a deny-then-grant).
  • appleStepHour registered in deviceScopedTables so Forget-device wipes it.
  • Kotlin schema twin + migration test, and both byte-identical schema_oracle.json copies updated (roomVersion 31, v38 migration, table entry).

Fixes on top of the original

  • Corrected what a missing hour means. Only hours with a step sample produce a row, so an absent hour is "no steps recorded" — a dead phone and a still hour look identical. Step data carries no separate "was recording" signal, so the doc now states a gap is evidence, not proof, and a consumer must not label it "phone off". (The original framing implied it showed exactly which hours were recorded.)
  • Refreshed the stale renumbering commentary and the v27 store header left over from the earlier renumber.
  • Added a test pinning SCHEMA_VERSION to the migration target so the two can't drift apart — the Record the app build in the backup, and beside a computed day #1410 failure mode.

Scope / caveats, stated plainly

Plumbing only — no UI reads the table yet (same precedent as #346, #895). The one irreversible part is the migration: once shipped you carry the table. Android carries the schema but no importer writes it (HealthKit has no Android analogue) — exactly the arrangement the older appleDaily table already has, and required to keep .noopbak byte-identical.

Data cost is small: 24 rows/day, so the 90-day backfill is ~2,160 rows and ongoing growth is ~9k rows/year.

Validation

  • Full Android suite: 4,037 tests / 489 classes, 0 failures, including SchemaOracleTest (compares Room's KSP-generated schema against the oracle, so the hand-written table entry is verified against real Room output) and the 5 migration tests.
  • Swift store + schema-oracle tests run in swift-packages CI (WhoopStore needs macOS).
  • HealthKitBridge is app-target Swift with no default CI → app-build dispatched for this branch.

Credit: @vishk23 (original implementation, on-device verification of the 90-day backfill and the hour-by-hour overlay).

…ackfill

Re-cut of #369 (by @vishk23) against current main. That branch was 385 commits
behind and BOTH of its migration slots had been taken since: GRDB
v32-apple-step-hour collided with v32-rr-src-channel (main is at v37) and Room 26
collided with MIGRATION_25_26 (main is at 30). Renumbered to the current tail
rather than mutating shipped migrations.

The daily Apple Health import flattens iPhone steps to one appleDaily.steps
total, so an hour the phone spent dead or on a desk is invisible - the day just
reads low. This adds an hourly table beside it:

- GRDB v38-apple-step-hour + Room MIGRATION_30_31 (@database version 31,
  SCHEMA_VERSION 31): additive CREATE TABLE only, appleStepHour(deviceId, ts,
  steps), PK (deviceId, ts) like every other per-sample table.
- Store API upsertAppleStepHours/appleStepHours mirroring OuraRawStore, with an
  idempotent ON CONFLICT upsert keyed on the natural key.
- HKStatisticsCollectionQuery at hour interval over the same sync window, same
  anchor and error handling as the existing daily collect(), plus a one-time
  90-day backfill flagged only after rows actually land (HealthKit returns EMPTY
  rather than an error when step access is denied, so gating on no-throw alone
  would burn the one-time widen on a deny-then-grant).
- appleStepHour registered in deviceScopedTables so Forget-device wipes it.
- Kotlin schema twin + AppleStepHourMigrationTest, and both byte-identical
  schema_oracle.json copies updated (roomVersion 31, v38 migration, table entry).

Fixes on top of the original:

- Corrected the documented claim about what a missing hour means. Only hours with
  a step sample produce a row, so an absent hour is "no steps recorded" - a dead
  phone AND a still hour both look identical. Step data carries no separate
  "was recording" signal, so the doc now says a gap is evidence, not proof, and a
  consumer must not label it "phone off".
- Refreshed the stale renumbering commentary and the "v27 store" header left over
  from the earlier renumber.
- Added a test pinning SCHEMA_VERSION to the migration target so the two cannot
  drift apart (the #1410 failure mode).

SCOPE: plumbing only, no UI reads the table yet - same precedent as #346/#895.
Android carries the schema but no importer writes it (HealthKit has no Android
analogue), exactly as the older appleDaily table already does, so .noopbak stays
byte-identical.
Re-review also found the Android Forget-device path incomplete: appleStepHour was
registered in the Swift deviceScopedTables (whose test enumerates every
deviceId-keyed table automatically) but NOT in the hand-maintained Android twin
in DeviceRegistryDao. Android never writes the table, but a .noopbak restored
FROM iOS carries its rows, so "delete this device's data" — and in particular
"Remove Apple Health imported data", which is exactly this data — left the hourly
step history behind. Added the delete + re-key queries, wired both call sites,
and extended the fan-out guard test to pin appleStepHour.

Validation: full Android suite 4037 tests / 489 classes green (the fan-out guard
caught the gap above before this landed), including SchemaOracleTest (compares
Room KSP-generated schema against the oracle) and the new migration test.
test (WhoopStore) green in CI — that covers the v38 migration, the store API, the
GRDB-side oracle and the automatic deviceScopedTables coverage assertion.
HealthKitBridge is app-target Swift, validated by app-build.

Co-authored-by: vishk23 <119831996+vishk23@users.noreply.github.com>
@ryanbr
ryanbr force-pushed the feat/apple-hourly-steps-v38 branch from 14fd961 to eca7443 Compare August 18, 2026 09:22
@ryanbr
ryanbr merged commit f259768 into main Aug 18, 2026
14 checks passed
@ryanbr
ryanbr deleted the feat/apple-hourly-steps-v38 branch August 18, 2026 09:28
ryanbr added a commit that referenced this pull request Aug 18, 2026
Build numbers only. MARKETING_VERSION / versionName stay 10.1.1 — the staging
line is one above the latest release (v10.1.0) and holds there until 10.1.1
actually ships.

versionCode 341 -> 342 and CURRENT_PROJECT_VERSION 223 -> 224, both +1 past the
previous staging build so a sideloaded update installs in place rather than
being refused as a downgrade.

Picks up since the last testing build (f2271cd): backup build provenance
(#1422), the CTL/ATL/TSB training-load engine (#1423) and its Trends card
(#1425), sleep-vs-wake HR contrast (#1424), the v20 optical decoder + CRC gate
(#895), the Sleep UI split (#1426), the WHOOP4 clock-correlation fix (#1427),
the settings waist-row layout + copy fix (#1428), hourly Apple step import
(#1429), Low refresh (#1430), and Power saving moving into More with a 35%
ceiling (#1431).
gdorgian pushed a commit to gdorgian/noop that referenced this pull request Aug 21, 2026
Folds in 129 upstream commits since the 10.1.0 sync and moves the fork's release
identity to 10.5.0 / build 230, the version line this tree now carries.

What arrives: training load (CTL/ATL/form), the resting-HR VO2max estimate
(ryanbr#1391), Apple Health write-back plus hourly step import with a 90-day backfill
(ryanbr#1432/ryanbr#1429), honest Oura reads (ryanbr#1384/ryanbr#1450/ryanbr#1467/ryanbr#1360), Power saving as its
own screen (ryanbr#1431), Polar PMD support, and the u32 decoder fixes behind ryanbr#869.

Fork-side resolutions, all recorded in docs/fork/decisions.md:

- android/ dropped wholesale again (22 delete/modify conflicts), per FORK_GUIDE.
- Upstream's in-memory per-day analyze cache (ryanbr#1005) is not carried; the
  persisted dayScanFingerprint/analysisInputRevision skip (v38-v40) stays,
  because it survives relaunch and is invalidated by the write-side revision.
  Upstream's v38-apple-step-hour is therefore registered as v41-apple-step-hour
  (v38-v40 are taken and SchemaOracleTests requires vN to match order).
- Every re-bounded read keeps limit: Int.max; the fixes in those same hunks
  (same-source gravity for ryanbr#1360, the Oura vendor-rate split) are taken in full.
- Power saving's Settings card and search entry are deleted in favour of the new
  screen, reachable from the More tab and the sidebar.
- altstore-source.json and docs/IOS.md keep the fork's versions.

Verification: Strand (macOS) and NOOPiOS build; StrandTests 2183 pass; every
Packages suite passes (WhoopStore 463 after the migration renumber, Analytics
1614, Protocol 622, Oura 204, Polar 25, Import 249, Design 98, SemanticMemory 19,
NoopLocalAccess 9); Tools 52 + linux-capture 212 pass; doc_comment_lint,
check-sideload-manifest (needed the healthkit background-delivery entitlement
declared) and i18n_audit --ci all exit 0 — the last of these is green here while
main is red on a pre-existing CoachSettingsView literal, now catalogued.

Not verified: BLE behaviour, which needs a strap.
@ryanbr ryanbr mentioned this pull request Aug 21, 2026
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant