From 0382faef7a79f93e9ac9487efa8dd95461773792 Mon Sep 17 00:00:00 2001 From: fidok15 <175498067+fidok15@users.noreply.github.com> Date: Wed, 11 Nov 2026 14:40:11 +0100 Subject: [PATCH] docs: replace the stale design spec with an accurate focus-scoring page The October spec described an early UI (setup step, cat menu icon, power button, MenuBarExtra, file names that no longer exist) and said nothing goes over the network, which Spotify covers now. Its scoring section was still right, so that part lives on as docs/focus-scoring.md, matched against FocusScorer's constants. --- README.md | 2 +- docs/focus-scoring.md | 53 ++++++ .../specs/2026-10-13-focusguard-design.md | 157 ------------------ 3 files changed, 54 insertions(+), 158 deletions(-) create mode 100644 docs/focus-scoring.md delete mode 100644 docs/superpowers/specs/2026-10-13-focusguard-design.md diff --git a/README.md b/README.md index 5056ef5..c8595f7 100644 --- a/README.md +++ b/README.md @@ -114,7 +114,7 @@ The score is the share of focused frames in a sliding window. A nudge fires only The sensitivity slider moves the threshold. If you study from paper on your desk, looking down may count as looking away. In that case, lower the sensitivity. -The approach follows [Driver-State-Detection](https://github.com/e-candeloro/Driver-State-Detection) by Ettore Candeloro (MIT). [The design spec](docs/superpowers/specs/2026-10-13-focusguard-design.md) has the exact numbers. +The approach follows [Driver-State-Detection](https://github.com/e-candeloro/Driver-State-Detection) by Ettore Candeloro (MIT). [How FocusGuard scores focus](docs/focus-scoring.md) has the exact numbers. ## For developers diff --git a/docs/focus-scoring.md b/docs/focus-scoring.md new file mode 100644 index 0000000..7ef5a30 --- /dev/null +++ b/docs/focus-scoring.md @@ -0,0 +1,53 @@ +# How FocusGuard scores focus + +The camera and Vision code only produce numbers about your face. Everything below runs in `Sources/FocusCore/FocusScorer.swift`, has no camera or UI code, and is covered by `swift run focus-check`. The thresholds live in one place in that file (`ScorerParams` and `Knobs`) and are meant as calibration knobs. + +## Signals + +Each camera frame (about 10 per second) goes through Apple's Vision framework: `VNDetectFaceLandmarksRequest` for the eye contours and pupils, plus the face's yaw, pitch and roll. That gives the eye aspect ratio (EAR), the pupil offset inside the eye, and the head pose. + +## Why not an end-to-end "focus %" model + +End-to-end engagement models trained on DAiSEE reach about 60–73 % accuracy even in papers and generalize poorly. The established approach is pretrained face and landmark models plus transparent scoring, which is what [Driver-State-Detection](https://github.com/e-candeloro/Driver-State-Detection) (MIT) does and what FocusGuard follows. + +## Calibration +The first 10 s of each study phase record the user's baseline: neutral `yaw`/`pitch` (camera may sit low or to the side) and baseline EAR. All later checks use deltas from the baseline. + +## Per-frame state + +Each frame gets exactly one of these states: + +- `absent`: no face found. +- `eyesClosed`: EAR < 0.7 × baseline EAR, sustained for > 0.5 s. Ordinary blinks never count. +- `lookingAway`: |Δyaw| > 25°, or |Δpitch| > 20°, or pupil offset within the eye > 0.35. +- `focused`: otherwise. +- Frames with low Vision confidence are skipped (neither focused nor distracted). + +## Score +- `focusScore` = percentage of `focused` frames within the sliding window W. +- `smoothedScore` = EMA of `focusScore` for display. +- A 1 s distraction inside a 30 s window lowers the score to ~97%, not 5%. + +## Alert rule + +A nudge fires only when all four conditions hold: + +1. `focusScore` < threshold T continuously for H seconds. +2. The user is still distracted right now: under 50% focused frames in the last 2 s. A windowed score lags, so without this a distraction that already ended could still trigger a bubble. +3. At least C has passed since the last bubble. A continuous distraction therefore gets a reminder every C, never more often. +4. At least 60 s have passed since the study phase started (grace period). + +Absence: no face for > 20 s shows one "Where did you go?" bubble; the next one only after the user returns and leaves again. Absent frames count as unfocused in the score. + +## Sensitivity slider +One slider (0…1) linearly interpolating between presets; default = Standard (0.5). + +| Preset | W | T | H | C | +|---|---|---|---|---| +| Gentle (0.0) | 60 s | 40% | 20 s | 5 min | +| Standard (0.5) | 30 s | 50% | 10 s | 3 min | +| Sensitive (1.0) | 15 s | 60% | 5 s | 90 s | + +## Privacy + +Camera frames are processed in memory and dropped. Nothing from the camera is written to disk or sent over the network. diff --git a/docs/superpowers/specs/2026-10-13-focusguard-design.md b/docs/superpowers/specs/2026-10-13-focusguard-design.md deleted file mode 100644 index c17dd2b..0000000 --- a/docs/superpowers/specs/2026-10-13-focusguard-design.md +++ /dev/null @@ -1,157 +0,0 @@ -# FocusGuard — Design Spec - -Date: 2026-10-13 -Status: approved; updated 2026-10-29 to match the implementation -License: MIT - -## 1. Goal - -A macOS menu bar app that runs Pomodoro-style study sessions and uses the webcam plus on-device ML to estimate the user's focus level. When focus drops in a *sustained* way, a manga-style speech bubble nudges the user. Short dips (e.g. one second at 5% followed by a return to 80%) must never trigger an alert. - -Audience: open source on GitHub. Anyone can download a release `.app` or build from source. - -Success criteria: -- Starting a session takes two clicks (timer, then Start). -- The camera is on only during study phases. -- A 1-second distraction never produces an alert; a sustained one produces exactly one alert per cooldown period. -- Camera frames never leave the Mac and are never written to disk. -- The UI reads as a hand-drawn anime/manga scene, not a standard settings panel. - -## 2. Platform and stack - -- macOS 14+, Swift, SwiftUI `MenuBarExtra` with `.menuBarExtraStyle(.window)`. -- `LSUIElement = YES` (no Dock icon). `NSCameraUsageDescription` set. -- ML: Apple Vision framework (built-in pretrained models): `VNDetectFaceLandmarksRequest` (eye contours, pupils) and face observation `yaw` / `pitch` / `roll`. -- Focus scoring logic ported from [e-candeloro/Driver-State-Detection](https://github.com/e-candeloro/Driver-State-Detection) (MIT): eye aspect ratio, PERCLOS-style windowed fraction, gaze and head-pose thresholds with time gating. Credit in README. -- No third-party dependencies. -- Localization: English + Polish via `Localizable.xcstrings`, following the system language. - -### Model research summary (why not an end-to-end "focus %" model) - -Hugging Face and GitHub were searched. End-to-end engagement models (DAiSEE-trained, e.g. `sujals1238/DAISEE`, `Shadhujan/daisee_engagement_model`) are hobby checkpoints with ~60–73% accuracy even in papers and generalize poorly; the most popular DAiSEE repo never released weights. The established approach is pretrained face/landmark models plus transparent scoring, which is what Driver-State-Detection does. `litert-community/L2CS-Gaze360-LiteRT` (MIT gaze net) is a possible later upgrade if Vision's pupil-based gaze proves too coarse. Out of scope for v1. - -## 3. Components - -| File | Responsibility | -|---|---| -| `FocusGuardApp.swift` | App entry, `MenuBarExtra`, menu bar cat icon (3 states). | -| `SessionTimer.swift` | Phases `idle / calibrating / study / break / done`; study length, break length, repetitions; pause/resume. | -| `CameraService.swift` | `AVCaptureSession`, ~10 fps, front camera; started on `study`, stopped on `break`/`idle`/`done`. Frames stay in memory. | -| `FaceAnalyzer.swift` | Runs Vision on a frame, returns `FaceSignals { faceFound, ear, gazeOffset, yaw, pitch, confidence }`. | -| `FocusScorer.swift` | Pure logic, no camera or UI: `FaceSignals` + timestamp in, `focusScore` (0–100), `smoothedScore`, and alert events out. Fully unit-tested. | -| `BubbleWindow.swift` | Borderless `NSPanel` under the menu bar icon showing the manga speech bubble. | -| `Scene/` | SwiftUI views composing the illustrated desk scene (section 5). | -| `SettingsStore.swift` | `@AppStorage`: sensitivity slider value, camera preview on/off, last session durations. | - -Data flow: timer click → setup → `calibrating` (10 s) → `study` → camera → `FaceAnalyzer` → `FocusScorer` → scene + optional bubble → `break` (camera off) → repeat → `done` (summary). - -## 4. Focus scoring and anti-spam - -### Calibration -The first 10 s of each study phase record the user's baseline: neutral `yaw`/`pitch` (camera may sit low or to the side) and baseline EAR. All later checks use deltas from the baseline. - -### Per-frame state (~10 fps), exactly one of: -- `absent`: no face found. -- `eyesClosed`: EAR < 0.7 × baseline EAR, sustained for > 0.5 s. Ordinary blinks never count. -- `lookingAway`: |Δyaw| > 25°, or |Δpitch| > 20°, or pupil offset within the eye > 0.35. -- `focused`: otherwise. -- Frames with low Vision confidence are skipped (neither focused nor distracted). - -Angle/EAR/gaze thresholds live as constants in one place, marked as calibration knobs. - -### Score -- `focusScore` = percentage of `focused` frames within the sliding window W. -- `smoothedScore` = EMA of `focusScore` for display. -- A 1 s distraction inside a 30 s window lowers the score to ~97%, not 5%. - -### Alert rule — all must hold -1. `focusScore` < threshold T continuously for H seconds. -2. The user is still distracted right now: under 50% focused frames in the last 2 s. A windowed score lags, so without this a distraction that already ended could still trigger a bubble. -3. At least C has passed since the last bubble. A continuous distraction therefore gets a reminder every C, never more often. -4. At least 60 s have passed since the study phase started (grace period). - -(The originally planned T + 10 hysteresis was dropped: the cooldown already prevents flapping around the threshold.) - -Absence: no face for > 20 s shows one "Where did you go?" bubble; the next one only after the user returns and leaves again. Absent frames count as unfocused in the score. - -### Sensitivity slider -One slider (0…1) linearly interpolating between presets; default = Standard (0.5). - -| Preset | W | T | H | C | -|---|---|---|---|---| -| Gentle (0.0) | 60 s | 40% | 20 s | 5 min | -| Standard (0.5) | 30 s | 50% | 10 s | 3 min | -| Sensitive (1.0) | 15 s | 60% | 5 s | 90 s | - -## 5. UI: illustrated first-person desk scene - -Style: anime/manga line art (hand-drawn outlines, soft cel shading). Palette is loosely inspired by the reference image (warm wood, cream paper, green cutting mat, sky blue) but not fixed; the drawing style is what must be kept. - -Window ~540×300, landscape. First-person view, no hands: a desk with a timer, open notebook, coffee mug, pens, a little mess (not too messy). Above the desk a window looking outside: trees, sky, rooftops. A cat sleeps on the windowsill. - -Scene objects are the controls: - -| Object | Behavior | -|---|---| -| Timer | Click in `idle` opens setup (study 5–90 min, break 1–30 min, repetitions 1–8, Start). During study shows a red wedge for remaining time plus digits. | -| Notebook | Handwritten-style text: current focus % and a pencil-style sparkline. After the session shows the summary (average focus, Swift Charts line, alert count, "New session"). | -| Cat | Sleeping = idle or high focus; head raised = focus dropping; the alert bubble appears "from" the cat. | -| Window / sky | Time of day from the system clock (see below). During breaks the window opens. | -| Coffee mug | Animated steam while a session runs. Click = pause/resume. | -| Sticky note | Opens settings: sensitivity slider, camera preview toggle. Language follows the system. | -| Polaroid on the wall | Optional small live camera preview (default hidden, togglable in settings). | - -### Layers (as built) -v1 ships four complete scene illustrations (`art/scene_morning|day|sunset|night.jpeg`), one per time of day, with the cat in a different spot in each. The cat is cut out of the art with a feathered mask and animated in place (breathing, squish and hearts when petted). The timer dial and notebook pages are blank in the art and filled by SwiftUI overlays. Window: 720×402 pt. A small power button (top-right, also ⌘Q) quits the app. - -### Layers (original plan, superseded) -Pre-rendered PNG layers (transparent backgrounds), composed bottom to top: -1. `sky_morning`, `sky_day`, `sky_sunset`, `sky_night`: what is seen through the window (trees, rooftops, sky). -2. `room`: wall, window frame, windowsill; window glass area transparent so the sky layer shows through. `room_open` variant with open window for breaks. -3. `cat_sleep`, `cat_alert`, `cat_awake`. -4. `desk`: desk surface with mug, pens, papers, notebook and timer bodies. -5. SwiftUI overlays: timer wedge/hands/digits, notebook text and chart, mug steam, drifting clouds, night room tint and warm lamp glow, polaroid preview. - -Time of day: 05–09 morning, 09–17 day, 17–20 sunset, 20–05 night; cross-fade between sky layers; room tint follows. - -Until the user supplies the generated art, the app runs on simple placeholder shapes in the same layout. Art prompts for Gemini are written as part of the implementation plan; generated art is committed to the repo and README notes it is AI-generated. - -### Speech bubble -Borderless `NSPanel` anchored under the menu bar icon: comic speech bubble with tail and thick outline, small cat face, randomized messages (e.g. "Hey! Your focus is slipping~ (。•́︿•̀。)"). Disappears after 6 s or on click. - -### Menu bar icon -Template image of a cat in 3 states: sleeping (idle), awake (study), alert. - -## 6. Errors and permissions - -- Camera permission denied: in-scene note "FocusGuard needs the camera" with a button opening System Settings › Privacy › Camera. Timer still works without scoring. -- Camera busy or disconnected: status "Camera unavailable", retry every 5 s, no bubbles. -- Low light / low confidence: frames skipped, not counted as distraction. -- Sleep: `ProcessInfo.beginActivity` during study so the timer is not suspended. - -## 7. Privacy - -Frames are processed in memory and discarded. Nothing is written to disk or sent over the network. README has a dedicated Privacy section. - -## 8. Testing - -- `swift run focus-check`: a plain executable self-check (no XCTest/swift-testing, so it runs with the Command Line Tools alone) feeding synthetic signal sequences to `FocusScorer`: - - 1 s dip → 0 alerts. - - Standard preset: 20 s distraction → 0 alerts, 30 s distraction → 1 alert (~25 s in). - - 10 min distraction → 4 alerts (cooldown 180 s). - - no alerts during the 60 s grace period. - - blinks (< 0.5 s closures) → no distraction; long eye closure → alert. - - absence > 20 s → exactly one absence bubble. - - sensitivity slider interpolation, `SessionClock` transitions, sleep catch-up, pause. -- `swift run FocusGuard --snapshots ` (debug builds) renders the scene in key states to PNGs for visual checks. -- Camera/Vision verified manually. - -## 9. Release - -- GitHub Actions: build `.app`, zip, attach to GitHub Release. -- Not notarized (requires paid Apple Developer account); README explains right-click › Open. Notarization added if an account becomes available. -- README: description, screenshot, privacy, build instructions, credits (Driver-State-Detection, MIT). - -## 10. Out of scope (v1) - -Multi-day history, L2CS gaze model, sync, Mac App Store, user-editable thresholds beyond the sensitivity slider.