Version 2.0 adds a complete investigation center: date/person/camera search, event images and Frigate clips, auditable review, person profiles, camera health, HA automation tools and explicit privacy retention.
Recommended Frigate connection: use https://FRIGATE_IP:8971 with a dedicated
Frigate viewer account. FaceID logs in server-side and renews the session; credentials
and media are never exposed to the browser. Keep TLS verification enabled with a
trusted certificate. Port 5000 remains compatible, but it is Frigate's unauthenticated
internal API and should not be exposed on a LAN or the internet.
FaceID is a small, self-hosted service that adds reliable, trainable face recognition
on top of Frigate. It uses the same model family as Immich and
CompreFace (InsightFace buffalo_l: SCRFD detection + ArcFace embeddings) and was
built because Frigate's built-in face recognition UX didn't cut it:
The 1.0 pipeline evaluates diverse, quality-ranked faces from the finished Frigate clip, keeps a durable decision audit, measures false accepts/rejects from operator labels, and emits conservative cross-camera scenarios. Optional local AI adds searchable security context, but neither AI nor clothing appearance is allowed to declare identity.
- No train-tab treadmill. Matching is nearest-neighbor over face embeddings — every image you assign is a visible reference point, with no training cycles and no queue that refills with already-known faces. To be clear: this is not immune to bad data — an imbalanced or mislabeled gallery still degrades matching (a person with many reference images wins borderline matches more often). The difference is that the failure mode is an image you can see and delete, not an opaque model update.
- Strangers are first-class. Unknown faces are collected, auto-clustered (DBSCAN, the same trick photo apps use) and reviewed in a web UI: one click assigns a whole cluster to a person — or ignores it. People you enroll (the mailman you want notifications for) get recognized; people you ignore go permanently silent.
- An ignore list that actually sticks. Ignored faces stay as negative anchors: never notified, never matched to your family, never resurfacing in review — and FaceID learns their new looks over time (only on unambiguous matches with a clear margin over every enrolled person, visibly marked "auto", deletable anytime). Anchors are grouped per person; groups can be merged, curated, or released into a real person with one click if you change your mind.
- Train from anywhere, without bloat. Assign faces from your cameras, upload photos from your photo library, or enroll whole folders via CLI. A per-person photo cap keeps galleries lean: when it's exceeded, the reference most similar to the rest is set aside (so unusual angles are kept, not lost) — visibly, on the person card, where you can restore it. The cap is adjustable in the Settings tab.
- Home Assistant native. MQTT discovery creates sensors per camera and a device for every enrolled person: last location, last-seen time, recent presence, appearances today, average confidence and total appearances. Rich attributes include 7/30-day counts, most frequent camera and a camera breakdown.
- Optional Frigate write-back. Confirmed names can be written as
sub_label, so you can filter clips by person in Frigate's Explore view — including retroactively: the history scan tags past events, and assigning a face in the review UI tags its original event too. - Yours to keep. A Settings tab holds the matching thresholds (live-editable) plus one-click backup & restore of your gallery, and an optional built-in daily auto-backup — your hand-curated face data is the one irreplaceable thing, so it's easy to safeguard.
All faces below are AI-generated (StyleGAN) — no real persons.
Unknown review — new faces arrive auto-clustered; assign a whole cluster with one click, or scan your camera history to bootstrap the gallery:
Persons — your gallery; upload photos or send faces back to review:
Frigate --MQTT frigate/events--> FaceID
^ | several distinct person snapshots
| v
+--optional sub_label------ InsightFace -> top-2 gallery matches
|
score + runner-up margin + repeated agreement -> recognized
below unknown threshold -> unknown
otherwise -> ambiguous review
MQTT -> Home Assistant: sensor.faceid_<camera> + faceid/event (JSON)
The Activity tab shows each final decision, evidence, scenario and optional context. Each row includes the captured face and an enlarged preview, so labels are based on visual evidence. Label complete events as a known person or Unknown person. The Calibration tab then measures TAR, FAR and FRR by camera and person and recommends a threshold/margin for the configured false-accept target. It intentionally keeps all frames from one event together; random frame splitting would produce misleadingly optimistic accuracy.
Automation consumers should use faceid/v1/events; the legacy faceid/event topic
remains for compatibility. The v1 payload contains a schema version, final decision,
evidence, scenario and any appearance hint. Home Assistant device triggers are
discovered automatically, and standalone installs can configure asynchronous webhooks.
Optional AI context uses an Ollama-compatible /api/generate and /api/embed service
for factual scene descriptions, tags and searches such as “red backpack”. It is never
used for recognition, Frigate sub_label, or ground truth.
After FaceID reconnects to MQTT, each enrolled person appears as a separate device under
Settings → Devices & services → MQTT. For a person whose slug is alice, discovery
creates:
| Entity | Example | Useful for |
|---|---|---|
sensor.faceid_alice_last_location |
Front Door |
Where the person was last seen |
sensor.faceid_alice_last_seen |
timestamp | “Seen in the last 10 minutes” automations |
binary_sensor.faceid_alice_presence |
on/off | Short presence window after recognition |
sensor.faceid_alice_today |
4 |
Daily dashboard counters |
sensor.faceid_alice_average_confidence |
73.2 % |
Monitoring recognition quality |
sensor.faceid_alice_appearances |
128 |
Total finalized recognition events |
The last-location sensor attributes also contain appearances_7_days,
appearances_30_days, most_seen_camera, camera_breakdown, last_score and active
cameras. Entity IDs may receive a numeric suffix if an older entity with the same ID
already exists; use the entity shown in Home Assistant rather than assuming the exact ID.
FaceID performs all recognition locally on your hardware — no cloud APIs, no accounts, no telemetry. The only thing ever fetched from the internet is the open-source recognition model itself, once, on first start:
- What: InsightFace
buffalo_lmodel pack (SCRFD face detection + ArcFace recognition, the same open models Immich and CompreFace use) - From where: the official InsightFace GitHub release
- Size: ~300 MB, cached on disk afterwards (survives restarts and app updates)
After that download, core recognition works completely offline. Your camera images and face data never leave your machine. If AI context is enabled, frames are sent only to the explicitly configured Ollama-compatible URL; use a local address to keep all data on your network.
- Frigate 0.16+ (snapshot + sub_label APIs), reachable over HTTP
- An MQTT broker (the one Frigate already uses is fine)
- A CPU with AVX (any Intel/AMD from the last decade; ~1.5 GB RAM; no GPU needed).
Running HAOS/your host in a VM? The default virtual CPU model (e.g. Proxmox
kvm64) hides AVX — set the VM CPU type tohostand cold-restart the VM, or the recognition runtime will refuse to start.
(Apps were formerly known as apps.)
-
Add this repository to your app store — one click:
(or manually: Settings → Apps → App Store → ⋮ → Repositories → add
https://github.com/r11a/faceid) -
Install the FaceID app, set your Frigate URL in the options (MQTT is picked up automatically from the Mosquitto app) and start it.
-
Open the FaceID panel in the sidebar. First start downloads the model (~300 MB).
The app is built locally on your machine (amd64/aarch64). See faceid-addon/DOCS.md for all options.
Tested on Debian 12/13 and Ubuntu 22.04+; any Linux with Python 3.10+ works.
1. System packages
apt install python3-venv python3-dev build-essential libglib2.0-0 libgl1 libxcb1 libgomp12. Get the code and install the Python environment
git clone https://github.com/r11a/faceid /opt/faceid
cd /opt/faceid
python3 -m venv venv
venv/bin/pip install -r requirements.txt3. Configure
cp docs/example-config.yaml config.yaml
nano config.yaml # set: Frigate URL, MQTT host + credentials, your camera names4. Run as a service
cp faceid.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable --now faceid5. Verify
The first start downloads the model pack (~300 MB, one time — see above). Follow the
log with journalctl -u faceid -f until you see MQTT verbunden (Success), then open
http://<host>:8600 and check that the header shows your person/queue counters.
- Scan your camera history (optional but recommended): in the Unknown tab, click
"Scan camera history" — faces from past Frigate events land in the review queue,
pre-clustered per person, and already-known people are tagged in Frigate retroactively.
(CLI alternative:
venv/bin/python -m app.backfill --days 14) - Assign clusters in the UI (Unknown tab): pick a name per cluster — select individual tiles first if a cluster contains a stray face. The ⛶ button shows the full snapshot for context.
- Once a few people exist, use "apply suggestions" to bulk-assign everything the gallery already recognizes with ≥ 50 % similarity. Repeat as the gallery grows.
- Optionally upload 5–10 clear photos per person (Persons tab) as clean anchors, or
enroll a folder:
venv/bin/python -m app.enroll "Alice" /path/to/photos.
Not everyone deserves a notification. FaceID distinguishes three actions on an unknown face, and the difference matters:
| Action | Meaning |
|---|---|
| Assign | This is someone I track — recognize, notify, tag in Frigate |
| Ignore | I know who this is and never want to hear about them — silent forever |
| Discard | Garbage crop (blurry, not a face) — delete, no memory kept |
Ignored faces become negative anchors, grouped by person in the Ignored tab:
- Reappearances are silently dropped (a log line is all you get), and genuinely new
looks are auto-learned into the right group — guarded so a household member can
never silently become an anchor (requires high similarity and a clear margin over
every enrolled person; auto anchors are marked and deletable). Opt out with
ignore_learning: false. - Curate groups: merge two groups when they're the same person, move selected anchors between groups, or send a face back to the review queue.
- Change your mind: release a whole group into an existing or brand-new person — the anchors become that person's reference gallery and tracking starts immediately.
- You can also ignore an entire enrolled person via "ignore person" on their card.
Training tips: camera snapshots beat photo-library images (same lens, angle and light as at recognition time). Diversity beats volume. Create dedicated persons for regular strangers (mailman, neighbors) instead of discarding them — that keeps them from being force-matched to your family.
Frigate runs detection on a downscaled stream (often 1280x960 or even 640x360) but records in full camera resolution (e.g. 2560x1920). Snapshots come from the detect stream, so faces arrive smaller and softer than they need to be.
With Settings → Sharper reference photos enabled (default), a face heading for the review queue is re-fetched from the recording instead: measured across a dozen real events, faces came out roughly twice as large (e.g. 84px → 178px). Better references mean better recognition — and, as a bonus, similarity scores spread out, making genuine duplicates easy to tell from "same person, other angle".
Details: FaceID downloads the event clip once and scans frames across it, because Frigate picks its snapshot from a moment that can't be queried afterwards. Every candidate face is compared against the snapshot face and the best identity match wins — so with several people in frame, the wrong face can't be enrolled. When no frame yields a usable face (roughly one event in three), the original snapshot is kept.
Live recognition still uses the fast snapshot path; only enrollment takes the slower, sharper route (a few seconds and one clip download per event). Needs recordings enabled for the camera.
Sometimes the detect snapshot holds no usable face at all and the event is skipped entirely. The history scan can go back over those via the clip:
python -m app.backfill --days 28 --rescueAbout one in five such events yields a face this way. Because there is no snapshot face
to check identity against, the only guard is detection quality — and it has to be strict:
in a measured run, finds below ~0.8 were overwhelmingly back-of-head shots, motion blur
and false positives (a church spire scored 0.57), while finds above it were real faces.
The default is --rescue-min-det 0.85; lowering it multiplies the queue faster than it
adds usable references.
Expect a clip download (8–32 MB) and several seconds per affected event, so this is a manual run, not something the live pipeline does.
Keep --days within your recording retention. Rescue needs the clip, so events older
than record.retain yield nothing and only cost time — a 28-day run against a 10-day
retention spends most of its hour on events it cannot help.
FaceID listens on frigate/events. Events created through Frigate's API are not
tracked objects, so they never appear there — FaceID simply never learns about them.
This matters if you use a camera's own person detection as a reliability bridge: an automation sees the camera report a person and creates a Frigate event for it. Those events carry a snapshot and usually a clip, but stay invisible to FaceID.
Set poll_interval (seconds, e.g. 30) to also poll Frigate's event API:
faceid:
poll_interval: 30On one installation that added ~22 events a day at the front door, of which 12 in 20 held a usable face — roughly doubling what FaceID could learn from. Polled events run through the same pipeline; they carry no bounding box, so their snapshot is the full frame rather than a person crop. Off by default, since it costs one API request per interval.
Setting up such a bridge, checking beforehand whether it pays off, and the pitfalls of box-less snapshots: docs/camera-bridge.md.
The defaults are deliberately cautious, and on a real gallery that caution turned out to
be expensive. Measure yours rather than guessing — scripts/measure-recognition.py and
scripts/coverage.py produce the numbers.
On a 128-photo household gallery the separation was far wider than the defaults assume:
| correct person | different person | |
|---|---|---|
| best match score | median 0.50 | median 0.18, max 0.31 |
With a default threshold of 0.50 sitting exactly on the median of correct matches, half of all genuine recognitions were discarded to keep a distance nothing ever came close to. Two changes followed, both measured:
match_top_k3 → 1. Averaging the best k photos punishes people whose references cover many angles — their own less similar photos drag the score down, so the best-covered people scored worst.match_threshold0.50 → 0.45, still 0.14 above the highest score any stranger ever reached.
Together these lifted recognition on a held-out set of real events from 90% to 100%, and moved the weakest favourite's worst match from 0.01 above the cut-off to 0.09 above it — without a single misassignment.
Do not copy these numbers. Run the scripts on your own gallery: how far strangers get is the number that decides how low you can safely go.
Two scripts answer the questions that otherwise invite guesswork:
python scripts/coverage.py # what is each person missing?
python scripts/measure-recognition.py --baseline /tmp/old --days 3coverage.py reports, per person: photo count, diversity, viewing angles from the
landmarks, which cameras she was enrolled from, greyscale/IR shots, and a leave-one-out
self test — then names the concrete gap.
measure-recognition.py compares the current gallery against an older one (unpack a
backup from data/backups) and adds a practical probe against recent Frigate events,
including how much headroom each recognition has above the threshold. Events whose face
is already in the gallery are excluded — they score ~1.0 and measure nothing.
Recognition is only as good as the reference photos, so FaceID keeps galleries diverse rather than large:
-
A successful recognition never adds a photo. Recognising you at the door tags the Frigate event and updates the sensor — but the gallery stays untouched. This is deliberate: a gallery that grows from its own matches reinforces whatever it already believes, and a single wrong match would quietly breed more of the same. References only come from what you assign in the review queue, or upload yourself.
-
New photos are only kept if they add something — a near-duplicate of one you already have is skipped.
-
A per-person cap (default 40, adjustable in Settings) bounds how many references a person keeps. When exceeded, FaceID sets aside the photo that is most similar to all the others — i.e. the most redundant one — so a rare side/angle shot is preserved while a 30th near-identical front shot is the first to go.
-
Remove duplicates on demand: even under the cap, near-identical photos add nothing. Settings → Remove duplicates finds both truly identical images (perceptual hash) and near-identical faces (embedding similarity) and sets them aside (live preview), keeping the gallery diverse — as a button, so you stay in control. (Camera crops score lower than phone photos, so useful sensitivity is ~0.60–0.70.)
-
Nothing vanishes silently: trimmed photos appear on the person card with a short reason and a one-click restore (or delete), and the set-aside pile is itself capped (
trimmed_keep) so it never grows without bound.
If someone is recognized poorly from a certain angle, just add a photo from that angle — being unusual, it's automatically kept.
Full details: docs/trimming.md explains the why, the exact selection rule (with numbers), and how to restore or curate set-aside photos.
The LOG tab shows the last 500 log lines straight in the UI — including the quiet cases that decide whether a setup works: whether Frigate answers at startup, which cameras were announced to Home Assistant, and for every event whether a face was found at all. If nothing is ever recognised, that tab usually says why within a few lines. There is a warnings-only filter and a copy button for pasting into an issue.
Your gallery (enrolled persons + ignore anchors) is the one thing you can't regenerate — so FaceID makes it easy to keep. Everything is on the Settings tab:
- Download backup — a
.tar.gzofpersons/+ignored/(not the unknown queue). - Restore — replace everything, or merge in only what's missing (handy for moving people between instances).
- Automatic daily backup — enable it, pick an hour and how many to keep. It runs
inside FaceID (no external cron needed); the folder defaults to
data/backups, which survives app updates. Point it at a mounted share to get backups off the box.
Automate it yourself if you prefer: the download is a plain endpoint, so any host cron or Home Assistant automation can pull it:
# nightly host cron — keep 14 days
curl -fsS http://<faceid-host>:8600/api/backup -o /backups/faceid-$(date +\%F).tar.gz
find /backups -name 'faceid-*.tar.gz' -mtime +14 -deleteSettings changed here (thresholds + backup) are stored in data/settings.json and
override config.yaml / app options, persisting across restarts and updates.
Sensors appear automatically via MQTT discovery (sensor.faceid_<camera> for every
camera in discovery_cameras). The state lists everyone recognized within
presence_window seconds (Alice, Bob), then falls back to nobody. Attributes carry
the person list and the last recognition (score, event id).
For automations, trigger on the faceid/event topic — one JSON message per
(Frigate event, person): see docs/ha-automation-example.yaml
for a phone-notification automation with the Frigate snapshot attached.
- Home Assistant app: you get notified automatically when a new version is available (Settings → Apps → FaceID → Update). The changelog is shown right in the update dialog.
- Standalone:
cd /opt/faceid && git pull && systemctl restart faceid. Watch the GitHub releases to get notified.
See CHANGELOG.md for the full history.
- The web UI supports optional HTTP Basic Auth (
faceid.authin config.yaml) — strongly recommended for standalone installs; the HA app is protected by ingress and your Home Assistant login instead. Either way: it manages biometric data — keep it on a trusted LAN and don't expose port 8600 to the internet (Basic Auth without TLS is not internet-grade protection). - All face data stays in
data/on your host (JPEG crops + embeddings). Delete a person and their data is gone. - Depending on where you live, informing household members/visitors about face recognition on your cameras may be legally required. Be a good human.
See docs/example-config.yaml — every option is commented. The two knobs that matter most:
| Option | Meaning |
|---|---|
match_threshold (0.50) |
raise if strangers get matched to known persons, lower if known persons end up in the review queue |
unknown_threshold (0.35) |
below this the face is a clear unknown; the band above it is ambiguous |
match_margin (0.08) |
required lead over the second-best enrolled person |
min_confirmations (2) |
distinct agreeing frames required before publishing or ignoring |
cluster_eps (0.55) |
raise to merge unknown clusters more aggressively, lower if different people land in one cluster |
match_top_k (3) |
a person's score is the mean of their top-k reference similarities — dampens photo-count bias (1 = raw max) |
max_faces_per_person (40) |
soft cap; adding more drops the most redundant reference (0 = unlimited) |
ignore_threshold (= match_threshold) |
similarity at which a face counts as ignored |
ignore_margin (0.12) |
required lead of an ignore anchor over the best enrolled person |
ignore_learning (true) |
learn new looks of ignored people as additional anchors (guarded) |
hires_enroll (true) |
fetch new review-queue faces from the recording (sharper references) |
poll_interval (0) |
seconds; >0 also polls Frigate's event API for events MQTT never announces |
set_sub_label (false) |
opt in to writing confirmed names back to Frigate |
audit_retention_days (90) |
retention for the SQLite decision/evidence history |
MIT