From 12a64cfed9ba35ff6c49e8707904386d4dfd0cac Mon Sep 17 00:00:00 2001 From: shinyobjectz Date: Fri, 13 Nov 2026 07:43:37 -0800 Subject: [PATCH 1/2] Fingerprint gets its own section of the docs (docs/manual/fingerprint): what it is and the Fingerprint page, training one (renders, notes, marks, a box on the part of a sound you mean, the post-mortem), checking a cut (score, --was, scripts and stills, things, profiles), and projects and teams, with the judged-by chip's Keep with this project and Keep with the workspace (FP14e) written up for the first time. The guide it replaces (guides/fingerprint.md) is split into those pages; the manual's index lists them. raea pins raea-skills main f40ca31 (#56): the skill gains fingerprint.md, how an agent uses the person's fingerprint, routed from SKILL.md. SKILLS_STRICT=1 tools/skills.sh check passes --- docs/manual/fingerprint/check.md | 67 ++++++++++++ docs/manual/fingerprint/index.md | 47 +++++++++ docs/manual/fingerprint/projects.md | 67 ++++++++++++ docs/manual/fingerprint/train.md | 73 +++++++++++++ docs/manual/guides/fingerprint.md | 157 ---------------------------- docs/manual/index.md | 7 ++ skills | 2 +- 7 files changed, 262 insertions(+), 158 deletions(-) create mode 100644 docs/manual/fingerprint/check.md create mode 100644 docs/manual/fingerprint/index.md create mode 100644 docs/manual/fingerprint/projects.md create mode 100644 docs/manual/fingerprint/train.md delete mode 100644 docs/manual/guides/fingerprint.md diff --git a/docs/manual/fingerprint/check.md b/docs/manual/fingerprint/check.md new file mode 100644 index 00000000..5dfa97c7 --- /dev/null +++ b/docs/manual/fingerprint/check.md @@ -0,0 +1,67 @@ +--- +title: "Check a cut with it" +summary: "Score a cut with your fingerprint before it's posted, check that a fix removed a flag instead of hiding it, judge scripts and stills, find things floating in a 3D shot, and compare a cut with work you admire." +order: 2 +sources: ["docs/fingerprint.feature", "apps/fingerprint/src", "systems/fingerprint-use/src", "systems/fingerprint-things/src"] +--- + +## Score a cut + +```sh +raea fingerprint score out/cut.mp4 # the moments it expects a note on, first +raea fingerprint score out/cut.mp4 --sheet cut.png # the same on a sheet of frames, with where boxed +raea fingerprint score out/cut.mp4 --timeline # the piece second by second, in words +raea fingerprint which out/cut.mp4 # which fingerprint judged it, and why +``` + +`score` names which fingerprint judged the piece, then lists the moments it expects a note on, each in your words. On +a picture axis that has learned where, a moment also says where in the frame and, on a render with a layer pass, the +layers there ("top right on Floor"). `--json` gives every axis for every window, for a script. + +An axis that **knows** can hold a post back until you've looked. One that is **learning** only advises. Your agents +run `score` before they post, fix each moment that holds the cut back, or tell you why they didn't. + +## Check a fix + +```sh +raea fingerprint score out/v2.mp4 --was out/v1.mp4 +``` + +For each moment that held v1 back, this says whether v2 fixed it, still has it, or looks **disguised**: nearer the +axis's disguised defects than clean work, so the fix hid the defect instead of removing its cause. Only a fixed flag +counts as fixed. + +## Scripts, briefs and stills + +A fingerprint reads more than video. A script, a brief or another text file gets EmbeddingGemma 2 rows, one per +paragraph, and a still gets the picture columns. Each axis judges only what the file has, so a script is never called +blurry. raea's own axes judge pictures, sound and motion. To judge writing, add an axis that reads text to your +fingerprint.ts and teach it like any other. + +## Things in a shot + +```sh +raea render shot.tsx --layers -o shot.mp4 # the video and its layers (shot.layers) +raea render shot.tsx --aov -o shot.aov # its depth, beside it +raea fingerprint things shot.mp4 --json > things.json +raea check shot.mp4 --things things.json +``` + +`things` finds a piece's shots, the things in each (a render's layers, and anything marked on its notes) with their +depth, and the camera's path. It says where a thing floats, where two pass through each other, where the camera lurches +in the middle of a move, and where nothing is on screen. `raea check --things` turns those into findings beside the +rest. Floating needs real depth: with the render's `--aov` pass beside it (`shot.aov` for `shot.mp4`) it uses the +render's own depth, and without one it estimates depth from the picture, which can't always tell a hanging box from +one on the floor. + +## Profiles of work you admire + +```sh +raea fingerprint profile golden/*.mp4 --as reels --against others.txt +raea fingerprint score cut.mp4 --profile reels +``` + +A profile takes the ranges your references span: loudness, bass against treble, cuts a minute, motion, brightness, +contrast, saturation and sharpness. A cut scored against it hears which measures fall outside, where, and which way +to move. The profile checks itself: each reference has to fit the others' ranges more often than unrelated work does, +or raea says it doesn't tell this taste apart. diff --git a/docs/manual/fingerprint/index.md b/docs/manual/fingerprint/index.md new file mode 100644 index 00000000..97597ae0 --- /dev/null +++ b/docs/manual/fingerprint/index.md @@ -0,0 +1,47 @@ +--- +title: "Fingerprint: teach raea your eye" +summary: "What raea learns about good work from your notes, marks and approvals, on your Mac: the axes it judges, the Fingerprint page, and what it can't judge yet." +order: 0 +sources: ["docs/fingerprint.feature", "docs/fingerprint.md", "apps/fingerprint/src", "systems/fingerprint/src", "systems/fingerprint-use/src", "apps/app/src/fingerprint"] +--- + +A fingerprint is your sense of good work, as raea has learned it on your Mac. It's a list of axes, the things you +notice: blur, a black frame, hiss, clipping, a glow over everything, a frozen picture, and taste axes such as fake, +barren or unreadable text. Each axis has a small model over encoders that run on your Mac (SigLIP 2 for pictures, CLAP +for sound, YAMNet for sound events, EmbeddingGemma 2 for scripts and other text), so nothing leaves the machine unless +you share it. + +Your agents use it before they post a cut: `raea fingerprint score` lists the moments your fingerprint expects a note +on, each named in your words, and the agent fixes them or tells you why it didn't. It tells you the axis and the +moment, never a number. + +## In this section + +- [Train your fingerprint](train.md): learning from your own renders, your notes and marks, a box on the part of a + sound you mean, and a post-mortem when a project is done. +- [Check a cut with it](check.md): scoring a cut, checking a fix, scripts and stills, things in a shot, and profiles + of work you admire. +- [Projects and teams](projects.md): one fingerprint for each kind of work, which one judges a project, keeping one + with a project or workspace, and sharing with a team. + +## The Fingerprint page + +Open **Fingerprint** in the sidebar. The card at the top is your mark, one column of bits for each axis, grouped by +sense: pictures, sound and motion. Point at a sense to see its axes and how well each one judges. Below it, every axis +says whether it **knows** (it's trusted, and can hold a post back until you've looked), is **learning** (it gives +advice), or is still **to learn** (it only has its words). **Can't judge yet** lists what no axis can see, such as +whether a piece works as a whole. + +The name in the page's title is a menu of your fingerprints. **Settings** holds what one is for, what it builds on, +whether it's your default, and **Learn from renders**. + +A fingerprint's axes are a file you can read and edit, `~/.raea/fingerprints/NAME/fingerprint.ts`, typed by +`raea/fingerprint` ([the SDK page](../reference/sdk/fingerprint.md)). Add, remove or reword an axis there; its +evidence is raea's to keep. Every command is on [the CLI page](../reference/cli/fingerprint.md). + +## What it can't do yet + +Axes judge two-second moments, so a fingerprint can't say whether a piece works as a whole, or judge story and pace +across shots. It can say where in the frame only for blur, glitches and flat, barren surfaces; glow it can only +flag. Sound axes judge the whole sound in each moment, so a box's pitch band is kept but not yet learned from. A +stylised look or a held shot can read as a defect. And taste axes with only words never hold a post back. diff --git a/docs/manual/fingerprint/projects.md b/docs/manual/fingerprint/projects.md new file mode 100644 index 00000000..d76bf4a4 --- /dev/null +++ b/docs/manual/fingerprint/projects.md @@ -0,0 +1,67 @@ +--- +title: "Projects and teams" +summary: "One fingerprint for each kind of work, which one judges a project and why, keeping one with a project or its workspace from the app, and sharing one with a team." +order: 3 +sources: ["docs/fingerprint.feature", "apps/fingerprint/src", "apps/app/src/fingerprint", "apps/app/host/fingerprint.ts"] +--- + +## One for each kind of work + +You can keep several fingerprints, each saying what it's for. One can build on another: a client's ads can start from +your own and add what that client cares about. It inherits your axes and evidence, counts its own labels twice, and +never changes yours. + +```sh +raea fingerprint new client-ads --about "Short ads for one brand: pace, type on screen" --extends me +``` + +## Which one judges a project + +Which one judges a piece is decided by where it lives: the project's fingerprints first, then the workspace's, then +your own. At each level, the one whose `applies` matches the piece's kind wins, else that level's default. + +In the app, a project's feed names it beside the project's name at the top: the fingerprint chip. Open it to see which +fingerprint judges the project and why. When it's one of yours, **Open** takes you to its Fingerprint page. The chip +shows once you have a fingerprint. + +From a terminal: + +```sh +raea fingerprint which projects/brand/out/cut.mp4 # which one judges this, and why +``` + +## Keep one with a project + +The chip's menu has **Keep with this project** and **Keep with the workspace**. Pick one of your fingerprints under +either and raea copies it into that folder and commits it, so it travels with the folder's repository: everyone on +the team judges the project with the same taste, and the chip names it from then on. + +The same from a terminal: + +```sh +raea fingerprint share client-ads --in project projects/brand # copy it into the project +raea fingerprint use client-ads --in project projects/brand # and make it the one that judges there +``` + +Use `--in workspace` for every project in the workspace. + +## Sharing with a team + +Sharing copies the axes and the evidence (embeddings, spans and labels) into the workspace or project, to travel with +its repository. It never shares media or trained weights: each teammate's Mac trains its own from everyone's +evidence, and each person's evidence sits in its own folder so two people labelling at once never edit the same +file. When raea's encoders change, `raea fingerprint reembed FOLDER` moves the evidence to the new ones wherever it +finds the media. + +```sh +raea fingerprint diff me ana # where two tastes differ, and the moments that show it +``` + +## Notes teach the one that judged + +Your notes and marks on a piece teach the fingerprint that judged it, so a client's notes stay with the client's +fingerprint. When one of its lessons is really yours, move it up: + +```sh +raea fingerprint move --fp client-ads --axis glow # its glow labels become yours (me) +``` diff --git a/docs/manual/fingerprint/train.md b/docs/manual/fingerprint/train.md new file mode 100644 index 00000000..c385cf8e --- /dev/null +++ b/docs/manual/fingerprint/train.md @@ -0,0 +1,73 @@ +--- +title: "Train your fingerprint" +summary: "Teach a fingerprint from your own renders, your notes, marks and approvals, a box on the part of a sound you mean, and a post-mortem when a project is done; and when it trusts an axis." +order: 1 +sources: ["docs/fingerprint.feature", "apps/fingerprint/src", "systems/fingerprint/src", "apps/app/src/fingerprint"] +--- + +A new fingerprint judges from the words for each axis on day one. Everything after that is evidence: your own work, +and what you say about it. + +## From your renders + +To train the mechanical axes, choose **Learn from renders** in the Fingerprint page's Settings, or run: + +```sh +raea fingerprint bootstrap ~/renders/*.mp4 +``` + +raea damages moments of your own finished work on purpose (a blur, a black frame, a gap in the sound) and learns the +difference, held out by source, so it can't learn a shortcut. Each axis also has disguised versions of its defect, such +as a blur that was sharpened afterwards or a freeze hidden under a slow zoom. An axis that misses its disguises stays +advice. + +The same run teaches picture axes where a defect sits. It blurs, punches a hole in, greys out or glows one patch of a +frame, and learns to find the patch. An axis that finds it 85 times in 100 on renders it didn't learn from says where: +`score` then gives the moment's place in the frame, such as "top right", and on a render with a layer pass the layers +there, such as "on Floor", and `--sheet` boxes it on the frame. + +## From your reviewing + +Your own reviewing teaches it the rest: + +```sh +raea fingerprint teach cut.mp4 --axis blur --at 4.2 --bad # a moment you marked +raea fingerprint approve final.mp4 # approved work is clean, except where you left a note +raea fingerprint pair v1.mp4 v2.mp4 --axis glow # v2 is better on glow +raea fingerprint read projects/stay # read your pinned notes into labels +raea fingerprint readings --ask 50 # check 50 of those readings in the app +raea fingerprint learn-notes projects/stay # once they read right, your notes become evidence +``` + +Notes become evidence only after you've checked how they were read: the kind right 45 times in 50 and the axis 40. A +note that asks for a creative change goes to the brief, never into an axis. Each project's notes teach the +fingerprint that judges that project ([Projects and teams](projects.md)). + +## Point at what you mean + +A note can say exactly what it's about. On the picture, drop a pin and mark, box or lasso the thing you mean. + +On the sound, choose **Spectrogram** on the timeline's sound track (or open the sound in Listen) and, with the Comment +tool, drag a box around the part you mean, such as a hiss up high or a hum down low. A sound on its own shows its +spectrogram under the waveform, where any drag draws a box. The note keeps the box's time as its span and its pitch +band, your agent reads both with the note, and `learn-notes` labels that span and keeps the band in the label. Sound +axes still judge the whole sound in each moment, so the band is kept but not yet learned from. + +## When an axis is trusted + +An axis is trusted once it has 16 labels of each kind from 3 projects, ranks real defects above clean work on +projects it never saw, and catches its disguises. `raea fingerprint report` says where each one stands. An axis that is +still under its bar with 300 labels is **stuck**: more labels won't help it, so `report` names it as one to try on +another encoder. + +## Looking back on a project + +When a project is done, a post-mortem asks you what the fingerprint is least sure of: + +```sh +raea fingerprint postmortem projects/film # at most ten yes/no questions in the app, each with its moment +raea fingerprint postmortem projects/film --tally # your answers become evidence; each head's score before and after +``` + +It scores every version in the project's feed and first asks where a head disagrees with a label you gave on that +moment, then where a head is nearest its flag, one question per axis within four seconds. diff --git a/docs/manual/guides/fingerprint.md b/docs/manual/guides/fingerprint.md deleted file mode 100644 index 2e837246..00000000 --- a/docs/manual/guides/fingerprint.md +++ /dev/null @@ -1,157 +0,0 @@ ---- -title: "Fingerprint: teach raea your eye" -summary: "What raea learns about good work from your notes, marks and approvals, on your Mac: the axes it judges, how one is trained and trusted, keeping one fingerprint for each kind of work, and sharing one with a team." -order: 6 -sources: ["docs/fingerprint.feature", "docs/fingerprint.md", "apps/fingerprint/src", "systems/fingerprint/src", "systems/fingerprint-use/src", "systems/fingerprint-things/src", "apps/app/src/fingerprint"] ---- - -A fingerprint is your sense of good work, as raea has learned it on your Mac. It's a list of axes, the things you -notice: blur, a black frame, hiss, clipping, a glow over everything, a frozen picture, and taste axes such as fake, -barren or unreadable text. Each axis has a small model over encoders that run on your Mac (SigLIP 2 for pictures, CLAP -for sound, YAMNet for sound events, EmbeddingGemma 2 for scripts and other text), so nothing leaves the machine unless -you share it. - -Your agents use it before they post a cut: `raea fingerprint score` lists the moments your fingerprint expects a note -on, each named in your words, and the agent fixes them or tells you why it didn't. It tells you the axis and the -moment, never a number. - -## The Fingerprint page - -Open **Fingerprint** in the sidebar. The card at the top is your mark, one column of bits for each axis, grouped by -sense: pictures, sound and motion. Point at a sense to see its axes and how well each one judges. Below it, every axis -says whether it **knows** (it's trusted, and can hold a post back until you've looked), is **learning** (it gives -advice), or is still **to learn** (it only has its words). **Can't judge yet** lists what no axis can see, such as -whether a piece works as a whole. - -The name in the page's title is a menu of your fingerprints. **Settings** holds what one is for, what it builds on, -whether it's your default, and **Learn from renders**. - -## Training one - -A new fingerprint judges from the words for each axis on day one. To train the mechanical axes, choose **Learn from -renders**, or run: - -```sh -raea fingerprint bootstrap ~/renders/*.mp4 -``` - -raea damages moments of your own finished work on purpose (a blur, a black frame, a gap in the sound) and learns the -difference, held out by source, so it can't learn a shortcut. Each axis also has disguised versions of its defect, such -as a blur that was sharpened afterwards or a freeze hidden under a slow zoom. An axis that misses its disguises stays -advice. - -The same run teaches picture axes where a defect sits. It blurs, punches a hole in, greys out or glows one patch of a -frame, and learns to find the patch. An axis that finds it 85 times in 100 on renders it didn't learn from says where: -`score` then gives the moment's place in the frame, such as "top right", and on a render with a layer pass the layers -there, such as "on Floor", and `--sheet` boxes it on the frame. - -Your own reviewing teaches it the rest: - -```sh -raea fingerprint teach cut.mp4 --axis blur --at 4.2 --bad # a moment you marked -raea fingerprint approve final.mp4 # approved work is clean, except where you left a note -raea fingerprint pair v1.mp4 v2.mp4 --axis glow # v2 is better on glow -raea fingerprint read projects/stay # read your pinned notes into labels -raea fingerprint readings --ask 50 # check 50 of those readings in the app -``` - -Notes become evidence only after you've checked how they were read: the kind right 45 times in 50 and the axis 40. A -note that asks for a creative change goes to the brief, never into an axis. An axis is trusted once it has 16 labels of -each kind from 3 projects, ranks real defects above clean work on projects it never saw, and catches its disguises. -`raea fingerprint report` says where each one stands. An axis that is still under its bar with 300 labels is -**stuck**: more labels won't help it, so `report` names it as one to try on another encoder. - -A note can also say exactly what it's about. On the picture, drop a pin and mark, box or lasso the thing you mean. On -the sound, choose **Spectrogram** on the timeline's sound track and, with the Comment tool, drag a box around the part -you mean, such as a hiss up high or a hum down low. A sound on its own shows its spectrogram under the waveform, where -any drag draws a box. The note keeps the -box's time as its span and its pitch band. `learn-notes` labels that span and keeps the band in the label. - -## Scripts, briefs and stills - -A fingerprint reads more than video. A script, a brief or another text file gets EmbeddingGemma 2 rows, one per -paragraph, and a still gets the picture columns. Each axis judges only what the file has, so a script is never called -blurry. raea's own axes judge pictures, sound and motion. To judge writing, add an axis that reads text to your -fingerprint.ts and teach it like any other. - -## Looking back on a project - -When a project is done, a post-mortem asks you what the fingerprint is least sure of: - -```sh -raea fingerprint postmortem projects/film # at most ten yes/no questions in the app, each with its moment -raea fingerprint postmortem projects/film --tally # your answers become evidence; each head's score before and after -``` - -It scores every version in the project's feed and first asks where a head disagrees with a label you gave on that -moment, then where a head is nearest its flag, one question per axis within four seconds. - -## Things in a shot - -```sh -raea render shot.tsx --layers -o shot.mp4 # the video and its layers (shot.layers) -raea render shot.tsx --aov -o shot.aov # its depth, beside it -raea fingerprint things shot.mp4 --json > things.json -raea check shot.mp4 --things things.json -``` - -`things` finds a piece's shots, the things in each (a render's layers, and anything marked on its notes) with their -depth, and the camera's path. It says where a thing floats, where two pass through each other, where the camera lurches -in the middle of a move, and where nothing is on screen. `raea check --things` turns those into findings beside the -rest. Floating needs real depth: with the render's `--aov` pass beside it (`shot.aov` for `shot.mp4`) it uses the -render's own depth, and without one it estimates depth from the picture, which can't always tell a hanging box from -one on the floor. - -## One for each kind of work - -You can keep several fingerprints, each saying what it's for. One can build on another: a client's ads can start from -your own and add what that client cares about. It inherits your axes and evidence, counts its own labels twice, and -never changes yours. - -```sh -raea fingerprint new client-ads --about "Short ads for one brand: pace, type on screen" --extends me -raea fingerprint which projects/brand/out/cut.mp4 # which one judges this, and why -raea fingerprint use client-ads --in project projects/brand -``` - -Which one judges a piece is decided by where it lives: the project's fingerprints first, then the workspace's, then -your own. At each level, the one whose `applies` matches the piece's kind wins, else that level's default. - -Your notes and marks on a piece teach the fingerprint that judged it, so a client's notes stay with the client's -fingerprint. When one of its lessons is really yours, move it up: - -```sh -raea fingerprint move --fp client-ads --axis glow # its glow labels become yours (me) -``` - -## Profiles of work you admire - -```sh -raea fingerprint profile golden/*.mp4 --as reels --against others.txt -raea fingerprint score cut.mp4 --profile reels -``` - -A profile takes the ranges your references span: loudness, bass against treble, cuts a minute, motion, brightness, -contrast, saturation and sharpness. A cut scored against it hears which measures fall outside, where, and which way -to move. The profile checks itself: each reference has to fit the others' ranges more often than unrelated work does, -or raea says it doesn't tell this taste apart. - -## Sharing with a team - -```sh -raea fingerprint share studio --in workspace -raea fingerprint diff me ana # where two tastes differ, and the moments that show it -``` - -Sharing copies the axes and the evidence (embeddings, spans and labels) into the workspace or project, to travel with -its repository. It never shares media or trained weights: each teammate's Mac trains its own from everyone's -evidence, and each person's evidence sits in its own folder so two people labelling at once never edit the same -file. When raea's encoders change, `raea fingerprint reembed FOLDER` moves the evidence to the new ones wherever it -finds the media. - -## What it can't do yet - -Axes judge two-second moments, so a fingerprint can't say whether a piece works as a whole, or judge story and pace -across shots. It can say where in the frame only for blur, glitches and flat, barren surfaces; glow it can only -flag. Sound axes judge the whole sound in each moment, so a box's pitch band is kept but not yet learned from. A -stylised look or a held shot can read as a defect. And taste axes with only words never hold a post back. diff --git a/docs/manual/index.md b/docs/manual/index.md index afa2ece5..79c49d95 100644 --- a/docs/manual/index.md +++ b/docs/manual/index.md @@ -44,6 +44,13 @@ The raea skill (`raea skill` prints it) is the short version of the same loop; t - [3D scenes](guides/3d.md): cameras, lights, meshes and materials in TSX, drawn by Bevy. - [Apps on the web target or native](guides/apps.md): a React file that runs as a desktop app. +## Fingerprint + +- [Fingerprint: teach raea your eye](fingerprint/index.md): what raea learns about good work from your notes, on your Mac. +- [Train your fingerprint](fingerprint/train.md): your renders, notes and marks, and a box on the part of a sound you mean. +- [Check a cut with it](fingerprint/check.md): score a cut before it's posted, and check that a fix really fixed it. +- [Projects and teams](fingerprint/projects.md): which fingerprint judges a project, and keeping one with it for the team. + ## Reference - [The CLI](reference/cli.md): a page for each of the 43 `raea` verbs, with its forms, where it runs and what each flag does. diff --git a/skills b/skills index e58e2a29..f40ca317 160000 --- a/skills +++ b/skills @@ -1 +1 @@ -Subproject commit e58e2a2970f2ae3338a565e70ef913da0043d1c2 +Subproject commit f40ca317c2bc9b71d8f167a6d010b1ef97e601e0 From 30dfca463da9d4390a631b369d32ced58e465ef7 Mon Sep 17 00:00:00 2001 From: shinyobjectz Date: Fri, 13 Nov 2026 07:47:24 -0800 Subject: [PATCH 2/2] raea pins raea-skills main d71042f (#57): the fingerprint page says what each axis reads (pictures, sound, motion, text, each on its own encoder) and that no shipped axis judges recorded against generated sound. SKILLS_STRICT=1 tools/skills.sh check passes --- skills | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills b/skills index f40ca317..d71042f2 160000 --- a/skills +++ b/skills @@ -1 +1 @@ -Subproject commit f40ca317c2bc9b71d8f167a6d010b1ef97e601e0 +Subproject commit d71042f2e743dd0033a324549f2c0665f02f1536