Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions docs/manual/fingerprint/check.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
title: "Check a cut with it"
summary: "Score a cut with your fingerprint before it's posted, check that a fix removed a flag instead of hiding it, judge scripts and stills, find things floating in a 3D shot, and compare a cut with work you admire."
order: 2
sources: ["docs/fingerprint.feature", "apps/fingerprint/src", "systems/fingerprint-use/src", "systems/fingerprint-things/src"]
---

## Score a cut

```sh
raea fingerprint score out/cut.mp4 # the moments it expects a note on, first
raea fingerprint score out/cut.mp4 --sheet cut.png # the same on a sheet of frames, with where boxed
raea fingerprint score out/cut.mp4 --timeline # the piece second by second, in words
raea fingerprint which out/cut.mp4 # which fingerprint judged it, and why
```

`score` names which fingerprint judged the piece, then lists the moments it expects a note on, each in your words. On
a picture axis that has learned where, a moment also says where in the frame and, on a render with a layer pass, the
layers there ("top right on Floor"). `--json` gives every axis for every window, for a script.

An axis that **knows** can hold a post back until you've looked. One that is **learning** only advises. Your agents
run `score` before they post, fix each moment that holds the cut back, or tell you why they didn't.

## Check a fix

```sh
raea fingerprint score out/v2.mp4 --was out/v1.mp4
```

For each moment that held v1 back, this says whether v2 fixed it, still has it, or looks **disguised**: nearer the
axis's disguised defects than clean work, so the fix hid the defect instead of removing its cause. Only a fixed flag
counts as fixed.

## Scripts, briefs and stills

A fingerprint reads more than video. A script, a brief or another text file gets EmbeddingGemma 2 rows, one per
paragraph, and a still gets the picture columns. Each axis judges only what the file has, so a script is never called
blurry. raea's own axes judge pictures, sound and motion. To judge writing, add an axis that reads text to your
fingerprint.ts and teach it like any other.

## Things in a shot

```sh
raea render shot.tsx --layers -o shot.mp4 # the video and its layers (shot.layers)
raea render shot.tsx --aov -o shot.aov # its depth, beside it
raea fingerprint things shot.mp4 --json > things.json
raea check shot.mp4 --things things.json
```

`things` finds a piece's shots, the things in each (a render's layers, and anything marked on its notes) with their
depth, and the camera's path. It says where a thing floats, where two pass through each other, where the camera lurches
in the middle of a move, and where nothing is on screen. `raea check --things` turns those into findings beside the
rest. Floating needs real depth: with the render's `--aov` pass beside it (`shot.aov` for `shot.mp4`) it uses the
render's own depth, and without one it estimates depth from the picture, which can't always tell a hanging box from
one on the floor.

## Profiles of work you admire

```sh
raea fingerprint profile golden/*.mp4 --as reels --against others.txt
raea fingerprint score cut.mp4 --profile reels
```

A profile takes the ranges your references span: loudness, bass against treble, cuts a minute, motion, brightness,
contrast, saturation and sharpness. A cut scored against it hears which measures fall outside, where, and which way
to move. The profile checks itself: each reference has to fit the others' ranges more often than unrelated work does,
or raea says it doesn't tell this taste apart.
47 changes: 47 additions & 0 deletions docs/manual/fingerprint/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
title: "Fingerprint: teach raea your eye"
summary: "What raea learns about good work from your notes, marks and approvals, on your Mac: the axes it judges, the Fingerprint page, and what it can't judge yet."
order: 0
sources: ["docs/fingerprint.feature", "docs/fingerprint.md", "apps/fingerprint/src", "systems/fingerprint/src", "systems/fingerprint-use/src", "apps/app/src/fingerprint"]
---

A fingerprint is your sense of good work, as raea has learned it on your Mac. It's a list of axes, the things you
notice: blur, a black frame, hiss, clipping, a glow over everything, a frozen picture, and taste axes such as fake,
barren or unreadable text. Each axis has a small model over encoders that run on your Mac (SigLIP 2 for pictures, CLAP
for sound, YAMNet for sound events, EmbeddingGemma 2 for scripts and other text), so nothing leaves the machine unless
you share it.

Your agents use it before they post a cut: `raea fingerprint score` lists the moments your fingerprint expects a note
on, each named in your words, and the agent fixes them or tells you why it didn't. It tells you the axis and the
moment, never a number.

## In this section

- [Train your fingerprint](train.md): learning from your own renders, your notes and marks, a box on the part of a
sound you mean, and a post-mortem when a project is done.
- [Check a cut with it](check.md): scoring a cut, checking a fix, scripts and stills, things in a shot, and profiles
of work you admire.
- [Projects and teams](projects.md): one fingerprint for each kind of work, which one judges a project, keeping one
with a project or workspace, and sharing with a team.

## The Fingerprint page

Open **Fingerprint** in the sidebar. The card at the top is your mark, one column of bits for each axis, grouped by
sense: pictures, sound and motion. Point at a sense to see its axes and how well each one judges. Below it, every axis
says whether it **knows** (it's trusted, and can hold a post back until you've looked), is **learning** (it gives
advice), or is still **to learn** (it only has its words). **Can't judge yet** lists what no axis can see, such as
whether a piece works as a whole.

The name in the page's title is a menu of your fingerprints. **Settings** holds what one is for, what it builds on,
whether it's your default, and **Learn from renders**.

A fingerprint's axes are a file you can read and edit, `~/.raea/fingerprints/NAME/fingerprint.ts`, typed by
`raea/fingerprint` ([the SDK page](../reference/sdk/fingerprint.md)). Add, remove or reword an axis there; its
evidence is raea's to keep. Every command is on [the CLI page](../reference/cli/fingerprint.md).

## What it can't do yet

Axes judge two-second moments, so a fingerprint can't say whether a piece works as a whole, or judge story and pace
across shots. It can say where in the frame only for blur, glitches and flat, barren surfaces; glow it can only
flag. Sound axes judge the whole sound in each moment, so a box's pitch band is kept but not yet learned from. A
stylised look or a held shot can read as a defect. And taste axes with only words never hold a post back.
67 changes: 67 additions & 0 deletions docs/manual/fingerprint/projects.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
title: "Projects and teams"
summary: "One fingerprint for each kind of work, which one judges a project and why, keeping one with a project or its workspace from the app, and sharing one with a team."
order: 3
sources: ["docs/fingerprint.feature", "apps/fingerprint/src", "apps/app/src/fingerprint", "apps/app/host/fingerprint.ts"]
---

## One for each kind of work

You can keep several fingerprints, each saying what it's for. One can build on another: a client's ads can start from
your own and add what that client cares about. It inherits your axes and evidence, counts its own labels twice, and
never changes yours.

```sh
raea fingerprint new client-ads --about "Short ads for one brand: pace, type on screen" --extends me
```

## Which one judges a project

Which one judges a piece is decided by where it lives: the project's fingerprints first, then the workspace's, then
your own. At each level, the one whose `applies` matches the piece's kind wins, else that level's default.

In the app, a project's feed names it beside the project's name at the top: the fingerprint chip. Open it to see which
fingerprint judges the project and why. When it's one of yours, **Open** takes you to its Fingerprint page. The chip
shows once you have a fingerprint.

From a terminal:

```sh
raea fingerprint which projects/brand/out/cut.mp4 # which one judges this, and why
```

## Keep one with a project

The chip's menu has **Keep with this project** and **Keep with the workspace**. Pick one of your fingerprints under
either and raea copies it into that folder and commits it, so it travels with the folder's repository: everyone on
the team judges the project with the same taste, and the chip names it from then on.

The same from a terminal:

```sh
raea fingerprint share client-ads --in project projects/brand # copy it into the project
raea fingerprint use client-ads --in project projects/brand # and make it the one that judges there
```

Use `--in workspace` for every project in the workspace.

## Sharing with a team

Sharing copies the axes and the evidence (embeddings, spans and labels) into the workspace or project, to travel with
its repository. It never shares media or trained weights: each teammate's Mac trains its own from everyone's
evidence, and each person's evidence sits in its own folder so two people labelling at once never edit the same
file. When raea's encoders change, `raea fingerprint reembed FOLDER` moves the evidence to the new ones wherever it
finds the media.

```sh
raea fingerprint diff me ana # where two tastes differ, and the moments that show it
```

## Notes teach the one that judged

Your notes and marks on a piece teach the fingerprint that judged it, so a client's notes stay with the client's
fingerprint. When one of its lessons is really yours, move it up:

```sh
raea fingerprint move --fp client-ads --axis glow # its glow labels become yours (me)
```
73 changes: 73 additions & 0 deletions docs/manual/fingerprint/train.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
title: "Train your fingerprint"
summary: "Teach a fingerprint from your own renders, your notes, marks and approvals, a box on the part of a sound you mean, and a post-mortem when a project is done; and when it trusts an axis."
order: 1
sources: ["docs/fingerprint.feature", "apps/fingerprint/src", "systems/fingerprint/src", "apps/app/src/fingerprint"]
---

A new fingerprint judges from the words for each axis on day one. Everything after that is evidence: your own work,
and what you say about it.

## From your renders

To train the mechanical axes, choose **Learn from renders** in the Fingerprint page's Settings, or run:

```sh
raea fingerprint bootstrap ~/renders/*.mp4
```

raea damages moments of your own finished work on purpose (a blur, a black frame, a gap in the sound) and learns the
difference, held out by source, so it can't learn a shortcut. Each axis also has disguised versions of its defect, such
as a blur that was sharpened afterwards or a freeze hidden under a slow zoom. An axis that misses its disguises stays
advice.

The same run teaches picture axes where a defect sits. It blurs, punches a hole in, greys out or glows one patch of a
frame, and learns to find the patch. An axis that finds it 85 times in 100 on renders it didn't learn from says where:
`score` then gives the moment's place in the frame, such as "top right", and on a render with a layer pass the layers
there, such as "on Floor", and `--sheet` boxes it on the frame.

## From your reviewing

Your own reviewing teaches it the rest:

```sh
raea fingerprint teach cut.mp4 --axis blur --at 4.2 --bad # a moment you marked
raea fingerprint approve final.mp4 # approved work is clean, except where you left a note
raea fingerprint pair v1.mp4 v2.mp4 --axis glow # v2 is better on glow
raea fingerprint read projects/stay # read your pinned notes into labels
raea fingerprint readings --ask 50 # check 50 of those readings in the app
raea fingerprint learn-notes projects/stay # once they read right, your notes become evidence
```

Notes become evidence only after you've checked how they were read: the kind right 45 times in 50 and the axis 40. A
note that asks for a creative change goes to the brief, never into an axis. Each project's notes teach the
fingerprint that judges that project ([Projects and teams](projects.md)).

## Point at what you mean

A note can say exactly what it's about. On the picture, drop a pin and mark, box or lasso the thing you mean.

On the sound, choose **Spectrogram** on the timeline's sound track (or open the sound in Listen) and, with the Comment
tool, drag a box around the part you mean, such as a hiss up high or a hum down low. A sound on its own shows its
spectrogram under the waveform, where any drag draws a box. The note keeps the box's time as its span and its pitch
band, your agent reads both with the note, and `learn-notes` labels that span and keeps the band in the label. Sound
axes still judge the whole sound in each moment, so the band is kept but not yet learned from.

## When an axis is trusted

An axis is trusted once it has 16 labels of each kind from 3 projects, ranks real defects above clean work on
projects it never saw, and catches its disguises. `raea fingerprint report` says where each one stands. An axis that is
still under its bar with 300 labels is **stuck**: more labels won't help it, so `report` names it as one to try on
another encoder.

## Looking back on a project

When a project is done, a post-mortem asks you what the fingerprint is least sure of:

```sh
raea fingerprint postmortem projects/film # at most ten yes/no questions in the app, each with its moment
raea fingerprint postmortem projects/film --tally # your answers become evidence; each head's score before and after
```

It scores every version in the project's feed and first asks where a head disagrees with a label you gave on that
moment, then where a head is nearest its flag, one question per axis within four seconds.
Loading
Loading