Repository navigation
AK0-AK4: raea ask, questions about a piece's picture and sound, paid from the person's plan - #211
Merged
Merged
Conversation
…om the person's plan docs/ask.feature sets the model and plans AK1-AK9: one question format and one clef client for every caller (the CLI, pipelines and scripts, raea check, the fingerprint's double check, the app's agents, critique and game checks, raea judge); ask understands a question before it spends; the person's pool first, then an agent key, then their own Cloudflare keys; one parallel runner that cuts each clip once; standard sets (rubric, picture with sound, the fingerprint's unlabelled axes, the check packs' questions, decisions); and answers kept and labelled so each question earns its threshold, better words can be measured, and a calibrated question can later be learned locally.
…sets and clef's limits A question is clef's own shape (yes, choose, grade, rate), what it needs to be asked (`when`, and the question asked in its place with `swap`, so sound to sound-silent is data), what it looks at and listens to, its window, label words and calibration. Every answer reads as a problem score from 0 to 1 whatever its type, so one calibration fits all; a question flags past its calibrated threshold, or past even odds as advice. A wording is versioned by a stable hash. The standard sets: the rubric, read from its guide (one copy); av, picture with sound; defects, the fingerprint's seven axes with no labels, in its own words; platform, the check packs' 17 model questions with their calibration (3 pass the 0.8 gate); decide, picks and before/after. The limits are raea-api's where stricter, so a call that passes them passes on every route. sdk/raea/src/ask.gen.ts is generated from the types and checked by a test.
…t; raea judge moved onto it One call: the state, the questions by id, and stills (JPEG), videos (MP4) and sounds (MP3) cut with raea's FFmpeg to clef's limits (480 px H.264 at a small bitrate, mono 16 kHz MP3, JPEG at most 1,600 px; ffprobe says first what a file holds). It goes the first way that answers: the raea app's host (/host/ask, which adds its sign-in), an agent key (RAEA_AGENT_KEY with RAEA_API, for pipelines and servers), the person's own Cloudflare account; with none it says how to get one. No plan or no ask scope hands on to the next way; a spent pool stops unless the person has their own keys. Unchanged questions on unchanged media come from a cache by request hash; RAEA_CLEF_REPLAY gives recorded answers to tests. raea judge asks through it (Question::from_wire reads its rubric's clef-native questions), so apps/judge's own clef.rs is gone and the judge pays from the plan when the app is signed in. docs/settings.md lists the new settings.
… asked in parallel and kept raea-app-ask. prepare.rs reads what raea knows of the piece before it spends: what the file holds, the nearest brief.ts (through raea spec) and a words file. A question that doesn't apply is dropped or swapped with why (a silent piece asks sound-silent in place of sound); not knowing whether anyone speaks, both variants go in one call with the voice question, and the answer sets the other aside. runner.rs plans a batch into calls (by file, window, files beside it and what the questions look at, 64 a call), cuts each clip once, calls in parallel (--jobs), streams the answers, and when the pool is spent partway keeps what came and reports the rest as waiting. record.rs keeps every call and answer under ~/.raea/ask with the clips it cut, by hash, at most 2 GB. verbs.rs is raea ask: own words (--choose, --grade, --rate, --flag), the standard sets (--set, --std, ask list), windows (--from, --to, --at), --with, --batch (a JSON list or lines, - for stdin), --dry, --json and --gate for pipelines. sdk/raea's raea/ask gives scripts ask and askMany over raea ask --batch --json. The clef client's failures are typed so a spent pool waits rather than fails; decide.changed needs files beside it. The manual's CLI and SDK pages are written with it; AK1, AK1b, AK2 and AK3 are done in docs/ask.feature.
…itself The app's critique and game check now ask through raea ask (raea-feed) with their own state: --brief none (or "brief": "none" in a batch) keeps raea ask from adding the nearest brief's facts to it, and with no brief and the caller's state, the caller says what the piece is. docs/ask.feature AK4 says what's built and in review: the pool route's sounds, criteria, pool left and ask scope on raea-api, and the app's /host/ask, ask tool and critique.
L1's dupes step counted 300 exact groups against 294 allowed, and each of the six new groups was ask code beside an existing twin. Recorded answers are read in one step, the cache folder names its PathBuf, the media prune reads each entry's metadata in one chain, a temp clip is removed with .ok(), the client encodes base64 with the engine itself, and a question id is checked over its bytes.
This was referenced Nov 20, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
raea askasks questions about a piece's picture and sound. clef-omni (the decision model behind raea's checks) answers each one in words with a probability. The call is paid from the person's raea plan, and every answer is kept so a question can be calibrated over time. This PR is AK0 through the raea side of AK4 in docs/ask.feature.What's in it
raea-core-ask (
core/ask) defines:rubric, read from sdk/guides/craft/rubric.md (one source, not a copy);av, for picture against sound;defects, the fingerprint axes that have no labels yet;platform;decide.The types are exported to
sdk/raea/src/ask.gen.ts.raea-io-clef (
io/clef) is a single clef client, whichraea judgenow uses too. It handles:/host/ask, on the person's plan;RAEA_AGENT_KEYwithRAEA_API, for a pipeline or CI;When the pool is spent, it stops (unless the person has their own keys), and the questions not yet asked wait instead of being dropped.
raea ask(apps/ask) reads the piece before it spends anything:sound-silentinstead of the rubric'ssound. When it's unknown whether anyone speaks, a voice question rides in the same call and settles which variant counts.--jobs).--json).--batchtakes many clips (a JSON list, JSON lines, or-for stdin).--dryshows exactly what would be sent and its estimated cost, and spends nothing.--gate/--gate anygive exit codes a pipeline can use.raea ask listlists the standard questions.~/.raea/ask; the clips are capped at 2 GB.--brief noneis for a caller that writes the whole state itself, as the app's critique does.raea/askin sdk/raea providesask()andaskMany()overraea ask --batch - --json.The manual pages are docs/manual/reference/cli/ask.md and sdk/ask.md, and docs/settings.md lists the new settings.
The
askverb is dispatched through apps/solid, because raea-cli is at its 2,000-line limit.Related PRs
pool_left_usdand agent-keyaskscope./host/ask, the agents'asktool, and the critique's judge moved ontoraea ask.The pool route waits on raea-api's deploy. Until then,
raea askuses the person's own keys.Checked
raea-app-ask has 6 tests, which never spend:
--with;raea-io-clef's tests cover routing, refusals, the spent pool and replay. raea-core-ask's tests cover parsing, versions and the library.
raea-cli's help tests pass.
One real call on own keys cost $0.0003. A test pattern scored 0.88 on the defect question, "nobody speaking" scored 0.00, and "the sound fits" scored 0.06. The answer was cached and recorded.
L1 (
tools/agents/check --land --lane ask, on origin/main plus this branch) passes:Five steps fail, none on this branch's files:
raea controller, show's labels), and their pages are out of date. ask's own pages are current.