Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions directory/skills/raea/3d/games.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,10 +236,12 @@ function Hop() {
registerRoot(() => <Composition id="hop3d" component={Hop} width={1280} height={720} fps={30} durationInFrames={FRAMES} />);
```

## Sound: one-shots on transitions, a bed under the voice
## Sound: one-shots on transitions, music ducked under the voice

A game's sound is Tone.js on raea (`motion/music` has the whole of it). Build the instruments at the top level, give
the bed and the sounds their buses, and fire each one-shot on the frame its event happens:
A game's sound is Tone.js on raea (`motion/music` has the whole of it). Every input and outcome gets a sound, the
world's own sounds (machines, footsteps, a room's tone) sit where they are, and music, when the game has some, is a
loop written to come round, not a pad. Build the instruments at the top level, give the music and the sounds their
buses, and fire each one-shot on the frame its event happens:

- `useGameEvents(INITIAL, fold, (before, after) => machine.state(after.hero) === "land" && machine.state(before.hero) !== "land")`
gives the frames where a transition happened. `<Tone.Sfx when={landings.includes(frame)} play={(t) => thump.triggerAttackRelease("C2", "8n", t)} />`
Expand Down Expand Up @@ -267,4 +269,5 @@ function Game() {
}
```

sdk/react/bench/tone/game is a working game with all of this (a machine, a ducked bed, an SAO coin, a voice line).
sdk/react/bench/tone/game is a working game with all of this (a machine, a chiptune loop that asks and answers,
ducked under a voice line, an SAO coin).
2 changes: 2 additions & 0 deletions directory/skills/raea/craft/floor.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ direction asks for it by name, and the direction then says why in a line.
- **Template captions.** White bold text with one yellow word, an emoji pop, a zoom on every sentence, a whoosh on
every entrance.
- **The same move everywhere.** One entrance, one zoom or one sound on every beat makes every beat weaker.
- **Music by default.** A bed under every second, or a held pad and drone standing in for a score. Decide what each
scene is heard as, and compose what it needs (`motion/music`).
- **Copy that sounds like an ad.** Buzzwords, "Not X. Y." cadences, aphorisms, a label above every headline.

## Never
Expand Down
5 changes: 3 additions & 2 deletions directory/skills/raea/craft/modes/story.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,9 @@ they want the whole film.
moves. The easy default is choosing a look shot by shot from references, and that ends up as several films.
- **Each shot's subject.** Choose the person, object or action each shot is about. The easy default is a wide
establishing shot, so question every one.
- **What the music does.** It either leads, and you cut to its map, or it sits under the picture as a score. The easy
default is a cut on every bar.
- **What the music does.** It either leads, and you cut to its map, or it scores the picture, written to the cut, or a
scene has none and belongs to its own sound or to silence. The easy defaults are a cut on every bar and a score
that is one held chord.
- **The poster frame.** Choose it before anything renders, and build the edit toward it.

## The rut
Expand Down
22 changes: 20 additions & 2 deletions directory/skills/raea/craft/rubric.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,32 @@ piece's modes isn't asked.
- land (pitch, story, explain): Does the end land: something this audience can do next, a payoff, or a return to the opening?
- pace (all): Do shot lengths change with what they hold, with no stretch sitting still longer than it earns, and cuts falling on words or beats?
- read (all): Is every word that matters large, sharp against what is behind it, and on long enough to read?
- sound (all): Is the voice clear over everything else, with music and effects marking moments rather than filling space?
- sound-silent (silent): With no voice, do the music and the effects carry it: cuts and moments landing on the beat, each effect marking a moment rather than filling space, nothing muddy? (asked in place of sound when no one speaks)
- sound (all): Is the voice clear over everything else, with music and effects marking moments rather than filling space, and any music written for this cut (a tune, harmony that moves, a rhythm, an ending on a cut) rather than a pad or drone held under it?
- sound-silent (silent): With no voice, do the music and the effects carry it: cuts and moments landing on the beat, the music composed (a tune, moving harmony, a rhythm, sections that change with the picture), each effect marking a moment rather than filling space, nothing muddy? (asked in place of sound when no one speaks)
- world (all): Do colour, type, motion and sound hold one direction from the first frame to the last?
- own (all): With the name covered, could this only be this project, rather than any product in its category?
- feel (play): Does the player's input change the picture right away and in a way they can read?
- again (play): After a failure, is it clear what went wrong and is the next attempt one press away?
- task (tool): Can the task the screen exists for be done without stopping to work out where to look?

## Music that sounds generated

Listen for these before the decision model does; each is how a made score gives itself away, and the fix
(`motion/music`):

- **One held texture.** A pad, a sine sub and a long reverb under everything, no attack anywhere. Write a motif, chords
that move and a part with rhythm; reverb per instrument, nothing on the bus.
- **Harmony that stands still.** One chord for four bars or more in a cue that's meant to move. Change it every bar or two.
- **A loop, not a cue.** The same two bars for thirty seconds. Vary the motif, change the arrangement every four to
eight bars, and turn where the edit turns.
- **Played by a machine.** Every note at one velocity, exactly on the grid. Spread the dynamics; move played parts a
few milliseconds; use a recorded groove.
- **No ending.** A fade-out. End on a chord or a hit on the last cut.
- **Everywhere.** Music under every scene because the piece "needs music". Some scenes belong to silence or to their
own sound.
- **The wrong source.** A whole score from TokenSynth takes (organ-like sustains, buzz) or from an SAO "ambient
music" prompt. Recorded instruments play the parts; TokenSynth fills a timbre no library has; SAO makes effects.

## Ranking

| | Means | Do |
Expand Down
2 changes: 1 addition & 1 deletion directory/skills/raea/craft/verbs/quieter.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ note names. Keep the point of view: a quieter piece still has one.
- Light: fewer sources pulsing, shorter pulses, lower bloom. Glow only where the eye should go.
- Colour: lower saturation on what's around the accent, not on the accent.
- Motion: fewer moving things at once, smaller moves, no bounce.
- Sound: duck the bed further under the voice; drop layers before lowering levels.
- Sound: duck the music further under the voice; drop parts and layers before lowering levels.
- Never solve quiet with blur, fog or grain that softens the work itself.

## Verify
Expand Down
13 changes: 10 additions & 3 deletions directory/skills/raea/craft/verbs/remix.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,32 @@ sound: voice, music, foley and the master.

## Assess
- Listen once for the voice alone: is every word clear over everything else?
- Then listen for what's filling space rather than marking moments: a bed at one level under everything, a whoosh on
- Then listen for what's filling space rather than marking moments: music at one level under everything, a whoosh on
every entrance.
- Then for what the music is: a tune you could hum, chords that move, a part with rhythm, an ending on a cut; or one
held texture (a pad, a drone) under a long reverb.
- With a song, check the cuts and hits against its map.

## Decide
- What each scene is heard as: silence, its own sound, source music, score. Some scenes need no music.
- What leads: the voice, or the music. Everything else sits under it.
- Which moments get a sound. A few marked moments beat a sound on every one.

## Change
- The voice first; duck music and foley under it.
- Sound marks moments: a hit on the cut that matters, silence before the drop. Remove what only fills.
- Music: a library instrument first; TokenSynth takes are curated, not trusted. Under a voice, a sparse score.
- Music: composed for the cut (`motion/music`) on recorded instruments first; TokenSynth takes are curated, not
trusted. Under a voice, thin the arrangement and let the tune answer between lines.
- Effects: layered (a transient, a body, a tail), with repeats varied (`motion/sound`).
- Master at -14 LUFS integrated, true peak at or below -1 dBTP.

## Verify
- Measure the loudness. Listen through once on small speakers and once on headphones.

## When it goes wrong
- The voice is clear on headphones and buried on a phone's speaker.
- The music pumps as it ducks under the voice. Use a slower release, or a lower bed.
- The music pumps as it ducks under the voice. Use a slower release, or lower music.
- The score has turned into a drone: one chord held under a long reverb. Move the harmony, add a part with rhythm,
and take the reverb off the bus.
- Clipping, or a true peak over the limit. Measure, and don't trust the meter by eye.
- The loudness is off the delivery's spec.
25 changes: 17 additions & 8 deletions directory/skills/raea/motion.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ Read the files a task needs with `raea guide motion/NAME`, one or two at a time,
| planning how shots cut into each other: hooks on the thirds, the eye carried through each cut | `motion/match-cuts` |
| moving anything: entrances, exits, counters, push-ins | `motion/animating` |
| cutting takes, framing a face, trimming, clip sound | `motion/footage` |
| music beds, foley, gains, fades, the master | `motion/sound` (the music itself: `motion/music`) |
| what each scene is heard as, effects and foley, gains, fades, the master | `motion/sound` (the music itself: `motion/music`) |
| where sounds come from: the local models and their setup, sample libraries, effect packs, MIDI | `motion/sources` |
| captions, or anything cued to what someone says | `motion/words` |
| putting parts over a face, platform zones, the proof | `motion/placement` |
| cards, stacks, type sizes, fonts | `motion/layout` |
Expand Down Expand Up @@ -63,9 +64,11 @@ Read the files a task needs with `raea guide motion/NAME`, one or two at a time,
(`text-brand-accent`, `bg-brand-surface`, `font-brand`, `text-title`, `rounded-brand` and the rest in AGENTS.md),
or `var(--brand-*)` where a class can't reach. Never write a hex palette of your own. A role the kit leaves empty
is yours to decide from the brand's own colours (`motion/brand`).
8. **Sound is part of the edit.** Each sound goes in its own `<Sequence from layout="none">`. Use `gainDb` for
levels, Tone.js for music (`motion/music`: TokenSynth or sample-library instruments in a Sampler) and `sao()` for
foley, and put one `<Mix lufs={-14} />` in the composition. Put the sounds in the tree in time order.
8. **Sound is part of the edit.** Decide what each scene is heard as (silence, its own sound, source music, score,
effects) before placing any; music is composed for the cut, never a bed by default. Each sound goes in its own
`<Sequence from layout="none">`. Use `gainDb` for levels, Tone.js for music (`motion/music`: sample-library or
TokenSynth instruments in a Sampler, playing a motif, chords that move and a rhythm) and `sao()` for foley, and
put one `<Mix lufs={-14} />` in the composition. Put the sounds in the tree in time order.
9. **Keep animated styles present on every frame.** The engine keeps a style property that React stops sending.
Write `transform: scale(1)` and `opacity: 1` rather than leaving them out when an animation settles. Always set
`fontFamily`, because the engine's default face is a serif.
Expand All @@ -80,13 +83,19 @@ unreadable text, the wrong loudness) and bans (invented claims, generated imager
## A whole composition, small

```tsx
import { AbsoluteFill, Composition, Easing, Mix, Sequence, Audio, interpolate, registerRoot, sao, useCurrentFrame, useVideoConfig } from "raea";
import { AbsoluteFill, Composition, Easing, Mix, Sequence, Audio, interpolate, registerRoot, sao, staticFile, useCurrentFrame, useVideoConfig } from "raea";
import * as Tone from "raea/tone";
import { css } from "./styles.gen";

// the bed: one TokenSynth instrument in a Sampler, a bar of notes on a loop (motion/music)
const keys = new Tone.Sampler({ urls: Tone.tokensynth("a soft felt piano, warm, intimate", ["C3", "C4"], { seconds: 2 }), volume: -14 }).connect(Tone.buses.music);
new Tone.Sequence((t, n) => keys.triggerAttackRelease(n, "4n", t), ["C4", "G3", "E4", "G3"], "4n").start(0);
// the music: a chord on the title and a two-note lift after it, on a library piano (`raea sounds get
// salamander/piano`: roots a minor third apart, D#4 as Ds4.mp3); a sting, not a bed (motion/music)
const ROOTS = ["C4", "D#4", "F#4", "A4", "C5", "D#5"];
const piano = new Tone.Sampler({ urls: Object.fromEntries(ROOTS.map((n) => [n, staticFile(`sounds/salamander-piano/${n.replace("#", "s")}.mp3`)])), release: 1, volume: -10 }).connect(Tone.buses.music);
new Tone.Part((t, n) => piano.triggerAttackRelease(n.notes, n.dur, t, n.vel), [
{ time: "0:0", notes: ["C4", "E4", "G4", "C5"], dur: "2n", vel: 0.8 },
{ time: "0:2", notes: ["D5"], dur: "8n", vel: 0.55 },
{ time: "0:2:2", notes: ["E5"], dur: "4n", vel: 0.65 },
]).start(0);
Tone.getTransport().start();
const WHOOSH = sao("soft airy whoosh, short, clean, no music", { seconds: 0.8 });

Expand Down
9 changes: 5 additions & 4 deletions directory/skills/raea/motion/assets.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
and `Tone.tokensynth(timbre, notes)` for an instrument's notes in a `Tone.Sampler` (`motion/music`).
- Recorded instruments and drum hits come from the open sample libraries: `raea sounds WORDS` finds one, `raea sounds
get PACK/INSTRUMENT` copies its files into `media/sounds/PACK-INSTRUMENT/` with a `LICENCE.txt`, and prints the
`staticFile` paths and the Sampler that plays them.
`staticFile` paths and the Sampler that plays them. Orchestras, drum kits, effect packs and MIDI grooves fetched by
hand, and how to set up SAO and TokenSynth, are in `motion/sources`.
- Show stills with `<Img src>`. The frame waits for the image to load, and a missing one fails the render, naming
`src`.
- Keep pictures real: the person's footage, stock already in the object's media, or things you build (components,
Expand All @@ -28,9 +29,9 @@ error and so is a URL (`https://...`). `staticFile` checks both. A module import
fails loudly with the line that has it instead of rendering a black clip.

Generated imagery is banned outright (floor): no AI-generated images or video, ever. Generated sound is allowed by
policy, and made on this computer: TokenSynth for an instrument's notes (music), SAO for foley, Pocket TTS
(`<Speech>`) for narration over the picture. The sample libraries are recordings under open licences (CC0, CC-BY, public domain);
a CC-BY one is credited where the piece is shown.
policy, and made on this computer: TokenSynth for an instrument's notes where no library has the timbre, SAO for
foley, Pocket TTS (`<Speech>`) for narration over the picture. The sample libraries are recordings under open licences
(CC0, CC-BY, public domain); a CC-BY one is credited where the piece is shown.

## Common mistakes

Expand Down
4 changes: 3 additions & 1 deletion directory/skills/raea/motion/gen.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
## What to do

- Generate a picture, a clip, music, an effect, a voice, a 3D model or words with `raea gen` when the piece needs
something that isn't in the footage, the stock or the scene. `-o` is the folder; each command prints the files it wrote; use it like any
something that isn't in the footage, the stock or the scene. Music only when the person asks for a generated
track: a score is composed on instruments for the cut (`motion/music`), and a generated track can't be cut on the
bar or edited note by note. `-o` is the folder; each command prints the files it wrote; use it like any
other media (`<Img src>`, `<Video src>`, `<Audio src>`, `<Model src>`).
- Quote first. `raea gen plan FLOW` and `--plan` on a one-shot cost nothing and say where each step runs and what it
costs in cents. A run past its cap is refused before anything is spent: `--max-cents N`, else
Expand Down
Loading
Loading