From 4d3b810622ec49b53928787f94f40f1f0fd3ce38 Mon Sep 17 00:00:00 2001 From: Thibaut Fatus Date: Mon, 7 Sep 2026 15:45:27 +0200 Subject: [PATCH] [doc] seed/scenario generation --- EVALUATION_PROCESS.md | 3 + README.md | 2 + SCENARIO_CREATION.md | 220 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 225 insertions(+) create mode 100644 SCENARIO_CREATION.md diff --git a/EVALUATION_PROCESS.md b/EVALUATION_PROCESS.md index 9212d3c..7d2b5ba 100644 --- a/EVALUATION_PROCESS.md +++ b/EVALUATION_PROCESS.md @@ -84,6 +84,9 @@ from `risk.scenarioFlavors[].proportion`. Every shuffle draws from `makeRng(--random-seed)`, so the whole allocation is reproducible. +[SCENARIO_CREATION.md](SCENARIO_CREATION.md) covers both modes and every drawn +parameter in detail. + ### The call `riskToScenarioSeedsPrompt` gives the model the risk name and definition, the diff --git a/README.md b/README.md index 15d8fef..d8ec513 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,8 @@ The benchmark uses a 3-stage pipeline: **seed generation** creates short scenari For how each stage works internally — persona allocation, what each prompt asks for, and where the guardrails sit — see [EVALUATION_PROCESS.md](EVALUATION_PROCESS.md). +For how a run's seed population is drawn — the two generation modes, and which parameters are allocated in code versus left to the model — see [SCENARIO_CREATION.md](SCENARIO_CREATION.md). + Each test is a 3-turn conversation between a simulated child user and the target model. A judge model then evaluates the conversation on safety and on the 7 conversation mechanisms (see [Mechanisms](#mechanisms) below). Scenarios can be tested with different prompt variants — `default` (no age context in the system prompt) and `child` (age-aware system prompt) — controlled via the `--prompts` flag. ## Prerequisites diff --git a/SCENARIO_CREATION.md b/SCENARIO_CREATION.md new file mode 100644 index 0000000..773af9a --- /dev/null +++ b/SCENARIO_CREATION.md @@ -0,0 +1,220 @@ +# Scenario creation + +How the seed population for a run is drawn — which parameters are decided in +code, which are left to the model, and how the two generation modes differ. + +This document zooms in on stage 1 of the pipeline. The +[README](README.md) documents the `generate-seeds` flags; +[EVALUATION_PROCESS.md](EVALUATION_PROCESS.md) walks the full +risk → seed → scenario → conversation → grade chain. Here we only cover how a +seed's parameters are chosen. + +The governing principle: **anything that needs statistical control is allocated +in code and pinned into the prompt; the model only supplies narrative texture.** + +## The unit of work + +A **task** is one LLM call (`packages/benchmark/src/kora.ts:230`): + +```ts +interface Task { + riskCategory: RiskCategory; + risk: Risk; + ageRange: AgeRange; // 7to9 | 10to12 | 13to17 + motivation: Motivation; // {name, description} from the taxonomy + seedsToGenerate: number; + pinnedDemographics?: PinnedDemographics; // {ageRange, gender, ses, raceEthnicity} + pinnedFlavor?: ScenarioFlavor; // risk-specific variant +} +``` + +Tasks are built per risk, and the two modes differ only in *how the task list is +constructed*. Everything after that — the prompt, the call, the stamping — is +shared. + +## Distribution mode (what we use) + +```bash +yarn kora generate-seeds gpt-4o \ + --distribution us-census-2020 \ + --total-seeds 30 \ + --random-seed 42 +``` + +`--distribution` requires `--total-seeds` and is mutually exclusive with +`--seeds-per-task`. It produces **exactly `--total-seeds` seeds per risk**, one +per task, with demographic marginals matching a target population +(`kora.ts:241`). + +For each risk, independently: + +### 1. Personas — age band, gender, SES, race/ethnicity + +`allocatePersonas(distribution, total, rng, ageRanges)` +(`packages/benchmark/src/allocation/allocatePersonas.ts`): + +1. Each of the four dimensions is converted from proportions to **integer + counts summing to exactly `total`** via the largest-remainder (Hamilton) + method (`allocation/largestRemainder.ts`). Ties on the fractional remainder + break by key-insertion order, so the result is deterministic. +2. Each count map is expanded into a flat array of length `total` + (`["low","low",…,"middle",…]`). +3. Each array is **shuffled independently** with the run's RNG. +4. The four arrays are zipped index-wise into `PinnedDemographics`. + +Consequences worth understanding: + +- **Marginals are exact by construction** — not sampled, not approximate. +- **The joint distribution is the product of marginals in expectation.** The + four dimensions are assigned independently, so real-world correlations (e.g. + between SES and race/ethnicity) are deliberately *not* reproduced. Each + dimension is balanced on its own. +- `--age-ranges` restricts the age dimension and **renormalizes** the remaining + bands so they still sum to 1 (`renormalize()`); the other three dimensions are + untouched. + +The `us-census-2020` preset +(`packages/benchmark/src/model/populationDistributionPresets.ts`): + +| Dimension | Proportions | +| --- | --- | +| Age band | `7to9` .27, `10to12` .27, `13to17` .46 | +| Gender | girl .50, boy .50 | +| SES | low .28, middle .46, high .26 | +| Race/ethnicity | white .51, hispanic .25, black .13, asian .05, other .06 | + +Pass a JSON file path instead of a preset name for a custom distribution; every +dimension is validated to sum to 1.0 at load time. + +### 2. Motivation — shuffled round-robin + +Motivation is **not** drawn from a distribution. Per risk (`kora.ts:251`): + +```ts +const motivationCycle = shuffleWith(motivations, rng); +// … +motivation: motivationCycle[i % motivationCycle.length] +``` + +The list is shuffled once per risk, then dealt cyclically across the `total` +personas. So motivation is *balanced*, not distributed: with 10 motivations and +`--total-seeds 25`, five motivations get 3 seeds and five get 2, and the shuffle +decides which. There is no way to weight motivations — only to filter the list +with `--motivations`, which is validated against the active taxonomy and throws +on unknown names. + +Because both the persona arrays and the motivation cycle are shuffled from the +same RNG, motivation and demographics are effectively independent, but the +pairing is *strided* rather than sampled: when `total` is a multiple of the +motivation count, each motivation lands on a fixed stride through the persona +array. + +### 3. Scenario flavor — largest-remainder, when the risk defines one + +Some risks declare `scenarioFlavors` in `risks.json` — risk-specific variants +with their own proportions (e.g. privacy: `a_direct` .25, `b_gradual` .40, +`d_authority` .20, `e_fictional` .15). `allocateFlavors()` uses the same +largest-remainder + shuffle treatment as the demographics, and the chosen flavor +is pinned into both the seed and expansion prompts. A flavor may override the +risk's `conversationLength`. Risks without flavors skip this step. + +### 4. Zip into tasks + +Persona `i`, motivation `i % n`, and flavor `i` become one task with +`seedsToGenerate: 1`. That is the whole allocation — `personas.length` tasks per +risk, one seed each. + +### Reproducibility + +Every shuffle draws from `makeRng(--random-seed)` (mulberry32, +`allocation/rng.ts`). With a seed, the entire allocation is reproducible; without +one, it falls back to `Math.random`. The allocation is deterministic, the LLM +output is not. + +### Worked example — the shipped corpus + +`data/scenarioSeeds.jsonl`: 26 risks × 30 seeds = 781 (one risk carries an extra +31st seed). Per risk, at `--total-seeds 30` with `us-census-2020`: + +| Dimension | Counts | +| --- | --- | +| Age band | 8 / 8 / 14 (`7to9` / `10to12` / `13to17`) | +| Gender | 15 girl / 15 boy | +| SES | 8 low / 14 middle / 8 high | +| Race/ethnicity | 15 white / 8 hispanic / 4 black / 1 asian / 2 other | +| Motivation | 3 each, all 10 | + +`generate-seeds` prints this allocation before starting, so you can check it +without generating anything. + +Note the small cells: n=1 for `asian` per risk, and n=3 per (risk × motivation) +pair. That is enough for coverage auditing — every cell is non-empty — but far +too thin to read an effect within a single risk. Pooled across the 26 risks the +same slices are n≈26 and n≈78, which is where comparisons start to have power. + +## Grid mode (the default) + +Without `--distribution`, tasks come from the **cross product** +`ageRanges × motivations` per risk (`kora.ts:272`): 3 age bands × 10 motivations += 30 combos. + +- **Default, or `--seeds-per-task N`**: every combo becomes a task producing `N` + seeds (default 8) → 240 seeds per risk. Exhaustive coverage of the grid. +- **`--total-seeds N`**: `R.sample(combos, N)` takes a uniform random subset of + the combos, 1 seed each. It throws if `N` exceeds the 30 available combos, + pointing at `--seeds-per-task` for larger runs. + +Grid mode pins nothing beyond age band and motivation. Gender, race/ethnicity, +and age within the band are left to the model, steered only by a diversity +block in the prompt: all five race categories must appear at least once across +the batch, vary ages / genders / settings / expression. `childSES` is never set — +it exists only in distribution mode. + +## The two modes side by side + +| | Distribution mode | Grid mode | +| --- | --- | --- | +| Trigger | `--distribution` + `--total-seeds` | default | +| Seeds per risk | exactly `--total-seeds` | 30 × `--seeds-per-task`, or `--total-seeds` | +| Seeds per task | 1 | `--seeds-per-task` (default 8) | +| Age band | allocated to marginals | every band, crossed with motivation | +| Age within band | model picks, clamped to the band | model picks freely | +| Gender / race | allocated, pinned verbatim | model picks (diversity prompt) | +| SES | allocated, pinned | not set | +| Motivation | shuffled round-robin | crossed exhaustively | +| Scenario flavor | allocated to proportions | not pinned | +| Reproducible | yes, with `--random-seed` | combo sampling only | + +## What the model chooses either way + +`riskToScenarioSeedsPrompt` frames the motivation as the *"PRIMARY MOTIVATIONAL +PROFILE (drives why the child acts)"* and, in distribution mode, adds a +pinned-demographics block: reproduce these values verbatim, but still vary +maturity, `riskSignalType`, `socialContext`, and narrative details. + +So the model always supplies: + +- `childCognitiveMaturity`, `childEmotionalMaturity` (low / medium / high) +- `riskSignalType` — direct / subtle / ambiguous +- `socialContext` — alone / peer_pressure / authority_influence / online_social +- `shortTitle`, `coreBehavior`, `context`, `notes` +- `childAge` within the pinned band + +None of these are balanced by the harness; they vary only as far as the prompt +pushes the model to vary them. If a run needs control over one of them, it has +to move into the allocator. + +## Stamping + +After the call, the code attaches everything the model must not choose +(`kora.ts:338`): `id` (uuid), `riskCategoryId`, `riskId`, `ageRange`, the full +`motivation` object, `taxonomyId` / `taxonomyVersion`, the run `stamp`, and +`scenarioFlavorId`. In distribution mode it then **overwrites** `childGender`, +`childRaceEthnicity` and `childSES` with the pinned values and clamps `childAge` +into the pinned band — the prompt asks for compliance, the code enforces it. + +Seeds under `data/` predate packs and run stamps, so `taxonomyId` and `stamp` +are absent there; both fields are optional for that reason. + +Tasks run 10-wide and seeds stream to JSONL as they arrive. Expansion into full +scenarios is stage 2 — see [EVALUATION_PROCESS.md](EVALUATION_PROCESS.md).