Extend practice feedback and problem bank checks

.github/workflows/avatar.yml

@@ -0,0 +1,25 @@
+name: avatar
+
+on:
+  schedule:
+    - cron: '17 5 * * 1'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  avatar:
+    runs-on: ubuntu-24.04
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@v7
+      - run: sudo apt-get update && sudo apt-get install -y libglib2.0-dev mold
+      - uses: dtolnay/rust-toolchain@stable
+      - uses: actions/setup-node@v7
+        with:
+          node-version: 24
+          cache: npm
+      - run: npm ci
+      - run: npx playwright install --with-deps chromium
+      - run: BROWSER_CHECK_FLOW=avatar BROWSER_CHECK_REQUIRE_MODEL=1 ./scripts/browser-check.sh

.github/workflows/check.yml

@@ -541,6 +541,7 @@ jobs:
           - name: Linux x86_64
             os: ubuntu-24.04
             target: x86_64-unknown-linux-gnu
+            # Keep this major.minor aligned with Cargo.toml's rust-version.
             image: rust:1.98.0-bullseye
             glibc: '2.31'
             glibcxx: '3.4.28'

.gitignore

@@ -49,12 +49,6 @@ DONT-MERGE/
 web/vendor/face-detection/*.binarypb
 web/vendor/face-detection/*.tflite
 web/vendor/face-detection/*.wasm
-# A retired local artifact, not a vendored file: the avatar model is fetched by
-# the browser now and nothing puts it here any more. Ignored so that a checkout
-# predating that change keeps its leftover copy invisible to git until
-# scripts/fetch-vendor.sh removes it. Delete this line once no such checkout is
-# plausible.
-web/vendor/avatar/jim.vrm
 # Same rule for Pyodide, pinned in web/vendor/pyodide/SHA256SUMS: 13.7 MB of
 # interpreter, glue and stdlib, none of which changes without a version bump.
 web/vendor/pyodide/*.js

Cargo.toml

@@ -2,6 +2,7 @@
 name = "codetrial"
 version = "0.1.0"
 edition = "2024"
+rust-version = "1.98"
 
 # The gate already runs `cargo clippy -- -D warnings`, and an editor running a
 # bare `cargo clippy` used to disagree with it: warnings there, a red gate here,

README.md

@@ -33,8 +33,10 @@ the agent receives structured code rather than editor screenshots. Python and
 JavaScript run locally; C, C++, and Java run through Compiler Explorer, so
 source code leaves the browser for those three.
 
-Camera and microphone are required to start. Audio and code snapshots stay in
-memory unless [recording](#recording) is enabled, which is off by default.
+Output confirmation and a microphone are required to start. A camera is also
+required when [recording](#recording) is enabled; otherwise a candidate can
+continue without one and the report records that condition. Audio and code
+snapshots stay in memory unless recording is enabled, which is off by default.
 Candidate video reaches Gemini only with
 `CODETRIAL_GEMINI_CANDIDATE_VIDEO_ENABLED=true`. Face-presence analysis runs in
 the browser and reports itself unavailable rather than guessing.
@@ -118,6 +120,12 @@ conceptual hints, and uses the latest test run in the final assessment. Voice
 responses stop when the candidate interrupts. Say "can I get a hint?" when
 needed; hints affect the communication score.
 
+The media preflight always requires confirmed output and a working microphone.
+For an interview that is not recorded, a candidate may continue without a
+camera when it is unavailable or declined; the signed integrity trail and the
+report record that neutral condition and why. A recorded interview still
+requires its camera before it can start.
+
 Candidates can present the interview in Google Meet by sharing the CodeTrial tab
 with tab audio enabled. Meet owns the shared tab after that, and face-presence
 analysis is disabled for the session. See the
@@ -160,6 +168,7 @@ The common ones:
 | `CODETRIAL_DURATION_MIN` | `45` | Interview length preselected in the lobby (10–90); see [interview length](docs/interview-length.md) |
 | `GEMINI_LIVE_MODEL` | `gemini-3.1-flash-live-preview` | Realtime interviewer model |
 | `GEMINI_REPORT_MODEL` | `gemini-3.1-flash-lite` | Report model |
+| `CODETRIAL_MAX_INTERIM_REVIEWS` | `12` | Quiet-pause report-model reviews per interview; `0` disables them and `72` is the maximum |
 | `CODETRIAL_GEMINI_CANDIDATE_VIDEO_ENABLED` | `false` | Forward candidate video to Gemini |
 | `CODETRIAL_COMPILER_EXPLORER_ENABLED` | `true` | Enable remote C, C++, and Java runs |
 | `CODETRIAL_MAX_CONCURRENT_INTERVIEWS` | `16` | Interviews one `web` process hosts agents for |
@@ -228,6 +237,7 @@ recorded rather than left implicit. See
 | [LiveKit troubleshooting](docs/livekit-connection-troubleshooting.md) | Telling four connection failures apart |
 | [Observable delivery policy](docs/observable-delivery-policy.md) | What a report may and may not assess |
 | [Interview contract versions](docs/interview-contract-versions.md) | The five versions every report carries |
+| [Adding a problem](docs/adding-a-problem.md) | Add an imported or original interview exercise |
 | [Rubric calibration](docs/rubric-calibration.md) | Calibration status of the framework scores |
 | [Provider cost and degradation](docs/provider-cost-and-degradation.md) | Gemini budgets, restarts, concurrency |
 

config/codetrial.env.example

@@ -5,6 +5,9 @@ LIVEKIT_API_SECRET=your_livekit_api_secret
 GOOGLE_API_KEY=your_google_ai_studio_api_key
 GEMINI_LIVE_MODEL=gemini-3.1-flash-live-preview
 GEMINI_REPORT_MODEL=gemini-3.1-flash-lite
+# Reviews quiet candidate stretches with the report model. 0 reserves that
+# quota for final reports; the default is 12 and the maximum is 72.
+# CODETRIAL_MAX_INTERIM_REVIEWS=12
 GEMINI_VOICE=Puck
 CODETRIAL_ROOM_PREFIX=interview
 CODETRIAL_DURATION_MIN=45
@@ -58,4 +61,3 @@ CODETRIAL_GEMINI_CANDIDATE_VIDEO_ENABLED=false
 # lifecycle rule on the staging bucket, which is the backstop for objects this
 # pipeline never got to delete, and the Shared Drive's own sharing policy, which
 # must allow an expiring reader permission on a file.
-

docs/adding-a-problem.md

@@ -0,0 +1,67 @@
+# Adding a problem
+
+`problem-bank/` is the source of truth. Add the same id, in bank order, to
+`problems.json`, `judges.json`, and `variants.json`; then regenerate the
+candidate pages, judges, private server data, page map, and cards:
+
+```bash
+python3 scripts/gen-problems.py
+python3 scripts/gen-problem-cards.py
+python3 scripts/gen-problems.py --check
+python3 scripts/gen-problem-cards.py --check
+```
+
+Every problem needs difficulty, topics, constraints, starter code, and private
+`summary`, `optimal`, and `pitfalls` fields. The generator keeps those fields
+on the server; never copy them into the browser payload.
+A judge has executable cases, and a variant supplies the candidate-facing
+scenario, renamed entry point or class name, examples, clarifications,
+follow-ups, and hints. Class judges omit C because its harness supports only
+function exercises.
+
+An imported LeetCode exercise keeps `title` and published `examples` in
+`problems.json`, belongs to `scripts/top-interview-150.json`, and is checked
+against the study plan. An original exercise sets `"origin": "original"`,
+has no published `title` or `examples`, and stays outside that plan. Its
+candidate page and `web/problem-pages.json` deliberately omit `source`.
+
+Run the focused bank checks before the full gate:
+
+```bash
+python3 -m unittest -v -k outside_the_plan tests/test_gen_problems.py
+cargo test --test agent problem_bank_matches_imported_golden
+python3 scripts/gen-problems.py --check
+./scripts/test.sh
+```
+
+When an original exercise intentionally changes the private-rubric golden,
+refresh it explicitly and review the resulting fixture:
+
+```bash
+UPDATE_PROBLEM_GOLDEN=1 cargo test --test agent problem_bank_matches_imported_golden
+```
+
+## Validation messages
+
+The generator stops at the first broken contract. These messages identify the
+source file to fix; do not edit generated output to silence them.
+
+- `missing or duplicate problem id`, `origin must be leetcode or original`,
+  `an original problem has no published title or examples`, and `an imported
+  problem needs its published title` come from `problems.json` identity and
+  origin checks.
+- `missing rubric`, `unknown difficulty`, and `topics must contain 1..8
+  values` (or `topics must be non-empty and unique`) name required problem
+  metadata.
+- `variants must list every problem once, in bank order`, `a variant has
+  exactly`, `a function problem declares a new entry`, and `a class problem
+  declares a new className` name the scenario record to repair.
+- `the brief never names`, `starter does not define`, `case labels repeat`,
+  `examples show published case`, and the source-title messages mean the
+  candidate-facing scenario, starter, or selected judge case leaks an invalid
+  name or does not match the executable contract.
+- `variant titles must be unique, and unique as page names` and `page names
+  must not equal a problem id` protect links and saved history.
+- `plan diverges from problem-bank` means an imported id and the study plan
+  differ. Mark a course-owned exercise `original` instead of adding it to the
+  plan.

docs/development.md

@@ -31,9 +31,8 @@ not packaged as widely, so the gate falls back to its container image, pinned to
 the version the workflow installs, whenever the binary is absent and a docker
 daemon answers. CI installs all three, so what is optional locally is enforced
 on a pull request — the summary exists so that a contributor knows which of the
-two they are looking at. One lane needs `javac` 16 or newer and is skipped on an
-older JDK; that is the Java class-harness fixture and nothing else depends on
-it.
+two they are looking at. One lane needs working `javac` and `java`; the Java
+class-harness fixture uses Java 8 syntax, and nothing else depends on it.
 
 Where the time goes, measured on this repo rather than guessed, because the
 answer is not the one a first look gives. Warm, the two Rust lanes are seconds:
@@ -127,6 +126,12 @@ python3 scripts/gen-problems.py
 python3 scripts/gen-problem-cards.py
 ```
 
+The generated Rust tables include scenario metadata, topics, private guides,
+and the private rubric in `src/agent/problem_rubrics.rs`. Do not edit a
+generated table directly. [Adding a problem](adding-a-problem.md) describes
+the source files, required checks, and the difference between an imported and
+an original exercise.
+
 `scripts/top-interview-150.json` records which problems the study plan asks
 for. Refresh it from LeetCode with:
 
@@ -135,8 +140,10 @@ python3 scripts/gen-problems.py --sync-study-plan
 ```
 
 The sync refuses to write when the plan and `problem-bank/` disagree, naming
-the problems each side is missing. Port those first. `--check` holds the
-committed manifest to the same rule, so drift fails the gate offline.
+the imported problems each side is missing. Port those first. Original
+exercises are deliberately outside the plan, so sync and drift checks leave
+them alone. `--check` holds the committed manifest to the same imported set,
+so drift fails the gate offline.
 
 Two commands cover the porting. `--plan-drift` asks LeetCode what changed
 without writing anything, and `--scaffold SLUG` prints the `problems.json` and

docs/integrity-evidence.md

@@ -8,6 +8,16 @@ it, and what CodeTrial declines to look for at all.
 Integrity features produce evidence for a person to review. Nothing here scores
 a candidate, and nothing here is a claim that anybody cheated.
 
+## Optional camera preflight
+
+Output confirmation and a working microphone are always required. On a server
+that does not record interviews, a candidate can continue without a camera
+when no camera is available, permission is denied, or they decline to use it.
+The browser records a signed `CAMERA_NOT_USED` event with one of `no_device`,
+`denied`, or `declined`; the report presents it as a neutral session condition.
+No face-presence worker runs in that case. Recording consent keeps the camera
+required, because the recording notice describes a video recording.
+
 ## Response windows
 
 The replay page lists a *response window* for each turn the interviewer took:

docs/interview-contract-versions.md

@@ -8,12 +8,15 @@ can select it.
 
 ## The active bundle
 
-Bundle 5: live prompt 2, report prompt 5, rubric 1, report schema 1.
+Bundle 8: live prompt 3, report prompt 8, rubric 1, report schema 2.
 
 | Bundle | Introduced |
 |---|---|
-| 5 | Each problem posed as an interview scenario rather than the published problem: the live prompt holds the scenario, its private contract and the clarifications to answer when asked, the follow-ups arrive with the evidence that completes the coding round, and the prompt never holds the source title, the hint ladder or a solution walkthrough; `log_hint` serves the authored hints one rung per request and holds the last until the candidate has stated an approach, meaning Algorithm evidence observed from what they said or Coding evidence, which needs code they wrote; a request answered with a withheld rung gives no clue and is not counted as a hint; Coding, Test and Optimizations evidence is refused until the editor holds code the candidate wrote beyond the starter; the report prompt gives the reviewer both the published problem and the scenario, with the reference notes, and forbids naming the published problem in anything written to the candidate |
-| 4 | The observable-delivery policy, made explicit in the report prompt and the server validator, with no change to the rubric or the public shape |
+| 8 | Reports keep the fixed mid-level hiring bar and state the optional level the candidate practiced for beside it. |
+| 7 | Candidate-authored test cases reach the live interviewer and report brief, while judge pass totals remain separate. |
+| 6 | The post-interview server stamp adds optional debrief, topics, and practice level fields. `tests/golden/report-schema.json` remains the model output shape only; server-stamped fields are versioned at the browser sanitizer. |
+| 5 | Each problem posed as an interview scenario rather than the published problem: the live prompt holds the scenario, its private contract and the clarifications to answer when asked, the follow-ups arrive with the evidence that completes the coding round, and the prompt never holds the source title, the hint ladder or a solution walkthrough; `log_hint` serves the authored hints one rung per request and holds the last until the candidate has stated an approach, meaning Algorithm evidence observed from what they said or Coding evidence, which needs code they wrote; a request answered with a withheld rung gives no clue and is not counted as a hint; Coding, Test and Optimizations evidence is refused until the editor holds code the candidate wrote beyond the starter; the report prompt gives the reviewer both the published problem and the scenario, with the reference notes, and forbids naming the published problem in anything written to the candidate. PR #38 covers `8eaaef0`, `26410f7`, `4e316f2`, `0dc521f`, `b72984a`, and `3027bb8`. |
+| 4 | The observable-delivery policy, made explicit in the report prompt and the server validator, with no change to the rubric or the public shape. Confirmed 2026-09-16: prompt revisions `14ad45d`, `a3872ce`, `a43eb29`, `2655f6a`, `8d2f3df`, `dbc2060`, and `5dca169` shipped under this bundle. |
 | 3 | Framework phase scores kept explicitly formative, and prohibited from mechanical use in a hiring decision while calibration remains incomplete |
 | 2 | Provider-enforced structured report output and strict validation, with no change to rubric semantics or the public schema |
 
@@ -22,17 +25,20 @@ Bundle 5: live prompt 2, report prompt 5, rubric 1, report schema 1.
 A change to prompt behavior, score anchors, or report shape updates the relevant
 component and creates a new bundle version in the same change. Rust and browser
 constants, prompt and report goldens, migration fixtures, and replay fixtures
-move together. A released bundle number is never reused for different behavior.
+move together. A prompt-only change bumps its prompt constant and the bundle in
+both `src/agent.rs` and `ACTIVE_CONTRACT`, refreshes the prompt golden, and adds
+a row here; it needs no browser compatibility-list edit. A released bundle
+number is never reused for different behavior.
 
 ## Compatibility rules
 
 - Reports without `interviewContract` predate this contract. They stay readable
   and are labeled `legacy/unversioned`; they are never assigned the current
   rubric.
-- The browser scores the active bundle and bundle 4, which shares its rubric
-  and report schema and differs only in the prompts that wrote the report
-  (`SCORABLE_CONTRACTS` in `web/lib.js`). A report keeps the bundle it claims.
-  A bump that changes the rubric or the schema does not join that list.
+- The browser scores a report when its rubric is active, its schema is in
+  `SCORABLE_SCHEMAS`, its bundle is at least 4 and no newer than active, and
+  neither prompt version is newer than active. A report keeps the bundle it
+  claims. A rubric or schema change is not compatible until this rule says so.
 - The browser renders the active report schema normally. An older renderer may
   ignore additive fields only after the bundle and schema migration explicitly
   permits it.
@@ -47,7 +53,8 @@ move together. A released bundle number is never reused for different behavior.
 
 ## Release checklist
 
-Update the active server bundle; add the browser migration; refresh the prompt
-and report goldens; cover successful, incomplete, legacy, malformed, and future
-reports; verify HTML, Markdown, history and progress, and replay provenance;
-then run the complete local test suite.
+Update the active server bundle and `ACTIVE_CONTRACT`; add the browser
+migration; refresh the prompt and report goldens; add this bundle's table row;
+cover successful, incomplete, legacy, malformed, and future reports; verify
+HTML, Markdown, history and progress, and replay provenance; then run the
+complete local test suite.

docs/livekit-connection-troubleshooting.md

@@ -65,8 +65,7 @@ rest.
 - `tests/web.rs`: `responses_carry_baseline_security_headers`,
   `production_policy_names_no_loopback_origins`, and
   `the_recording_template_is_reachable_under_a_policy_that_permits_its_room`
-  all pass. Each asserts with `contains`, and this change only adds entries, so
-  no existing assertion moved.
+  all pass.
 - `cargo fmt --check` and `cargo clippy --lib` clean.
 - Confirmed against the live symptom: the CSP refusal disappeared from the
   browser console and the request reached LiveKit.

docs/provider-cost-and-degradation.md

@@ -28,6 +28,15 @@ network request, so no future loop change can exceed the budget by accident.
 Authentication failures, bad models, malformed responses, and other permanent
 failures get no transport retry.
 
+Quiet-pause interim reviews use that same report model and quota. A review is
+eligible after 8 seconds of candidate quiet and 75 seconds from interview start,
+no more often than every 75 seconds, and only after four new candidate turns.
+Before the cap, a 90-minute
+interview can make at most 72 such calls; shorter interviews cannot exceed that
+rate. `CODETRIAL_MAX_INTERIM_REVIEWS` defaults to 12, accepts `0` to disable
+the reviews, and is capped at 72. This is a quota guard, not a completeness
+limit: the final report still receives the complete transcript and editor state.
+
 ## What bounds concurrency
 
 The server admits at most `CODETRIAL_MAX_CONCURRENT_INTERVIEWS` live local

docs/providers.md

@@ -4,6 +4,12 @@ Provider pooling spreads rooms over more than one LiveKit project, so
 concurrent interviews draw on several projects' quotas instead of exhausting
 one. A single-project deployment needs none of this and is unaffected by it.
 
+Before a room token is minted, the server uses cached periodic probes and probes
+only providers it tries until one is available. A fresh 429 excludes exhausted
+connection minutes and a fresh 401/403 excludes a refused credential. The log
+names the project and status without printing credentials. Replacing a configured
+credential takes effect after restarting the server.
+
 ## Adding a project
 
 Add one `config/codetrial.env.<id>` per extra project, each with its own

problem-bank/judge-case-gaps.txt

@@ -0,0 +1,3 @@
+# GAPS: 0
+# Every line names a judge that still needs five cases and a domain-valid
+# boundary case. Remove a line when its authored cases pass the validator.