Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,11 @@ Thumbs.db
# typescript
*.tsbuildinfo
next-env.d.ts

#ai agent
.claude/
.cursor/
.codex/

# MCP tool artefacts — console logs and page snapshots from browser automation
.playwright-mcp/
22 changes: 13 additions & 9 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,19 @@ than none. Whoever asked you to make this change will tell you your scope.

## `TODO(name), Day n:` markers are assignments

Every stub names the person who owned it and the day it was due during the original build. Implement
the ones you were asked to implement; leaving the rest alone is correct behaviour, not incompleteness.
Deleting a marker while implementing around it destroys the only record of who owes what.

## Some tests are red on purpose

`compress-dom`, `bisect`, and `evaluatePredicate` ship with failing tests. They were written first, as
specifications. **Never** make them pass by weakening an assertion, adding `.skip`, or deleting a case:
that converts a specification into a lie, silently.
There are none left — every stub from the original build has been implemented. The convention is
documented here because the rule outlives the markers: a marker names the person who owned the work and
the day it was due, so **deleting one while implementing around it destroys the only record of who owes
what.** If you add a marker, name yourself in it. If you find one, either implement it or leave it
exactly where it is.

## The tests are green, and two suites must stay honest

All 285 tests pass. That is worth stating because of how some of them got there: the `compress-dom`,
`bisect` and `evaluatePredicate` suites were written first, as specifications, and were red for as long
as it took the implementations to satisfy them. **Never** make a test in those suites pass by weakening
an assertion, adding `.skip`, or deleting a case: that converts a specification into a lie, silently. A
red test there means the implementation is wrong.

`no-eval.test.ts` greps the source and fails if `eval(` or `new Function` appears anywhere. It is a
security boundary, not a lint rule.
Expand Down
14 changes: 14 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,20 @@ order of fidelity:
You don't need a real agent for most work. You do need one before claiming a tool works — see
[Testing a tool](#testing-a-tool).

`traces/` and `bugbait/` install separately — neither is a workspace member, and there is no lockfile at
the repo root. `npm ci` works in both, and `npm audit` reports zero vulnerabilities in both; if either
stops being true, that's a change to fix rather than to note. Two entries in those manifests exist for
reasons JSON has no room to state:

- **`overrides: { "postcss": "^8.5.26" }`** in both. The advisory is against a copy of postcss nested
under `next`, not the one either app depends on directly, so bumping the direct dependency does
nothing. The alternative `npm audit fix --force` offers is a Next major, which is a much larger change
than the one-line fix the advisory actually needs.
- **`vitest` on 4.x**, not the 2.x this was built against. Two advisories, one of them critical, are
fixed only in 4.x. The suite needed no changes — but the config had to become `vitest.config.mts`,
since `.ts` in a package without `"type": "module"` is loaded as CommonJS and Vite's native config
loader is about to stop tolerating that.

---

## Three rules that aren't negotiable
Expand Down
32 changes: 20 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,11 @@ Agent: find_element({ text: "Pay" })
Agent: bisect({ selector: "#pay-submit",
predicate: { kind: "propertyEquals", property: "disabled", equals: true },
from: 0, to: 47200 })
→ firstTrue: 28_450ms (6 iterations, 1.4s, ±250ms)
→ firstTrue: 28_577ms (10 iterations, 1.4s, ±250ms)

← the playhead visibly jumps six times while this runs
← the playhead visibly jumps ten times while this runs

Agent: read_dom_at({ timestamp: 28450, scope: "#checkout" })
Agent: read_dom_at({ timestamp: 28577, scope: "#checkout" })
→ form#checkout [visible]
input[name=email] value="ana@..." [valid]
select[name=province] [empty options: 0]
Expand Down Expand Up @@ -97,12 +97,12 @@ to an agent through `document.modelContext.registerTool()`. Lots of things *coul
that would work just as well as a REST API. Traces is not one of them, and the reasons are worth
stating precisely:

**1. The answer doesn't exist until the page computes it.** `read_dom_at(28450)` isn't a lookup. The
**1. The answer doesn't exist until the page computes it.** `read_dom_at(28577)` isn't a lookup. The
page must replay the mutation stream to that instant and reconstruct the DOM. A server holding the
same recording file cannot answer the question without becoming a replay engine itself.

**2. `bisect` doesn't fetch — it *programs* the page.** The agent sends a predicate and the page runs
a binary search across the replay timeline, six probes deep. The agent isn't retrieving data; it's
a binary search across the replay timeline, ten probes deep. The agent isn't retrieving data; it's
handing the page an algorithm to execute over time. There is no request/response shape for that.

**3. The human is inside the agent's loop, not watching it.** `ask_human_visual` doesn't resolve until
Expand Down Expand Up @@ -140,13 +140,16 @@ bisect({
predicate: { kind: "optionCount", equals: 0 },
from: 0, to: 47200
})
// → { firstTrue: 12_600, iterations: 6, elapsedMs: 1_180, precisionMs: 250,
// trace: [ { atMs: 23600, result: true }, { atMs: 11800, result: false }, ... ] }
// → { firstTrue: 12_721, lastFalse: 12_537, iterations: 10, elapsedMs: 1_180, precisionMs: 250,
// trace: [ { atMs: 0, result: false }, { atMs: 47200, result: true },
// { atMs: 23600, result: true }, { atMs: 11800, result: false }, ... ] }
```

Six probes instead of forty-seven calls. The `trace` isn't decoration either — the UI animates it, so
a human watches the playhead jump six times and *sees* the agent's reasoning as motion on the
timeline.
Ten probes instead of forty-seven calls: the two boundary probes that bracket the window, then eight
halvings to close it to 250 ms. The count is the same wherever the transition sits — that is what a
binary search buys. (`elapsedMs` is the one number above that depends on the machine; the probe count
does not.) The `trace` isn't decoration either — the UI animates it, so a human watches the playhead
jump ten times and *sees* the agent's reasoning as motion on the timeline.

Predicates are a **closed set of seven structured shapes**, never a string, and never evaluated. That's
a security property (see [threat model](docs/threat-model.md)) but also a usability one: models fill in
Expand Down Expand Up @@ -368,13 +371,18 @@ Full analysis, including what is deliberately **out of scope**:
| Dependency | Licence | Why |
|---|---|---|
| [rrweb](https://github.com/rrweb-io/rrweb) + rrweb-player | MIT | records and replays DOM mutations; the only mature option that reconstructs a real DOM rather than pixels |
| [Next.js](https://nextjs.org) | MIT | static export, trivial Vercel deploy, response headers for the origin trial token |
| [Next.js](https://nextjs.org) | MIT | prerendered pages, trivial Vercel deploy, response headers for the origin trial token |
| TypeScript | Apache-2.0 | strict mode; tool schemas and domain types stay honest |
| [Tailwind CSS](https://tailwindcss.com) | MIT | dense instrument UI without a component library |
| [zustand](https://github.com/pmndrs/zustand) | MIT | state readable and writable *from outside React* — see below |
| [IBM Plex Sans + Plex Mono](https://github.com/IBM/plex) | OFL-1.1 | one superfamily, so a mono timestamp and a sans label share a line without a step in it — `src/app/fonts.ts` |
| [vitest](https://vitest.dev) | MIT | the pure modules are tested; the UI is not |

Every dependency is permissively licensed and compatible with MIT redistribution. We deliberately
Every dependency is permissively licensed and compatible with MIT redistribution. The fonts are the one
thing not in `package.json`: `next/font` fetches them during `next build` and serves them from this
origin, so the deployed bundle carries the woff2 files under OFL-1.1 — the licence text and the reserved
font names live with [the upstream project](https://github.com/IBM/plex/blob/master/LICENSE.txt), and
nothing here modifies or renames a face. We deliberately
avoided three tempting libraries: **DuckDB-Wasm** (it's the flagship example in the challenge's own
materials — using it would weaken the originality of the entry), **ffmpeg.wasm** (LGPL/GPL build
ambiguity we didn't want in an MIT repo), and **HyperFormula** (AGPL-3.0).
Expand Down
6 changes: 6 additions & 0 deletions bugbait/next.config.mjs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import { fileURLToPath } from 'node:url'

/**
* bugbait — the app whose bugs become the recordings Traces investigates.
*
Expand All @@ -8,6 +10,10 @@
*/
const nextConfig = {
reactStrictMode: true,

// Same reason as in traces/next.config.mjs: this folder installs on its own, so the tracing root is
// this folder, not whatever ancestor happens to contain a lockfile.
outputFileTracingRoot: fileURLToPath(new URL('.', import.meta.url)),
}

export default nextConfig
67 changes: 25 additions & 42 deletions bugbait/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion bugbait/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,11 @@
"eslint": "^9.0.0",
"eslint-config-next": "^15.5.0",
"playwright-core": "^1.62.1",
"postcss": "^8.4.47",
"postcss": "^8.5.26",
"tailwindcss": "^3.4.14",
"typescript": "^5.6.0"
},
"overrides": {
"postcss": "^8.5.26"
}
}
2 changes: 0 additions & 2 deletions bugbait/scripts/record-fixtures.mjs
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
/**
* Regenerates the sample recordings in `traces/public/recordings/` by driving a real browser.
*
* Owner: Vicko.
*
* **These are scripted fixtures, not human sessions,** and the difference is worth being precise about.
* Everything in the resulting file is real: a real Chrome, the real bugbait build, the real
* `rrweb.record()`, real HTTP to the real stub endpoints, real layout and real CSS. What is synthetic is
Expand Down
2 changes: 0 additions & 2 deletions bugbait/src/app/api/cities/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ import { LATENCY_MS } from '@/lib/bugs'
/**
* `GET /api/cities` — always succeeds, always returns cities.
*
* Owner: Vicko.
*
* There is no scenario branch here, and that is the interesting part. In the `race` scenario this
* endpoint behaves perfectly: 200, a full list, in under a second. The bug is entirely on the client,
* which reads the list once during its first render — before this response exists — and never looks
Expand Down
2 changes: 0 additions & 2 deletions bugbait/src/app/api/provinces/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ import { isBug, LATENCY_MS, SCENARIO_HEADER } from '@/lib/bugs'
/**
* `GET /api/provinces` — the request at the centre of the primary demo bug.
*
* Owner: Vicko.
*
* When the `empty-province` scenario is armed this returns **200 with `[]`**, and the distinction from
* a 500 is the whole point. An error status shows up red in a network tab and the investigation takes
* four seconds; a success that happens to be empty looks like nothing at all, and only the *shape* of
Expand Down
4 changes: 1 addition & 3 deletions bugbait/src/app/checkout/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@ import { RecorderPanel } from '@/components/recorder-panel'
/**
* The checkout form. Every bug lives here.
*
* Owner: Vicko.
*
* The form has to be genuinely ordinary — name, address, province, city, postcode, card, pay — because
* the investigation is only interesting if the page looks like something that would ship. A page with
* one dropdown and a broken button demonstrates nothing; the agent has to actually find the failing
Expand All @@ -34,7 +32,7 @@ import { RecorderPanel } from '@/components/recorder-panel'
* rendered markup. The recording captures the DOM; anything written there hands the agent the
* answer and makes the demo a re-enactment.
*
* Day 5 (vicko) — implemented. Three notes on how each bug is kept invisible in the finished DOM,
* Three notes on how each bug is kept invisible in the finished DOM,
* because that property is the argument of the whole project and the easiest thing to break by accident:
*
* - **empty-province.** `/api/provinces` is requested the moment this page mounts and resolves 1.5s
Expand Down
Loading
Loading