Skip to content

Commit f3bc6d2

Browse files
committed
chore: rename llmock→aimock in comments, strings, URLs, and docs
1 parent 5f8a0d1 commit f3bc6d2

28 files changed

+116
-115
lines changed

DRIFT.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,19 @@
11
# Live API Drift Detection
22

3-
llmock produces responses shaped like real LLM APIs. Providers change their APIs over time. **Drift** means the mock no longer matches reality — your tests pass against llmock but break against the real API.
3+
aimock produces responses shaped like real LLM APIs. Providers change their APIs over time. **Drift** means the mock no longer matches reality — your tests pass against aimock but break against the real API.
44

55
## Three-Layer Approach
66

77
Drift detection compares three independent sources to triangulate the cause of any mismatch:
88

9-
| SDK types = Real API? | Real API = llmock? | Diagnosis |
9+
| SDK types = Real API? | Real API = aimock? | Diagnosis |
1010
| --------------------- | ------------------ | -------------------------------------------------------------------- |
11-
| Yes | No | **llmock drift** — response builders need updating |
11+
| Yes | No | **aimock drift** — response builders need updating |
1212
| No | No | **Provider changed before SDK update** — flag, wait for SDK catch-up |
1313
| Yes | Yes | **No drift** — all clear |
1414
| No | Yes | **SDK drift** — provider deprecated something SDK still references |
1515

16-
Two-way comparison (mock vs real) can't distinguish between "we need to fix llmock" and "the SDK hasn't caught up yet." Three-way comparison can.
16+
Two-way comparison (mock vs real) can't distinguish between "we need to fix aimock" and "the SDK hasn't caught up yet." Three-way comparison can.
1717

1818
## Running Drift Tests
1919

@@ -40,9 +40,9 @@ Each provider's tests skip independently if its key is not set. You can run drif
4040

4141
### Severity levels
4242

43-
- **critical** — Test fails. llmock produces a different shape than the real API for a field that both the SDK and real API agree on. This means llmock needs an update.
44-
- **warning** — Test passes (unless `STRICT_DRIFT=1`). The real API has a field that neither the SDK nor llmock knows about, or the SDK and real API disagree. Usually means a provider added something new.
45-
- **info** — Always passes. Known intentional differences (usage fields are always zero, optional fields llmock omits, etc.).
43+
- **critical** — Test fails. aimock produces a different shape than the real API for a field that both the SDK and real API agree on. This means aimock needs an update.
44+
- **warning** — Test passes (unless `STRICT_DRIFT=1`). The real API has a field that neither the SDK nor aimock knows about, or the SDK and real API disagree. Usually means a provider added something new.
45+
- **info** — Always passes. Known intentional differences (usage fields are always zero, optional fields aimock omits, etc.).
4646

4747
### Example report output
4848

@@ -86,7 +86,7 @@ When a `critical` drift is detected:
8686

8787
## Model Deprecation
8888

89-
The `models.drift.ts` test scrapes model names referenced in llmock's test files, README, and fixtures, then checks each provider's model listing API to verify they still exist.
89+
The `models.drift.ts` test scrapes model names referenced in aimock's test files, README, and fixtures, then checks each provider's model listing API to verify they still exist.
9090

9191
When a model is deprecated:
9292

@@ -106,7 +106,7 @@ When a model is deprecated:
106106

107107
## WebSocket Drift Coverage
108108

109-
In addition to the 19 existing drift tests (16 HTTP response-shape + 3 model deprecation), WebSocket drift tests cover llmock's WS protocols (4 verified + 2 canary = 6 WS tests):
109+
In addition to the 19 existing drift tests (16 HTTP response-shape + 3 model deprecation), WebSocket drift tests cover aimock's WS protocols (4 verified + 2 canary = 6 WS tests):
110110

111111
| Protocol | Text | Tool Call | Real Endpoint | Status |
112112
| ------------------- | ---- | --------- | ------------------------------------------------------------------- | ---------- |
@@ -118,13 +118,13 @@ In addition to the 19 existing drift tests (16 HTTP response-shape + 3 model dep
118118

119119
**Auth**: Uses the same `OPENAI_API_KEY` and `GOOGLE_API_KEY` environment variables as HTTP tests. No new secrets needed.
120120

121-
**How it works**: A TLS WebSocket client (`ws-providers.ts`) connects to real provider endpoints using `node:tls` with RFC 6455 framing. Each protocol function handles the setup sequence (e.g., Realtime session negotiation, Gemini Live setup/setupComplete) and collects messages until a terminal event. The mock side uses the existing `ws-test-client.ts` plaintext client against the local llmock server.
121+
**How it works**: A TLS WebSocket client (`ws-providers.ts`) connects to real provider endpoints using `node:tls` with RFC 6455 framing. Each protocol function handles the setup sequence (e.g., Realtime session negotiation, Gemini Live setup/setupComplete) and collects messages until a terminal event. The mock side uses the existing `ws-test-client.ts` plaintext client against the local aimock server.
122122

123123
### Gemini Live: unverified
124124

125-
llmock's Gemini Live handler implements the text-based `BidiGenerateContent` protocol as documented in Google's [Live API reference](https://ai.google.dev/api/live)`setup`/`setupComplete` handshake, `clientContent` with turns, `serverContent` with `modelTurn.parts[].text`, and `toolCall` responses. The protocol format is correct per the docs.
125+
aimock's Gemini Live handler implements the text-based `BidiGenerateContent` protocol as documented in Google's [Live API reference](https://ai.google.dev/api/live)`setup`/`setupComplete` handshake, `clientContent` with turns, `serverContent` with `modelTurn.parts[].text`, and `toolCall` responses. The protocol format is correct per the docs.
126126

127-
However, as of March 2026, the only models that support `bidiGenerateContent` are native-audio models (`gemini-2.5-flash-native-audio-*`), which reject text-only requests. No text-capable model exists for this endpoint yet, so we cannot triangulate llmock's output against a real API response.
127+
However, as of March 2026, the only models that support `bidiGenerateContent` are native-audio models (`gemini-2.5-flash-native-audio-*`), which reject text-only requests. No text-capable model exists for this endpoint yet, so we cannot triangulate aimock's output against a real API response.
128128

129129
A canary test (`ws-gemini-live.drift.ts`) queries the Gemini model listing API on each drift run and checks for a non-audio model that supports `bidiGenerateContent`. When Google ships one, the canary will flag it and the full drift tests can be enabled.
130130

packages/aimock-pytest/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# aimock-pytest
22

3-
pytest fixtures for [aimock](https://github.com/CopilotKit/llmock) — mock LLM APIs, MCP tools, A2A agents, vector databases, and more.
3+
pytest fixtures for [aimock](https://github.com/CopilotKit/aimock) — mock LLM APIs, MCP tools, A2A agents, vector databases, and more.
44

55
## Install
66

scripts/update-competitive-matrix.ts

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -172,7 +172,7 @@ const DOCS_PATH = resolve(import.meta.dirname ?? __dirname, "../docs/index.html"
172172
const GITHUB_TOKEN = process.env.GITHUB_TOKEN ?? "";
173173
const HEADERS: Record<string, string> = {
174174
Accept: "application/vnd.github.v3+json",
175-
"User-Agent": "llmock-competitive-matrix-updater",
175+
"User-Agent": "aimock-competitive-matrix-updater",
176176
...(GITHUB_TOKEN ? { Authorization: `Bearer ${GITHUB_TOKEN}` } : {}),
177177
};
178178

@@ -403,7 +403,7 @@ function parseCurrentMatrix(html: string): {
403403
while ((m = thRegex.exec(tableHtml)) !== null) {
404404
headers.push(m[1].trim());
405405
}
406-
// headers[0] = "llmock", headers[1] = "MSW", headers[2..] = competitors
406+
// headers[0] = "aimock", headers[1] = "MSW", headers[2..] = competitors
407407

408408
// Extract rows
409409
const rows = new Map<string, Map<string, string>>();
@@ -422,7 +422,7 @@ function parseCurrentMatrix(html: string): {
422422

423423
const rowLabel = tds[0];
424424
const rowMap = new Map<string, string>();
425-
// tds[1] = llmock, tds[2] = MSW, tds[3..5] = competitors
425+
// tds[1] = aimock, tds[2] = MSW, tds[3..5] = competitors
426426
for (let i = 1; i < tds.length && i - 1 < headers.length; i++) {
427427
rowMap.set(headers[i - 1], tds[i]);
428428
}
@@ -433,7 +433,7 @@ function parseCurrentMatrix(html: string): {
433433
}
434434

435435
/**
436-
* Updates only competitor cells (not llmock or MSW) where:
436+
* Updates only competitor cells (not aimock or MSW) where:
437437
* - The current value indicates "No" (class="no">No</td>)
438438
* - The feature was detected in the competitor's README
439439
*
@@ -495,9 +495,9 @@ function applyChanges(html: string, changes: DetectedChange[]): string {
495495
while ((m = thRegex.exec(theadMatch[1])) !== null) {
496496
headers.push(m[1].trim());
497497
}
498-
// Column indices: "Capability" = 0 (no header link), then llmock=1, MSW=2,
498+
// Column indices: "Capability" = 0 (no header link), then aimock=1, MSW=2,
499499
// VidaiMock=3, mock-llm=4, piyook/llm-mock=5
500-
// In the <td> array: index 0 = capability, 1 = llmock, 2 = MSW, 3+ = competitors
500+
// In the <td> array: index 0 = capability, 1 = aimock, 2 = MSW, 3+ = competitors
501501
const compColumnIndex = (name: string): number => {
502502
const idx = headers.indexOf(name);
503503
return idx === -1 ? -1 : idx + 1; // +1 because first <td> is the row label

skills/write-fixtures/SKILL.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ The embedding vector is returned for each input in the request. If no embedding
7070

7171
### Image
7272

73+
<!-- prettier-ignore -->
7374
```typescript
7475
// Single image
7576
{

src/__tests__/aimock-cli.test.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -456,7 +456,7 @@ describe("runAimockCli: onReady and shutdown", () => {
456456
});
457457
});
458458

459-
it("shutdown calls llmock.stop()", async () => {
459+
it("shutdown calls aimock.stop()", async () => {
460460
const mockStop = vi.fn().mockResolvedValue(undefined);
461461
const startFromConfigFn = vi.fn().mockResolvedValue({
462462
llmock: { stop: mockStop },
@@ -496,7 +496,7 @@ describe("runAimockCli: onReady and shutdown", () => {
496496
});
497497
});
498498

499-
it("shutdown logs error and exits 1 when llmock.stop() rejects", async () => {
499+
it("shutdown logs error and exits 1 when aimock.stop() rejects", async () => {
500500
const mockStop = vi.fn().mockRejectedValue(new Error("close ENOTCONN"));
501501
const startFromConfigFn = vi.fn().mockResolvedValue({
502502
llmock: { stop: mockStop },

src/__tests__/cli.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ function writeFixture(dir: string, name: string): string {
9494
describe.skipIf(!CLI_AVAILABLE)("CLI: --help", () => {
9595
it("prints usage text and exits with code 0", async () => {
9696
const { stdout, code } = await runCli(["--help"]);
97-
expect(stdout).toContain("Usage: llmock");
97+
expect(stdout).toContain("Usage: aimock");
9898
expect(stdout).toContain("--port");
9999
expect(stdout).toContain("--fixtures");
100100
expect(code).toBe(0);

src/__tests__/competitive-matrix-summary.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ function writeSummary(summaryPath: string, changes: DetectedChange[]): void {
7373
// ── Helpers ─────────────────────────────────────────────────────────────────
7474

7575
function tmpPath(suffix: string): string {
76-
return join(tmpdir(), `llmock-cm-test-${suffix}-${Date.now()}.md`);
76+
return join(tmpdir(), `aimock-cm-test-${suffix}-${Date.now()}.md`);
7777
}
7878

7979
const tempFiles: string[] = [];

src/__tests__/config-loader.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -772,7 +772,7 @@ describe("startFromConfig", () => {
772772
expect(card.name).toBe("no-events-agent");
773773
});
774774

775-
it("with record config, llmock receives record settings", async () => {
775+
it("with record config, aimock receives record settings", async () => {
776776
const config: AimockConfig = {
777777
llm: {
778778
record: {

src/__tests__/drift/anthropic.drift.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
/**
22
* Anthropic Claude Messages API drift tests.
33
*
4-
* Three-way comparison: SDK types × real API × llmock output.
4+
* Three-way comparison: SDK types × real API × aimock output.
55
*/
66

77
import { describe, it, expect, beforeAll, afterAll } from "vitest";

src/__tests__/drift/bedrock-stream.drift.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
/**
22
* AWS Bedrock drift tests.
33
*
4-
* Three-way comparison: SDK types x real API x llmock output.
4+
* Three-way comparison: SDK types x real API x aimock output.
55
* Covers invoke-with-response-stream and converse endpoints.
66
*/
77

0 commit comments

Comments
 (0)