Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,7 @@ trim_trailing_whitespace = false

[Makefile]
indent_style = tab

# Oracle files are RFC 8785 canonical JSON, compared byte for byte: no final newline.
[spec/v1/examples/**/{expected/*.json,*.diagnostics.json}]
insert_final_newline = false
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ npm run verify # lint, format, typecheck, ADR contract, tests + coverage
`npm run lint:adrs` alone runs the decision-record contract, and `npm test`
runs the suite without enforcing coverage. `npm run test:coverage` (part of
`npm run verify`) enforces the ratchet in `vitest.config.ts`: statements
98.32%, branches 92.82%, functions 100%, lines 98.19%.
98.37%, branches 93.1%, functions 100%, lines 98.25%.

## Conventions

Expand Down
13 changes: 13 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,14 @@ target metamodel holding both, because a model-to-text template reads one model.
resolution: the consumer, the provider Application and Surface, the address the
consumer is given, and the policy peers that allow the connection. It is the
part of resolution both implementations must agree on before either renders.
The file is an object with one `applications` entry per Application, each an
`id` and its `edges`; an Application with no dependency carries an empty list.
The edge's own fields are fixed by the first case that has one.

**The canonical writers** are `src/infrastructure/canonical-json.ts` and
`emf/parity`'s `CanonicalJson`, held to the same cases. An oracle file is exactly
its canonical text, with no final newline, and a test fails any committed oracle
that is not byte-identical to its own canonicalisation.

**Canonical JSON** is RFC 8785 (JSON Canonicalization Scheme): keys sorted,
numbers in their shortest form, no insignificant whitespace. An absent
Expand Down Expand Up @@ -289,6 +297,11 @@ both implementations. The row names the TypeScript test; the model-driven witnes
for the same id is listed inside `emf/`, and `emf/`'s own gate fails when a model
row has no witness there.

**Whichever implementation lands a case first commits its reviewed oracle,
and the other matches it.** The first oracles are written by hand: `minimal`'s
parsed intent and dependency edges. The first `resolved.json` lands with the
Resolved Deployment metamodel that gives it a shape (#42).

An oracle file changes in the pull request that changes the behaviour it
records, and both implementations go red together until both are fixed. CI
never regenerates an oracle file from either implementation: a tool may write a
Expand Down
4 changes: 3 additions & 1 deletion docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ test file and holds at least one test; ids are unique; the count this
document states matches the number of rows it holds; and every id cited
anywhere in the tracked tree resolves to a row here.

This ledger holds **18** rows. The compiler's behaviours join it as they land.
This ledger holds **20** rows. The compiler's behaviours join it as they land.

| id | a contributor or a consumer can rely on | proved by |
|---|---|---|
Expand All @@ -41,3 +41,5 @@ This ledger holds **18** rows. The compiler's behaviours join it as they land.
| REQ-016 | A code scanning finding of any severity fails `Pipeline Complete`, so it blocks the merge rather than only landing in the Security tab | [test/pipeline-wiring.test.ts](../test/pipeline-wiring.test.ts) |
| REQ-017 | Every error code the specification defines is exercised by a test, or pending on the ticket that will exercise it, and no code the specification does not define is used in the tree | [test/codes-lint.test.ts](../test/codes-lint.test.ts) |
| REQ-018 | The compiler's inner rings cannot read the environment, the clock, randomness, a child process or the filesystem synchronously, and only `src/cli/boundary.ts` exits the process or writes output | [test/seams.test.ts](../test/seams.test.ts) |
| REQ-019 | Every committed oracle file is byte-identical to its own RFC 8785 canonicalisation, so a hand edit cannot leave one in a form the other implementation would not produce | [test/oracles.test.ts](../test/oracles.test.ts) |
| REQ-020 | The production implementation's canonical JSON writer sorts keys by UTF-16 code units, formats numbers as ECMAScript does, and refuses null, non-finite numbers, non-JSON values and lone surrogates, on the same cases as the model-driven writer | [test/canonical-json.test.ts](../test/canonical-json.test.ts) |
1 change: 1 addition & 0 deletions spec/v1/examples/minimal/expected/dependencies.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"applications":[{"edges":[],"id":"notes"}]}
1 change: 1 addition & 0 deletions spec/v1/examples/minimal/expected/intent.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"apiVersion":"intent.jorisjonkers.dev/v1","applications":[{"exposure":[{"audience":"anonymous","contentPolicy":"strict","host":"notes.jorisjonkers.dev","name":"public","routes":[{"match":"prefix","path":"/","process":"notes-api","surface":"http"}]}],"id":"notes","observability":{"alertClass":"business-hours","scrape":{"path":"/metrics","process":"notes-api","surface":"http"}},"processes":[{"cutover":"rolling","image":"notes-api","lifecycle":"application","name":"notes-api","placement":{"cpu":"50m","memory":"256Mi"},"probes":{"liveness":{"path":"/healthz/live","port":8080},"readiness":{"path":"/healthz/ready","port":8080}},"provides":{"http":8080},"runtime":"node","startupBudget":"20s"}]}],"kind":"Project","owner":"joris","project":"notes","schemaVersion":"1.0.0"}
1 change: 1 addition & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export { canonicalJson } from "./infrastructure/canonical-json.ts";
61 changes: 61 additions & 0 deletions src/infrastructure/canonical-json.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
export function canonicalJson(value: unknown): string {
return write(value, "");
}

function write(value: unknown, pointer: string): string {
if (value === null || value === undefined)
throw new TypeError(
`${String(value)} at ${pointer}: an absent optional field is absent, never null`,
);
if (Array.isArray(value))
return `[${value.map((item: unknown, i) => write(item, `${pointer}/${String(i)}`)).join(",")}]`;
switch (typeof value) {
case "string":
return writeString(value, pointer);
case "boolean":
return String(value);
case "number":
if (!Number.isFinite(value))
throw new TypeError(
`${String(value)} at ${pointer} is not a JSON number`,
);
return JSON.stringify(value);
case "object":
if (Object.getPrototypeOf(value) === Object.prototype)
return writeObject(value as Record<string, unknown>, pointer);
break;
}
throw new TypeError(`${describe(value)} at ${pointer} is not a JSON value`);
}

function writeObject(object: Record<string, unknown>, pointer: string): string {
const members = Object.keys(object)
.sort()
.map(
(key) =>
`${writeString(key, pointer)}:${write(object[key], `${pointer}/${escapePointer(key)}`)}`,
);
return `{${members.join(",")}}`;
}

function writeString(text: string, pointer: string): string {
const lone =
/[\uD800-\uDBFF](?![\uDC00-\uDFFF])|(?<![\uD800-\uDBFF])[\uDC00-\uDFFF]/.exec(
text,
);
if (lone !== null)
throw new TypeError(
`string at ${pointer} holds a lone surrogate at index ${String(lone.index)}`,
);
return JSON.stringify(text);
}

function escapePointer(segment: string): string {
return segment.replaceAll("~", "~0").replaceAll("/", "~1");
}

function describe(value: unknown): string {
return typeof value === "object"
? (value as object).constructor.name
: typeof value;
}
135 changes: 135 additions & 0 deletions test/canonical-json.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
// REQ-020 (docs/requirements.md). The same cases as emf/parity CanonicalJsonTest.
import { describe, expect, it } from "vitest";
import { canonicalJson } from "../src/index.ts";

const u = (...codes: number[]): string => String.fromCharCode(...codes);
const EURO = u(0x20ac);
const DALET = u(0xfb33);
const GRIN = u(0xd83d, 0xde00);
const CONTROL = u(0x80);
const O_UMLAUT = u(0xf6);

describe("canonicalJson", () => {
it("sorts object keys by UTF-16 code units at every depth", () => {
const inner = {
[EURO]: "Euro Sign",
"\r": "Carriage Return",
[DALET]: "Hebrew Letter Dalet With Dagesh",
"1": "One",
[GRIN]: "Emoji: Grinning Face",
[CONTROL]: "Control",
[O_UMLAUT]: "Latin Small Letter O With Diaeresis",
};

expect(canonicalJson(inner)).toBe(
`{"\\r":"Carriage Return","1":"One","${CONTROL}":"Control",` +
`"${O_UMLAUT}":"Latin Small Letter O With Diaeresis","${EURO}":"Euro Sign",` +
`"${GRIN}":"Emoji: Grinning Face","${DALET}":"Hebrew Letter Dalet With Dagesh"}`,
);
});

it("writes nested structures without insignificant whitespace", () => {
const document = {
processes: [{ name: "api" }, {}],
applications: [],
enabled: true,
disabled: false,
};

expect(canonicalJson(document)).toBe(
'{"applications":[],"disabled":false,"enabled":true,"processes":[{"name":"api"},{}]}',
);
});

it("escapes only what the specification escapes", () => {
const kept = `slash/ del${u(0x7f)} line${u(0x2028)} euro${EURO}`;
const text = `quote" backslash\\ controls\b\f\n\r\t${u(0x0f, 0x1f)} ${kept}`;

expect(canonicalJson(text)).toBe(
`"quote\\" backslash\\\\ controls\\b\\f\\n\\r\\t\\u000f\\u001f ${kept}"`,
);
});

it.each([
["0.0", "0"],
["-0.0", "0"],
["1.0", "1"],
["-1.5", "-1.5"],
["4.50", "4.5"],
["0.002", "0.002"],
["0.5", "0.5"],
["-0.000001", "-0.000001"],
["10.0", "10"],
["1.0E-6", "0.000001"],
["0.000001", "0.000001"],
["0.0000001", "1e-7"],
["1.0E-27", "1e-27"],
["123456789012345680000", "123456789012345680000"],
["1.0E21", "1e+21"],
["1.0E30", "1e+30"],
["1.2345E25", "1.2345e+25"],
["333333333.33333329", "333333333.3333333"],
["9007199254740991.0", "9007199254740991"],
["-9007199254740991.0", "-9007199254740991"],
["1.7976931348623157E308", "1.7976931348623157e+308"],
["4.9E-324", "5e-324"],
["295147905179352830000", "295147905179352830000"],
])("formats %s as %s", (literal, expected) => {
expect(canonicalJson(Number(literal))).toBe(expected);
});

it("refuses an absent value written as null or undefined, and says where", () => {
expect(() =>
canonicalJson({ applications: [{ id: "auth" }, null] }),
).toThrow(
"null at /applications/1: an absent optional field is absent, never null",
);
expect(() => canonicalJson({ x: undefined })).toThrow(
"undefined at /x: an absent optional field is absent, never null",
);
expect(() => canonicalJson(null)).toThrow("null at :");
});

it("escapes pointer segments in messages", () => {
expect(() => canonicalJson({ x: { "a/b~c": null } })).toThrow(
"null at /x/a~1b~0c:",
);
});

it("refuses numbers JSON cannot carry", () => {
expect(() => canonicalJson(Number.NaN)).toThrow(
"NaN at is not a JSON number",
);
expect(() => canonicalJson(Number.POSITIVE_INFINITY)).toThrow(
"Infinity at is not a JSON number",
);
});

it("refuses values that are not JSON", () => {
expect(() => canonicalJson(1n)).toThrow("bigint at is not a JSON value");
expect(() => canonicalJson(() => 1)).toThrow(
"function at is not a JSON value",
);
expect(() => canonicalJson(new Date(0))).toThrow(
"Date at is not a JSON value",
);
expect(() => canonicalJson({ m: new Map() })).toThrow(
"Map at /m is not a JSON value",
);
});

it("refuses strings that are not well-formed Unicode", () => {
expect(() => canonicalJson("lone \ud800 high")).toThrow(
"string at holds a lone surrogate at index 5",
);
expect(() => canonicalJson("lone \udc00 low")).toThrow(
"string at holds a lone surrogate at index 5",
);
expect(() => canonicalJson("ends \ud800")).toThrow(
"string at holds a lone surrogate at index 5",
);
expect(() => canonicalJson({ "bad \ud800": 1 })).toThrow(
"string at holds a lone surrogate at index 4",
);
});
});
110 changes: 110 additions & 0 deletions test/oracles.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
// REQ-019 (docs/requirements.md): every committed oracle file is in canonical form.
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { trackedText } from "../scripts/lib/tracked.ts";
import { canonicalJson } from "../src/index.ts";

const REPOSITORY = join(import.meta.dirname, "..");
const ORACLE =
/^spec\/v1\/examples\/(?:.+\/)?(?:expected\/[^/]+\.json|[^/]+\.diagnostics\.json)$/;

type Files = Readonly<Record<string, string>>;

function oracles(files: Files): Files {
return Object.fromEntries(
Object.entries(files).filter(([rel]) => ORACLE.test(rel)),
);
}

function oracleErrors(files: Files): string[] {
return Object.entries(oracles(files)).flatMap(([rel, text]) => {
let canonical: string;
try {
canonical = canonicalJson(JSON.parse(text));
} catch (error) {
return [`${rel}: ${(error as Error).message}`];
}
return text === canonical ? [] : [`${rel}: not in canonical form`];
});
}

describe("oracle files", () => {
const committed = oracles(trackedText(REPOSITORY));

it("include minimal's parsed intent and dependency edges", () => {
expect(Object.keys(committed)).toEqual(
expect.arrayContaining([
"spec/v1/examples/minimal/expected/intent.json",
"spec/v1/examples/minimal/expected/dependencies.json",
]),
);
});

it("are each byte-identical to their own canonical form", () => {
expect(oracleErrors(committed)).toStrictEqual([]);
});

it("carry one entry per Application in a dependencies oracle, each naming its edges", () => {
const shaped = (value: unknown): boolean => {
const applications = (value as { applications?: unknown }).applications;
return (
Array.isArray(applications) &&
applications.length > 0 &&
applications.every((entry: unknown) => {
const { id, edges } = entry as { id?: unknown; edges?: unknown };
return typeof id === "string" && Array.isArray(edges);
})
);
};
const dependencies = Object.entries(committed).filter(([rel]) =>
rel.endsWith("/dependencies.json"),
);

expect(dependencies.length).toBeGreaterThan(0);
for (const [rel, text] of dependencies)
expect(shaped(JSON.parse(text)), rel).toBe(true);
});
});

describe("oracleErrors", () => {
const at = "spec/v1/examples/x/expected/intent.json";

it.each([
["keys out of order", '{"b":1,"a":2}'],
["insignificant whitespace", '{"a": 1}'],
["a trailing newline", '{"a":1}\n'],
["a number not in shortest form", '{"a":1.50}'],
])("refuses %s", (_, text) => {
expect(oracleErrors({ [at]: text })).toStrictEqual([
`${at}: not in canonical form`,
]);
});

it("refuses a null, naming where it sits", () => {
expect(oracleErrors({ [at]: '{"a":[null]}' })).toStrictEqual([
`${at}: null at /a/0: an absent optional field is absent, never null`,
]);
});

it("refuses text that is not JSON", () => {
expect(oracleErrors({ [at]: "{" })[0]).toMatch(
/^spec\/v1\/examples\/x\/expected\/intent\.json: /,
);
});

it("holds refused-case diagnostics to the same form and ignores files that are not oracles", () => {
const refusal = "spec/v1/examples/refusals/y.diagnostics.json";

expect(
oracleErrors({
[refusal]: '[{"code":"X", "path":"/a"}]',
"spec/v1/examples/x/notes.json": '{"b":1,"a":2}',
"test/expected/intent.json": '{"b":1,"a":2}',
}),
).toStrictEqual([`${refusal}: not in canonical form`]);
});

it("passes a canonical oracle", () => {
expect(oracleErrors({ [at]: '{"a":1,"b":[true,"x"]}' })).toStrictEqual([]);
});
});
Loading
Loading