Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions changes/fix-attio-pilot-gaps.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
### Fixed

- **`new --from-config` with `auth.verify` makes a CLI without the template's sample.** `src/context.ts`,
`src/program.ts`, the test harness and `api`, `mcp` and `run-scope` tests reach the API and its fake, and the
sample's files are left out; before, they imported `src/sample/`, and `remove-sample` refused the CLI until about 30
lines were rewritten by hand. The hosts an OAuth sign-in in the description reaches, such as `app.attio.com`, join
`hosts`, so `lint:commands` no longer stops on them.
- **A function in a config's `source` stops `pnpm generate`** and names where it is, instead of being dropped
silently when the source is written out as JSON.
- **The seed's sync test reads the CRM's rows from `crmManifestFake`**, which pages them to their end. Over
`fakeServiceApi`, every page of an offset list held a row, so a sync read until its budget ran out and the test
timed out (382 pages in the Attio pilot).

### Added

- **`auth.verify.active`**: the field of the check's answer that must be `true`, for a service that answers a revoked
key with 200 and `active: false`, as Attio does. `session start` then says to sign in again, not `provider_error`.
- **A config with `source.users` mounts `users list|show`**, read live from the CRM.
2 changes: 1 addition & 1 deletion cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@
"version": "0.2",
"import": ["./config/wirecat/cspell.json"],
"ignorePaths": ["docs/commands.md", "docs/dev/**"],
"words": ["clidev", "gifsicle", "Petstore", "typechecks", "ydisk"]
"words": ["Attio", "clidev", "gifsicle", "Petstore", "typechecks", "ydisk"]
}
13 changes: 10 additions & 3 deletions kit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,15 +92,22 @@ read before the CLI has `node_modules`: its import resolves to the kit's own cli
- A `source` (a `crm` or a `tracker` category) needs `auth: { preset: "token", verify }`; `new` writes the files below only for what the config
holds at `new`, and `pnpm generate` then reads their values again.
- With `auth.verify`, `session start` is cli-core's `tokenSessionCommand`: it checks a token with that operation and
keeps the account's id from the answer.
keeps the account's id from the answer. `verify.active` names a field that must be `true`, for a service that
answers a revoked key with 200 and `active: false`, as Attio does; anything else is "sign in again".
- With `auth.verify` and a spec, the CLI starts without the template's sample: its wiring and tests reach the API and
its fake, so there is nothing for `remove-sample` to do. The hosts an OAuth sign-in in the description reaches join
`hosts`.
- With a `source`, `src/commands/crm.ts` mounts cli-core's CRM commands (`sync`, `store status`, `deals`, `search`…)
over `crmSourceFromManifest`, the store keyed by the account `session start` checked. A `tracker` source gets
over `crmSourceFromManifest`, the store keyed by the account `session start` checked; with `source.users`, also
`users list|show`, read live. A function in `source` stops `pnpm generate`: the source is written out as JSON, so
code goes into the hooks module. A `tracker` source gets
`src/commands/tracker.ts` in its place: cli-core's tracker commands (`sync`, `issues`, `projects`…) over
`trackerSourceFromManifest`, usually with `spec: { kind: "graphql", from: "<endpoint>" }`. `new` asks the endpoint
for its schema by introspection; a GraphQL schema names no sign-in, so the header the key travels in is `auth` in
`scripts/generate.ts`, the sample's `Authorization` until you check the service's documentation.
- Tests come with them: `src/commands/session.test.ts` signs in against a fake of the API, and
`src/crm-contracts.test.ts` runs `crmSourceContract` and `crmSyncContract`. `lint/contract-exceptions.json` excepts
`src/crm-contracts.test.ts` runs `crmSourceContract` and `crmSyncContract`, and a sync through the CLI over
`crmManifestFake`, which pages the rows to their end. `lint/contract-exceptions.json` excepts
`crmWriteContract`, as the source only reads. A list the generator finds no paging for fails the contract's
cut-list case on purpose: describe the paging in the API description or an override. A tracker's
`src/tracker-contracts.test.ts` checks the source against the generated API and syncs through the CLI over a fake
Expand Down
15 changes: 14 additions & 1 deletion kit/seeds/config/scripts/from-config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,19 @@ const out = join(root, "src/generated/wirecat-config.ts")
if (process.argv.includes("--if-missing") && existsSync(out)) process.exit(0)

const cli = parseCliConfig(config)
// The source is written out as JSON, which drops a function without a word: a rule that is code belongs in the hooks.
const functionsIn = (value: unknown, path: string): string[] =>
typeof value === "function"
? [path]
: value !== null && typeof value === "object"
? Object.entries(value).flatMap(([key, inner]) => functionsIn(inner, `${path}.${key}`))
: []
const functions = functionsIn(cli.source, "source")
if (functions.length)
throw new Error(
`wirecat.config: ${functions.join(", ")}: a function cannot be written into the CLI; ` +
"move it into the hooks module the config names in hooks (./src/hooks.ts)",
)
const verify = cli.auth.preset === "none" ? undefined : cli.auth.verify
const literal = (value: unknown) => (value === undefined ? "undefined" : JSON.stringify(value, null, 2))
// The hooks module is imported from src/generated/, inside the bundle: it must live under src/.
Expand All @@ -36,7 +49,7 @@ const text = formatted(
[
"// Generated from wirecat.config.ts by scripts/from-config.ts; run pnpm generate",
"",
`export const VERIFY: { operation: string; id: string; name?: string } | undefined = ${literal(verify)}`,
`export const VERIFY: { operation: string; id: string; name?: string; active?: string } | undefined = ${literal(verify)}`,
"",
`export const SOURCE: Readonly<Record<string, unknown>> | undefined = ${literal(cli.source)}`,
"",
Expand Down
2 changes: 2 additions & 0 deletions kit/seeds/config/src/commands/crm.ts
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ export const crmCommands = (): Command[] => {
app: IDENTITY,
open,
leads: Array.isArray(SOURCE?.kinds) && SOURCE.kinds.includes("lead"),
// People and companies have no live list in a manifest source yet; its users do.
...(SOURCE?.users ? { live: ["user"] as const } : {}),
})
return [...holder.commands]
}
12 changes: 4 additions & 8 deletions kit/seeds/config/src/commands/session.ts
Original file line number Diff line number Diff line change
@@ -1,20 +1,13 @@
import { CliError } from "@wirecat/cli-base"
import { hostFetch } from "@wirecat/cli-base/http"
import { serviceApiCall, tokenSessionCommand } from "@wirecat/cli-core/cli"
import { valueAt } from "@wirecat/cli-core/manifest"
import type { Command } from "commander"
import { API, configuration } from "../config.js"
import { contextOf } from "../context.js"
import { VERIFY } from "../generated/wirecat-config.js"
import { DESKTOP, IDENTITY } from "../identity.js"

const valueAt = (answer: unknown, path: string): unknown =>
path
.split(".")
.reduce<unknown>(
(at, key) => (at !== null && typeof at === "object" ? (at as Record<string, unknown>)[key] : undefined),
answer,
)

/**
* `session start | status | end | list` (WC-ACCT-12): the token is checked by the call `auth.verify` names in
* wirecat.config.ts, and the account is the id and name its answer holds.
Expand All @@ -34,6 +27,9 @@ export const sessionCommand = (): Command =>
token: () => token,
})
const answer = await call(VERIFY.operation, { path: {}, query: {}, headers: {}, fields: {} })
// A service such as Attio answers a revoked key with 200 and `active: false`, not with 401.
if (VERIFY.active && valueAt(answer, VERIFY.active) !== true)
throw new CliError("authentication_error", `${API.service} does not accept this token; sign in again`)
const id = valueAt(answer, VERIFY.id)
if (typeof id !== "string" && typeof id !== "number")
throw new CliError("provider_error", `${API.service} answered ${VERIFY.operation} without ${VERIFY.id}`, {
Expand Down
45 changes: 37 additions & 8 deletions kit/seeds/config/src/crm-contracts.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,18 @@ import { memoryKeyring } from "@wirecat/cli-base"
import { parseCrmManifest } from "@wirecat/cli-core/crm"
import {
type CrmManifestFake,
crmManifestFake,
crmManifestFixture,
crmSourceContract,
crmSyncContract,
FAKE_MANIFEST_TOKEN,
} from "@wirecat/cli-core/crm/testing"
import { type MessageStore, openStore } from "@wirecat/cli-core/store"
import { afterEach, describe, expect, it } from "vitest"
import { API } from "./config.js"
import { definitions } from "./generated/definitions.js"
import { HOOKS, SOURCE } from "./generated/wirecat-config.js"
import { FAKE_API_TOKEN, fakeServiceApi } from "./testing/fake-api.js"
import { HOOKS, SOURCE, VERIFY } from "./generated/wirecat-config.js"
import { fakeServiceApi } from "./testing/fake-api.js"
import { runCli } from "./testing/harness.js"

const manifest = parseCrmManifest(SOURCE, API.manifest)
Expand Down Expand Up @@ -45,18 +47,45 @@ describe("crmSyncContract over a store", () => {
it(name, run)
})

// fakeServiceApi answers every page of a list with a sample row, and an offset list ends only on an empty page: a sync
// over it reads until its budget runs out. The CRM's rows come from crmManifestFake, which pages them to their end; the
// call auth.verify names stays with fakeServiceApi, which crmManifestFake does not answer.
const signInAndCrm = (crm: CrmManifestFake) => {
const verify = API.manifest.operations.find((one) => one.id === VERIFY?.operation)
const path = verify?.binding.kind === "http" ? verify.binding.path : undefined
const pattern = path ? new RegExp(`${path.replace(/\{[^}]+\}/g, "[^/]+")}$`) : undefined
const signIn = fakeServiceApi(FAKE_MANIFEST_TOKEN)
return {
fetch: (input: string | URL | Request, init?: RequestInit) => {
const url = new URL(input instanceof Request ? input.url : input)
return pattern?.test(url.pathname) ? signIn.fetch(input, init) : crm.fetch(input, init)
},
}
}

it("sync through this CLI keeps the rows under the account session start checked", async () => {
const keyring = memoryKeyring()
const fake = fakeServiceApi()
const crm = crmManifestFake(manifest, { api: API.manifest, definitions })
fakes.push(crm)
const service = signInAndCrm(crm)
const started = await runCli(["personal", "session", "start", "--json"], {
keyring,
service: fake,
stdin: FAKE_API_TOKEN,
service,
stdin: FAKE_MANIFEST_TOKEN,
})
expect(started.code, started.err).toBe(0)
const synced = await runCli(["personal", "sync", "--json"], { keyring, service: fake })
const synced = await runCli(["personal", "sync", "--json"], { keyring, service })
expect(synced.code, synced.err).toBe(0)
const status = await runCli(["personal", "store", "status", "--json"], { keyring, service: fake })
expect(synced.json().item).toMatchObject({ complete: true })
const status = await runCli(["personal", "store", "status", "--json"], { keyring, service })
expect(status.code, status.err).toBe(0)
expect(fake.refusals).toEqual([])
const kinds = (status.json() as { items: { kind: string; stored: number; state: string }[] }).items
for (const kind of manifest.kinds)
expect(
kinds.find((one) => one.kind === kind),
kind,
).toMatchObject({
state: "complete",
stored: crm.rows[kind]?.length,
})
})
40 changes: 25 additions & 15 deletions kit/seeds/config/src/testing/fake-api.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
import { fakeApi, graphqlFake } from "@wirecat/cli-base/codegen"
import { fakeApi, graphqlFake, sampleOf } from "@wirecat/cli-base/codegen"
import { manifestBaseUrl } from "@wirecat/cli-core/cli"
import { API } from "../config.js"
import { definitions } from "../generated/definitions.js"
import { VERIFY } from "../generated/wirecat-config.js"

/** The token fakeServiceApi takes; never a real one (WC-TEST-2). */
export const FAKE_API_TOKEN = "fake0api0token"
Expand All @@ -16,6 +17,28 @@ const graphqlAuth = (token: string) => {
: { header: "Authorization", value: `Bearer ${token}` }
}

/** The answer of the call auth.verify names, with the token marked active where auth.verify.active says. */
const verifyAnswer = (): Record<string, () => { body: unknown }> => {
const operation = API.manifest.operations.find((one) => one.id === VERIFY?.operation)
const schema = operation?.response?.schema
const active = VERIFY?.active
if (!VERIFY || !active || !schema) return {}
return {
[VERIFY.operation]: () => {
const body = sampleOf({ type: "ref", ref: schema }, definitions)
const keys = active.split(".")
const last = keys.pop() as string
let here = body as Record<string, unknown>
for (const key of keys) {
here[key] ??= {}
here = here[key] as Record<string, unknown>
}
here[last] = true
return { body }
},
}
}

/** The service's API as its description says, answering only `token`: a sample of each answer schema. */
export const fakeServiceApi = (token = FAKE_API_TOKEN) =>
graphql
Expand All @@ -32,18 +55,5 @@ export const fakeServiceApi = (token = FAKE_API_TOKEN) =>
security: API.manifest.security,
securitySchemes: API.manifest.securitySchemes,
token,
answers: verifyAnswer(),
})

type Fetch = (input: string | URL | Request, init?: RequestInit) => Promise<Response>

/** The API's host goes to fakeServiceApi, any other to `rest` (the sample's fake while the sample stays). */
export const withServiceApi = (rest: { fetch: Fetch }, token: string): { fetch: Fetch } => {
const api = fakeServiceApi(token)
const host = new URL(manifestBaseUrl(API.manifest)).host
return {
fetch: (input, init) =>
new URL(input instanceof Request ? input.url : input).host === host
? api.fetch(input, init)
: rest.fetch(input, init),
}
}
127 changes: 127 additions & 0 deletions kit/src/config-wiring.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
import { withoutMount } from "./sample.js"
import { replaceOnce } from "./spec.js"

// A config whose `auth.verify` checks the token against the API has a fake of that API, so the CLI it makes starts on
// the API, not on the template's sample: `new` leaves the sample's files out and these rewrite the files that used it.
// `remove-sample` refused such a CLI before, since its wiring imported the sample.

const each = (file: string, text: string, pairs: readonly [string | RegExp, string][]): string =>
pairs.reduce((rest, [from, to]) => replaceOnce(file, rest, from, to), text)

const CONTEXT = "src/context.ts"
const withoutSampleContext = (text: string): string => {
const kept = each(CONTEXT, text, [
[" type EventSink,\n", ""],
['import { Pacer, pacePathFor } from "@wirecat/cli-core/sends"\n', ""],
[/^import \{ type CommentsService, commentsService \} from "\.\/sample\/comments\.js"\n/m, ""],
[/^import \{ transport \} from "\.\/sample\/transport\.js"\n/m, ""],
])
// The sample's service and its run wrapper go: the api group and the CRM commands run their own calls.
return each(CONTEXT, kept, [
[/\n\/\*\* The service for one run[\s\S]*?\n}\n/, "\n"],
[/\n\/\*\*\n \* Every remote call goes through here[\s\S]*?\n}\n/, ""],
])
}

const PROGRAM = "src/program.ts"
const withoutSampleProgram = (text: string): string =>
each(PROGRAM, withoutMount(text), [
['untrusted: ["comment text", "author names"],', 'untrusted: ["names and text the service\'s users wrote"],'],
['permissionExample: "comments.create=deny",', 'permissionExample: "api.list=deny",'],
])

const HARNESS = "src/testing/harness.ts"
const withoutSampleHarness = (text: string): string =>
each(HARNESS, text, [
[
'import { FAKE_TOKEN, fakeComments } from "./fake-comments.js"\n',
'import { FAKE_API_TOKEN as FAKE_TOKEN, fakeServiceApi } from "./fake-api.js"\n',
],
["service = fakeComments(),", "service = fakeServiceApi(),"],
])

const MCP = "src/mcp.test.ts"
const withoutSampleMcp = (text: string): string =>
each(MCP, text, [
[
'import { fakeComments } from "./testing/fake-comments.js"\n',
'import { fakeServiceApi } from "./testing/fake-api.js"\n',
],
['"comments.nope=deny"', '"api.nope=deny"'],
['toContain("comments.nope")', 'toContain("api.nope")'],
[/"comments\.create=deny"/g, '"api.list=deny"'],
['{ permissions: { "comments.list": "deny" } }', '{ permissions: { "api.list": "deny" } }'],
["const service = fakeComments()", "const service = fakeServiceApi()"],
['{ query: "comments list" }', '{ query: "api list" }'],
[/command: "comments list"/g, 'command: "api list"'],
])

const RUN_SCOPE = "src/run-scope.test.ts"
export const withoutSampleRunScope = (text: string): string =>
each(RUN_SCOPE, text, [
[
'import { FAKE_TOKEN, fakeComments } from "./testing/fake-comments.js"\n',
'import { FAKE_API_TOKEN as FAKE_TOKEN, fakeServiceApi } from "./testing/fake-api.js"\n',
],
[/^ +"comments [a-z]+": \[.*\],\n/gm, ""],
["fetch: fakeComments().fetch", "fetch: fakeServiceApi().fetch"],
])

/**
* run-scope over a config's wiring: `session start` checks the token with the API, so the answering fake is the API's,
* the sample gone; `sync` is cli-core's, run in its own run and tested by the source's contracts test.
*/
export const withConfigRunScope = (
text: string,
{ verifies, source, users = false }: { verifies: boolean; source?: "crm" | "tracker"; users?: boolean },
): string => {
const answering = verifies ? withoutSampleRunScope(text) : text
const sync = {
crm: ' sync: "cli-core\'s addCrmCommands runs it in its run; src/crm-contracts.test.ts runs crmSyncContract",',
tracker: ' sync: "cli-core\'s addTrackerCommands runs it in its run; src/tracker-contracts.test.ts syncs",',
}
return source
? replaceOnce(
RUN_SCOPE,
answering,
' "mcp serve": "listens until stopped",\n',
[
' "mcp serve": "listens until stopped",',
sync[source],
...(users
? [
' "users list": "cli-core\'s addCrmCommands runs it in its run, over source.users",',
' "users show": "cli-core\'s addCrmCommands runs it in its run, over source.users",',
]
: []),
"",
].join("\n"),
)
: answering
}

const API_TEST = "src/api.test.ts"
const withoutSampleApiTest = (text: string): string =>
each(API_TEST, text, [
[
'import { FAKE_TOKEN } from "./testing/fake-comments.js"\n',
'import { FAKE_API_TOKEN as FAKE_TOKEN } from "./testing/fake-api.js"\n',
],
])

const IDENTITY = "src/identity.ts"
const withoutSampleHost = (text: string): string =>
each(IDENTITY, text, [
[/^ *\/\/ SAMPLE: every host the CLI contacts.*\n/m, ""],
[/"comments\.example\.invalid",? ?/, ""],
])

/** The rewrites of the files that used the sample, by path. */
export const OFF_SAMPLE: Readonly<Record<string, (text: string) => string>> = {
[CONTEXT]: withoutSampleContext,
[PROGRAM]: withoutSampleProgram,
[HARNESS]: withoutSampleHarness,
[MCP]: withoutSampleMcp,
[API_TEST]: withoutSampleApiTest,
[IDENTITY]: withoutSampleHost,
}
Loading
Loading