diff --git a/CHANGELOG.md b/CHANGELOG.md index a94f0a1..9613a18 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,17 @@ Every entry says what changed as a caller sees it, why, and what to watch for ## Unreleased +### Added + +- **The generator puts operations in subgroups of `api`: `names.groups`.** With `"tags"` each operation gets its first + tag as its `group` in the manifest (a Postman folder is one), with `"first-word"` the first word of its command, for + a spec whose tags name no resource. `names.groupNames` renames a group or keeps it flat. cli-core then mounts + `contacts-list` in the group `contacts` as `api contacts list`, with the key `api.contacts.list`. Generation stops + on a group named `list`, `show` or `help`, a group named like a command, or two operations on one path. Without + `names.groups` the manifest is as before. `apiCommandPath` gives an operation's words after `api`. WordPress + routes get their resource as a tag (`posts`), so a wp-json CLI can group too. The rule is + [WC-CMD-33](docs/standards/commands.md#wc-cmd-33--group-a-generated-api-by-resource-api-group-command). + ### Fixed - **`--site` and `--login` have their role in the option vocabulary.** cli-core's `tokenSessionCommand` adds them to diff --git a/docs/standards/README.md b/docs/standards/README.md index 79c1bc0..70a31f8 100644 --- a/docs/standards/README.md +++ b/docs/standards/README.md @@ -29,7 +29,7 @@ the four steps that need a person. A new CLI is ready for its first release when |---|---|---:|---| | [Toolchain](toolchain.md) | `TOOL` | 20 | TypeScript, Biome, grit plugins, size caps, import cycles, spelling, Markdown, Go | | [Layout](layout.md) | `LAY` | 10 | Repository layout, entry points, exports, file size, one job per file, splitting a unit, the bundled bin | -| [Commands](commands.md) | `CMD` | 32 | Paths, vocabulary, flags, arguments, effects, lifetime, query, help, external references, joining `wcat`, settings in the config file, refs, network hosts | +| [Commands](commands.md) | `CMD` | 33 | Paths, vocabulary, flags, arguments, effects, lifetime, query, help, external references, joining `wcat`, settings in the config file, refs, network hosts, the generated `api` group | | [Output](output.md) | `OUT` | 25 | Data on stdout only, envelopes, errors, exit codes, partial answers, projection, list results in a TypeScript API | | [MCP](mcp.md) | `MCP` | 13 | The MCP tools, how they come from commands, what each permission shows | | [Accounts](accounts.md) | `ACCT` | 13 | The explicit profile, reads across accounts, scope flags, linked identities, sign-in commands, memo's account groups | diff --git a/docs/standards/commands.md b/docs/standards/commands.md index 8f7012f..0755b31 100644 --- a/docs/standards/commands.md +++ b/docs/standards/commands.md @@ -106,7 +106,10 @@ per profile, and only `defaults` is filled. - **Enforced by:** review only. - **Example:** the release notes give the old and the new path under "Changed — may break callers". Help, completion, generated pages, the skill, MCP names and permission keys move in the same change. -- **Exceptions:** none. +- **Exceptions:** the old flat permission key of a generated `api` command that moved into a subgroup stays an alias + that only tightens + ([WC-CMD-33](#wc-cmd-33--group-a-generated-api-by-resource-api-group-command)); the old command path still stops + working. - **Applies to:** all CLIs. **Since:** cli-base 0.41.0. **Rulings:** NEED-417. ### WC-CMD-9 · Answer the naming questions in the pull request before you build a new command @@ -495,6 +498,34 @@ the machine and checks every redirect. inside a sentence left out of `undeclared-host`, since the release after 0.98.0. **Rulings:** none. +## The generated `api` group + +### WC-CMD-33 · Group a generated `api` by resource: `api ` + +A CLI whose `api` group cli-core builds from a generated manifest (`serviceApiCommand`, `generatedApiCommand`) sets +`names.groups` in `generate`: `"tags"` puts each operation under its first tag (a Postman folder is one), and +`"first-word"` under the first word of its command, for a spec whose tags name no resource (HubSpot's `Basic` and +`Batch`). `names.groupNames` renames a group, or keeps a tag's operations flat with `""`. The command drops the +repeated group word: `contacts-list` in `contacts` is `api contacts list`. An operation without a tag, and a group of +one operation, stay directly under `api`. + +- **Why:** a flat list of a hundred operations is hard to read in help and hard for an agent to search; a group per + resource is how the service's own reference is laid out, and one permission key can then cover a resource. +- **Enforced by:** `src/codegen/groups.test.ts` · `cli-core: src/cli/service-api-groups.test.ts`. The generator + refuses a group that would shadow the group's own commands, a group named like a command, and two operations on one + path. +- **Example:** `api.contacts: deny` stops every `api contacts` command; `api.contacts.delete` is the delete's own key. + A profile that still names the old flat key `api.contacts-list` keeps that hold: cli-core checks it beside + `api.contacts.list` on every call, from the command line and over MCP, and warns to rename it. Its `deny`, `ask` or + `readonly` (on a write) applies; its `allow` lifts nothing. The old path `api contacts-list` is gone + ([WC-CMD-8](#wc-cmd-8--keep-no-aliases-a-renamed-path-stops-working-in-the-release-that-renames-it)): the release + that groups a CLI's `api` lists the move under "Changed — may break callers" and raises the minor version. + `api show` takes the new path, the old name or the operation id; `api list` prints the path and the group. +- **Exceptions:** a manifest generated without `names.groups` stays flat. gcal and drive keep their own + `api...` keys, and the messenger CLIs' `bot api` stays flat, until each is moved. +- **Applies to:** every CLI with a generated `api` group, from its next regeneration. **Since:** the release after + cli-base 0.120.0, with the cli-core release that mounts groups. **Rulings:** NEED-2754. + ## Sources These documents explain the rules on this page in more detail. Where one of them disagrees with this page, this page diff --git a/docs/standards/kit-surface.md b/docs/standards/kit-surface.md index ec5b79d..c018fb6 100644 --- a/docs/standards/kit-surface.md +++ b/docs/standards/kit-surface.md @@ -35,8 +35,9 @@ loads `@wirecat/cli-base/testing/setup`, and `.vale.ini` reads the styles in `co ## The API generator `@wirecat/cli-base/codegen` turns a service's API description into committed files: types, Valibot schemas, an -operation manifest and a coverage page. cli-core's `generatedApiCommand` builds the `api ` group from the -manifest. The [reference](../REFERENCE.md) has the API; this section says what goes in, what a CLI can correct, and +operation manifest and a coverage page. cli-core's `generatedApiCommand` builds the `api` group from the manifest: `api ` once +`names.groups` puts the operations in groups +([WC-CMD-33](commands.md#wc-cmd-33--group-a-generated-api-by-resource-api-group-command)), `api ` without. The [reference](../REFERENCE.md) has the API; this section says what goes in, what a CLI can correct, and what it cannot do yet. ### Inputs @@ -170,7 +171,7 @@ source whose subtypes the definitions cannot tell apart. `apiSkillFiles({ cli, folder, files, service, skill, operations, commands, banner })` writes the API part of the CLI's skill, in the shape of [openapi-to-skills](https://github.com/neutree-ai/openapi-to-skills): `references/api.md`, a short index of the groups with their effects, and `references/api/api-.md`, one file per group. An -operation's group is its first tag; one without a tag is in `api-untagged.md`. A tag with no Latin letter or digit +operation's group is its `group` (WC-CMD-33), else its first tag; one without either is in `api-untagged.md`. A tag with no Latin letter or digit cannot name a file and stops the generation. An agent reads the index, then only the group it needs, with ` skill show api-`. The `skill` command already ships and shows every file beside `SKILL.md`; nothing else is needed. @@ -206,7 +207,7 @@ the coverage page after a spec sync. - **cli-core checks the permission keys; the CLI still shows them.** From cli-core 0.684.0, `serviceApiCommand` reads each operation's `permissions`, `servers` and `securitySchemes`: it sends the call to the operation's server - with the sign-in its scheme names, and checks `api.` and each manifest key that applies to the call + with the sign-in its scheme names, and checks the command's own key (`api..`, its path) and each manifest key that applies to the call (`files.delete` when `permanently` is true) before it sends. That guard runs whatever the CLI declares. `--dry-run` and permission discovery show those keys only when the CLI merges `serviceApiPermissions(...)` into its own `commandPermissions`, field by field (`extra`, `alsoChecks`, `defaults`), as ydisk's `src/config.ts` does: diff --git a/src/codegen/groups.test.ts b/src/codegen/groups.test.ts new file mode 100644 index 0000000..9153cd1 --- /dev/null +++ b/src/codegen/groups.test.ts @@ -0,0 +1,78 @@ +import { describe, expect, it } from "vitest" +import { adaptOpenApi, apiCommandPath, applyNames, type CommandNames, generate, manifestGenerator } from "./index.js" + +type Paths = Record> + +const spec = (paths: Paths) => adaptOpenApi({ openapi: "3.1.0", info: {}, paths, components: { schemas: {} } }) + +const paths = (model: ReturnType) => + model.operations.map((operation) => + apiCommandPath({ command: operation.command ?? operation.id, group: operation.group }).join(" "), + ) + +const crm: Paths = { + "/contacts": { + get: { operationId: "contacts-list", tags: ["Contacts"] }, + post: { operationId: "contacts-create", tags: ["Contacts", "Batch"] }, + }, + "/pages/move": { post: { operationId: "move-page", tags: ["Pages"] } }, + "/pages": { get: { operationId: "pages-list", tags: ["Pages"] } }, + "/search": { post: { operationId: "search", tags: ["Search"] } }, + "/me": { get: { operationId: "me" } }, +} + +describe("api subgroups", () => { + it("groups by the first tag, cuts the repeated prefix, and keeps a lone or untagged operation flat", () => { + const model = applyNames(spec(crm), { groups: "tags" }) + expect(paths(model)).toEqual(["contacts list", "contacts create", "pages move-page", "pages list", "search", "me"]) + expect(model.operations.find(({ id }) => id === "search")?.group).toBeUndefined() + }) + + it("groups by the first word of the command where the tags name no resource, and renames a group", () => { + const hubspot: Paths = { + "/contacts": { + get: { operationId: "contacts-list", tags: ["Basic"] }, + post: { operationId: "contacts-batch-create", tags: ["Batch"] }, + }, + "/deals": { get: { operationId: "deals-list", tags: ["Basic"] }, post: { operationId: "deals-create" } }, + } + const names: CommandNames = { groups: "first-word", groupNames: { deals: "opportunities" } } + expect(paths(applyNames(spec(hubspot), names))).toEqual([ + "contacts list", + "contacts batch-create", + "opportunities deals-list", + "opportunities deals-create", + ]) + expect(paths(applyNames(spec(hubspot), { groups: "first-word", groupNames: { deals: "" } }))).toEqual([ + "contacts list", + "contacts batch-create", + "deals-list", + "deals-create", + ]) + }) + + it("writes the group into the manifest, and leaves a manifest without `groups` as it was", () => { + const manifestOf = (names: CommandNames) => + generate(spec(crm), [manifestGenerator({ path: "manifest.ts" })], { banner: [], names })[0]?.content ?? "" + expect(manifestOf({ groups: "tags" })).toContain('"group": "contacts"') + expect(manifestOf({})).not.toContain('"group"') + }) + + it("refuses a group named like api's own command or like a command, two commands on one path, and a stale name", () => { + const run = + (more: Paths, names: CommandNames = { groups: "tags" }) => + () => + applyNames(spec({ ...crm, ...more }), names) + expect( + run({ "/a": { get: { operationId: "a", tags: ["List"] } }, "/b": { get: { operationId: "b", tags: ["List"] } } }), + ).toThrow('group "list" is api\'s own command') + expect(run({ "/c": { get: { operationId: "contacts", tags: ["Contacts"] } } })).toThrow( + 'group "contacts" is also the command "contacts"', + ) + expect(run({ "/l": { get: { operationId: "list", tags: ["Contacts"] } } })).toThrow( + '"contacts-list" and "list" are both api contacts list', + ) + expect(run({}, { groups: "tags", groupNames: { Nope: "x" } })).toThrow('group "Nope" is no tag') + expect(run({}, { groups: "tags", groupNames: { Pages: "Big Pages" } })).toThrow('group "Big Pages" is not kebab') + }) +}) diff --git a/src/codegen/index.ts b/src/codegen/index.ts index 2a2ac06..e0102e4 100644 --- a/src/codegen/index.ts +++ b/src/codegen/index.ts @@ -86,7 +86,7 @@ export type { SourceInfo, SourceReference, } from "./model.js" -export { bindingLabel, identifier, kebab } from "./names.js" +export { apiCommandPath, bindingLabel, identifier, kebab } from "./names.js" export { adaptOpenApi, OpenApiError } from "./openapi.js" export { detectPagination } from "./pagination.js" export { type PatchedSpec, type PatchSpecOptions, patchSpec, type SpecPatch } from "./patch.js" diff --git a/src/codegen/manifest.ts b/src/codegen/manifest.ts index d28f13b..57c047b 100644 --- a/src/codegen/manifest.ts +++ b/src/codegen/manifest.ts @@ -18,6 +18,8 @@ import { SchemaTree } from "./tree.js" export interface ManifestOperation { id: string command: string + /** Mounted as `api `, the command without its `-` prefix; none is `api `. */ + group?: string binding: Binding effect: Effect summary?: string @@ -46,6 +48,7 @@ const toManifest = (operation: ApiOperation, tree: SchemaTree): ManifestOperatio withoutUndefined({ id: operation.id, command: operation.command ?? kebab(operation.id), + group: operation.group, binding: operation.binding, effect: operation.effect as Effect, summary: operation.summary, diff --git a/src/codegen/model.ts b/src/codegen/model.ts index 8f1b6e4..05342dd 100644 --- a/src/codegen/model.ts +++ b/src/codegen/model.ts @@ -126,6 +126,8 @@ export interface ApiOperation { id: string /** Set by `generate`'s `names`; otherwise the command is the kebab-case id. */ command?: string + /** The subgroup of `api` it sits in, set by `generate`'s `names.groups`; none keeps it directly under `api`. */ + group?: string binding: Binding summary?: string description?: string diff --git a/src/codegen/names.ts b/src/codegen/names.ts index b1f894b..714bd43 100644 --- a/src/codegen/names.ts +++ b/src/codegen/names.ts @@ -50,6 +50,13 @@ export const kebab = (id: string): string => .replace(/^-|-$/g, "") .toLowerCase() +/** The words after `api`: `contacts-list` in the group `contacts` is `contacts list`; `move-page` in `pages` keeps its name. */ +export const apiCommandPath = (operation: { command: string; group?: string }): string[] => { + const { command, group } = operation + if (!group) return [command] + return [group, command.startsWith(`${group}-`) ? command.slice(group.length + 1) : command] +} + /** How a reader names the call: `GET /pages/{id}`, `sendMessage`, `query issues`. */ export const bindingLabel = (binding: Binding): string => binding.kind === "http" diff --git a/src/codegen/pipeline.ts b/src/codegen/pipeline.ts index 23e00e3..40360c4 100644 --- a/src/codegen/pipeline.ts +++ b/src/codegen/pipeline.ts @@ -14,7 +14,7 @@ import type { SecurityRequirement, SecurityScheme, } from "./model.js" -import { bindingLabel, identifier, kebab } from "./names.js" +import { apiCommandPath, bindingLabel, identifier, kebab } from "./names.js" import { answerPaths, detectPagination, requestParameters } from "./pagination.js" import { signInRequirements } from "./sign-in.js" @@ -53,6 +53,13 @@ export interface CommandNames { fromPath?: boolean /** A path segment → its word in a command; `""` leaves it out: `{ "0-3": "deals", crm: "", objects: "" }`. */ segments?: Readonly> + /** + * Puts the operations in subgroups of `api`: by their first tag (a Postman folder is one), or by the first word of + * their command where the tags do not name resources (HubSpot's `Basic` and `Batch`). Left out, `api` stays flat. + */ + groups?: "tags" | "first-word" + /** A tag or a first word → its group, kebab case; `""` keeps those operations flat: `{ "Activity records": "activities" }`. */ + groupNames?: Readonly> } const COMMAND = /^[a-z0-9]+(-[a-z0-9]+)*$/ @@ -126,7 +133,51 @@ export const applyNames = (model: ApiModel, names: CommandNames): ApiModel => { return { ...operation, command } }) if (problems.length > 0) throw new CodegenError(problems) - return { ...model, operations } + return { ...model, operations: names.groups ? applyGroups(operations, names) : operations } +} + +// `api list` and `api show` are the group's own commands. +const RESERVED_GROUPS = new Set(["list", "show", "help"]) + +const applyGroups = (operations: readonly ApiOperation[], names: CommandNames): ApiOperation[] => { + const commandOf = (operation: ApiOperation) => operation.command ?? kebab(operation.id) + const sourceOf = (operation: ApiOperation) => + names.groups === "tags" ? operation.tags[0] : commandOf(operation).split("-")[0] + const problems: string[] = [] + const sources = new Set(operations.map(sourceOf)) + for (const source of Object.keys(names.groupNames ?? {})) + if (!sources.has(source)) + problems.push(`names: group "${source}" is no ${names.groups === "tags" ? "tag" : "word"}`) + const grouped = operations.map((operation) => { + const source = sourceOf(operation) + if (source === undefined) return { operation } + const named = names.groupNames && Object.hasOwn(names.groupNames, source) ? names.groupNames[source] : kebab(source) + return { operation, group: named || undefined } + }) + const sizes = new Map() + for (const { group } of grouped) if (group) sizes.set(group, (sizes.get(group) ?? 0) + 1) + // A group of one only makes its operation's path longer. + const result = grouped.map(({ operation, group }) => + group && (sizes.get(group) ?? 0) > 1 ? { ...operation, group } : operation, + ) + const commands = new Set(result.map(commandOf)) + const paths = new Map() + for (const operation of result) { + const path = apiCommandPath({ command: commandOf(operation), group: operation.group }).join(" ") + const other = paths.get(path) + if (other !== undefined) problems.push(`names: "${other}" and "${commandOf(operation)}" are both api ${path}`) + paths.set(path, commandOf(operation)) + } + for (const group of new Set(result.flatMap(({ group }) => (group ? [group] : [])))) { + if (!COMMAND.test(group)) problems.push(`names: group "${group}" is not kebab case; rename it in groupNames`) + if (RESERVED_GROUPS.has(group)) + problems.push(`names: group "${group}" is api's own command; rename it in groupNames`) + // The old flat key `api.` stays an alias of the moved command, so it must never name a whole group. + if (commands.has(group)) + problems.push(`names: group "${group}" is also the command "${group}"; rename one in groupNames or commands`) + } + if (problems.length > 0) throw new CodegenError(problems) + return result } /** Each read's pagination, detected unless an override set it, or said with `null` that the read does not page. */ diff --git a/src/codegen/skill.ts b/src/codegen/skill.ts index a62c097..54efcb8 100644 --- a/src/codegen/skill.ts +++ b/src/codegen/skill.ts @@ -1,6 +1,6 @@ /** * The API part of a CLI's agent skill: `references/api.md`, a short index, and `references/api/api-.md`, one - * file per first tag of the operations, so an agent loads only the group it needs (` skill show api-`). Each command, + * file per group of the operations (their `group`, else their first tag), so an agent loads only the group it needs (` skill show api-`). Each command, * its usage, arguments and flags come from the built program's command tree, as `--help` prints them; the manifest * adds only what the API says: the HTTP call, the effect, sign-in scopes, permission keys and paging. */ @@ -141,7 +141,8 @@ export const apiSkillFiles = (options: ApiSkillOptions): GeneratedArtifact[] => const groups = new Map() for (const operation of options.operations) { - const tag = operation.tags[0] ?? "untagged" + // The skill's files follow the command groups where the manifest has them, so `api-contacts.md` holds `api contacts`. + const tag = operation.group ?? operation.tags[0] ?? "untagged" const name = kebab(tag) const group = groups.get(name) if (!name) problems.push(`tag "${tag}" of ${operation.id} has no Latin letter or digit for a file name`) diff --git a/src/codegen/wp-json.ts b/src/codegen/wp-json.ts index 4ad52dd..099facd 100644 --- a/src/codegen/wp-json.ts +++ b/src/codegen/wp-json.ts @@ -216,6 +216,8 @@ export const wpJsonToOpenApi = (index: unknown, options: WpJsonOptions): WpJsonR continue } const item: Json = {} + // The resource after the namespace, `posts` in `/wp/v2/posts/(?P…)`, so `names.groups` can group by it. + const resource = route.slice(namespace.length + 2).split("/")[0] ?? "" const endpoints = Array.isArray(raw.endpoints) ? raw.endpoints.filter(object) : [] try { for (const endpoint of endpoints) { @@ -238,6 +240,7 @@ export const wpJsonToOpenApi = (index: unknown, options: WpJsonOptions): WpJsonR })) const rest = Object.entries(args).filter(([name]) => patterns[name] === undefined) const operation: Json = { parameters: pathParameters } + if (/^[A-Za-z][\w-]*$/.test(resource)) operation.tags = [resource] if (method === write) { if (rest.length > 0) operation.requestBody = {