From 3e9ef40580e1be268c1cd0e3979cfa231a78138b Mon Sep 17 00:00:00 2001 From: leemour Date: Wed, 18 Nov 2026 09:40:05 +0100 Subject: [PATCH 1/2] =?UTF-8?q?docs(generation):=20attio=20from=20a=20conf?= =?UTF-8?q?ig=20file=20=E2=80=94=2087=20lines=20of=20config=20and=20hook,?= =?UTF-8?q?=20against=202,340=20in=20hubspot-cli?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit attio-cli was the pilot of building a CRM CLI only from wirecat.config.ts and hooks. It took 64 config lines, a 23-line hook and 19 lines of wiring, in 1 h 20 min; 17 tests then failed on bugs in cli-base and cli-core, fixed in cli-base #642 and cli-core #3102. Counted with new-cli-lines' count.py against hubspot-cli and bitrix24-cli, with the scope difference stated. Co-Authored-By: Claude Opus 5.5 --- performance/README.md | 2 +- performance/generation/README.md | 2 + .../generation/source-manifest/README.md | 87 +++++++++++++++++++ .../generation/source-manifest/result.json | 36 ++++++++ 4 files changed, 126 insertions(+), 1 deletion(-) create mode 100644 performance/generation/source-manifest/README.md create mode 100644 performance/generation/source-manifest/result.json diff --git a/performance/README.md b/performance/README.md index aaef6e4..00fab4c 100644 --- a/performance/README.md +++ b/performance/README.md @@ -1,7 +1,7 @@ # Performance - [generation](generation/README.md): how much of a new CLI the shared foundation writes — `ydisk` against - notion-cli. + notion-cli, and `attio` from a config file. - [search](search/README.md): message and document search quality and speed. - [runs](runs/README.md): run records — the `runs` commands, and secrets hidden in them by content. - [documents](documents/README.md): reading documents into text, starting with HTML. diff --git a/performance/generation/README.md b/performance/generation/README.md index 3a2f2f7..5b3579c 100644 --- a/performance/generation/README.md +++ b/performance/generation/README.md @@ -4,3 +4,5 @@ How much of a new CLI the shared foundation writes, and how long a build takes o - [yandex-disk](yandex-disk/README.md): `ydisk` on cli-template 0.25.0, cli-core 0.689.0 and cli-base 0.110.0 — hand-written lines and hours from `new` to green, against notion-cli. +- [source-manifest](source-manifest/README.md): `attio` built only from `wirecat.config.ts` and one hook, on + cli-template 0.28.0 — hand-written lines and time, against hubspot-cli and bitrix24-cli. diff --git a/performance/generation/source-manifest/README.md b/performance/generation/source-manifest/README.md new file mode 100644 index 0000000..c18fdbb --- /dev/null +++ b/performance/generation/source-manifest/README.md @@ -0,0 +1,87 @@ + + +# Attio CLI from a config file: hand-written code and time + +Measured 2026-11-18 on the first build of `attio` (`@wirecat/attio-cli`). + +## Why it was run + +hubspot-cli was the first CRM CLI made from the template, and it still took 2,340 hand-written lines: most of them map +HubSpot's answers to cli-core's CRM records ([new-cli-lines](../../engineering/new-cli-lines/README.md)). The next +step of the plan moves that mapping into a config file: `wirecat.config.ts` names the API calls and the path of each +field in the answer, cli-core's runtime does the paging and the sync, and a builder writes code only for what the +config cannot say (a hook). + +Attio was the pilot. The question: how much does a CRM CLI take by hand when it is built only from the config and +hooks, and how long from `cli-template new` to `cli-dev ready`? Every place the config could not say something was +logged as a gap. The answer decides whether the next CRM CLIs start from a config file. + +## Setup + +- Linux x86_64, Node 24.19.0. One builder agent, with a fresh agent for each review. +- attio-cli made with `new --from-config` from cli-template 0.28.0, on cli-core 0.696.0 and cli-base 0.114.0 at the + start; finished on cli-base 0.117.0 and cli-core 0.698.0, which carry the gaps the pilot fixed (below). +- attio-cli syncs Attio's people, companies and workspace members into cli-core's store, and has an `api` group of + 108 operations generated from Attio's OpenAPI description. It has no deals, no custom attributes, no writes and no + OAuth yet (see "What was not measured"). +- hubspot-cli at `253e6c4` and bitrix24-cli at `7e2877e`, counted as in + [new-cli-lines](../../engineering/new-cli-lines/README.md), with the same exclusions for bitrix24-cli. +- The command, with [count.py](../../engineering/new-cli-lines/count.py) from new-cli-lines: + +```sh +python3 ../../engineering/new-cli-lines/count.py \ + --repo attio="$HOME"/Projects/AI/attio-cli@003b809 --since attio=8ab8f8b \ + --repo hubspot="$HOME"/Projects/AI/hubspot-cli@253e6c4 --since hubspot=28d04db \ + --repo bitrix24="$HOME"/Projects/AI/bitrix24-cli@7e2877e \ + --exclude 'bitrix24:src/calls/**' --exclude bitrix24:src/commands/calls.ts \ + --exclude bitrix24:src/commands/meetings.ts --exclude bitrix24:src/commands/sync-calls.ts \ + --exclude bitrix24:src/commands/transcribe.ts --exclude bitrix24:src/commands/models.ts \ + --exclude bitrix24:src/commands/products.ts --exclude bitrix24:src/crm/product.ts \ + --exclude bitrix24:src/crm/stage-history.ts --exclude bitrix24:src/crm/writer.ts \ + --exclude bitrix24:src/bitrix/oauth.ts +wc -l "$HOME"/Projects/AI/attio-cli/wirecat.config.ts +``` + +`8ab8f8b` is the commit `new` made. `count.py` counts `.ts` files under `src/`; `wirecat.config.ts` sits at the +repository root, so it is counted with `wc -l`. The raw numbers are in [result.json](result.json). + +## Results + +| | attio-cli | hubspot-cli | bitrix24-cli | +|---|---:|---:|---:| +| Config (`wirecat.config.ts`) | **64** | — | — | +| Hooks (`src/hooks.ts`) | **23** | — | — | +| Source written after `new`, hooks included | **42** (23 hook + 19 wiring) | 2,340 | — | +| Source lines, comparable | 402 | 2,636 | 4,148 | +| Tests written after `new` | 98 | 1,403 | 6,958 (all) | +| Test helpers written after `new` | 3 | 598 | 810 (all) | +| From `new` to `cli-dev ready` (all rows but the live check) | 1 h 20 min, with 17 tests failing on the kit's bugs; green after the fixes below (about 1 h 20 min more) and two releases | 50 min; 1 h 38 min with reviews | 11 days | + +The 19 lines of wiring are what `new` could not know: hosts, texts, the pace of requests, and the `live` option for the +users list. The 64 config lines map 3 kinds (people, companies, workspace members) with 18 field rules; the 23 hook +lines keep only the current value of each Attio attribute (Attio keeps old values in the same list). + +## What it showed + +- **87 lines of config and hook and 19 more of wiring, against 2,340 in hubspot-cli.** The scope is smaller + (see below), so the two totals are not a percentage saving: hubspot-cli also syncs deals, leads, activities, + owners and pipelines. +- **The config was not the slow part; the kit's bugs were.** The CLI was written in 1 h 20 min (21:58 → 23:18), and + `cli-dev ready` then showed everything verified but the tests: 17 failed. All 17 were bugs in the shared packages, + not in attio-cli: the strict fake refused its own sample of a `oneOf`, a sign-in check read the first branch of a + union (Attio's "token not active" answer), and pattern samples did not match their patterns. Fixing them took + cli-base [#642](https://github.com/WireCatLabs/cli-base/pull/642) and cli-core + [#3102](https://github.com/WireCatLabs/cli-core/pull/3102), about 1 h 20 min more with reviews, plus the releases. +- **Attio's API description generated with no patch** once cli-base #642 was in (108 operations). Before it, the + builder needed four patches of the description and three command names. +- **15 findings were logged** in the plan: kit bugs, and places where the config cannot say something. The ones that block more of Attio: deals (a status + attribute with no fixed stage list and no outcome hook), custom attributes (no per-workspace field list), and OAuth + as a second sign-in. + +## What was not measured + +- Deals, custom attributes and custom objects, writes, OAuth: attio-cli does not have them yet, so the comparison + covers people, companies and users only. hubspot-cli's numbers include deals, leads, activities, owners and + pipelines. +- The live check against a real Attio workspace: it is the owner's and had not run when this was written. +- Hours a person spent: the owner answered three questions. diff --git a/performance/generation/source-manifest/result.json b/performance/generation/source-manifest/result.json new file mode 100644 index 0000000..46d841e --- /dev/null +++ b/performance/generation/source-manifest/result.json @@ -0,0 +1,36 @@ +{ + "attio": { + "commit": "003b809", + "source": 402, + "tests": 811, + "test_helpers": 92, + "generated": 97839, + "excluded_source": 0, + "written_since_template": { + "source": 42, + "tests": 98, + "test_helpers": 3 + } + }, + "hubspot": { + "commit": "253e6c4", + "source": 2636, + "tests": 1951, + "test_helpers": 656, + "generated": 12027, + "excluded_source": 0, + "written_since_template": { + "source": 2340, + "tests": 1403, + "test_helpers": 598 + } + }, + "bitrix24": { + "commit": "7e2877e", + "source": 4148, + "tests": 6958, + "test_helpers": 810, + "generated": 0, + "excluded_source": 2293 + } +} From 310e4c76dfd83cd3a94c2a31512d3503b6f6250b Mon Sep 17 00:00:00 2001 From: leemour Date: Wed, 18 Nov 2026 21:03:38 +0100 Subject: [PATCH 2/2] docs(generation): attio 0.1.0 with deals: 126 config lines, an 8-line hook The owner asked for attio to work like bitrix24-cli. Deals and the workspace's own attributes cost 62 config lines and no new hook; the current-value hook became cli-core's custom.typed. Recounted at 0c33e81; 107 operations, not 108. Co-Authored-By: Claude Opus 5.5 --- .../generation/source-manifest/README.md | 39 +++++++++++++++---- .../generation/source-manifest/result.json | 15 +++++++ 2 files changed, 47 insertions(+), 7 deletions(-) diff --git a/performance/generation/source-manifest/README.md b/performance/generation/source-manifest/README.md index c18fdbb..4043d95 100644 --- a/performance/generation/source-manifest/README.md +++ b/performance/generation/source-manifest/README.md @@ -2,7 +2,7 @@ # Attio CLI from a config file: hand-written code and time -Measured 2026-11-18 on the first build of `attio` (`@wirecat/attio-cli`). +Measured 2026-11-18 on the first build of `attio` (`@wirecat/attio-cli`) and on its first release, 0.1.0. ## Why it was run @@ -22,7 +22,7 @@ logged as a gap. The answer decides whether the next CRM CLIs start from a confi - attio-cli made with `new --from-config` from cli-template 0.28.0, on cli-core 0.696.0 and cli-base 0.114.0 at the start; finished on cli-base 0.117.0 and cli-core 0.698.0, which carry the gaps the pilot fixed (below). - attio-cli syncs Attio's people, companies and workspace members into cli-core's store, and has an `api` group of - 108 operations generated from Attio's OpenAPI description. It has no deals, no custom attributes, no writes and no + 107 operations generated from Attio's OpenAPI description. It has no deals, no custom attributes, no writes and no OAuth yet (see "What was not measured"). - hubspot-cli at `253e6c4` and bitrix24-cli at `7e2877e`, counted as in [new-cli-lines](../../engineering/new-cli-lines/README.md), with the same exclusions for bitrix24-cli. @@ -72,16 +72,41 @@ lines keep only the current value of each Attio attribute (Attio keeps old value union (Attio's "token not active" answer), and pattern samples did not match their patterns. Fixing them took cli-base [#642](https://github.com/WireCatLabs/cli-base/pull/642) and cli-core [#3102](https://github.com/WireCatLabs/cli-core/pull/3102), about 1 h 20 min more with reviews, plus the releases. -- **Attio's API description generated with no patch** once cli-base #642 was in (108 operations). Before it, the +- **Attio's API description generated with no patch** once cli-base #642 was in (107 operations; an earlier draft said 108). Before it, the builder needed four patches of the description and three command names. - **15 findings were logged** in the plan: kit bugs, and places where the config cannot say something. The ones that block more of Attio: deals (a status attribute with no fixed stage list and no outcome hook), custom attributes (no per-workspace field list), and OAuth as a second sign-in. +## Deals and the workspace's own attributes (0.1.0) + +The first build left deals out. The owner asked for attio to work like bitrix24-cli: deals and their pipeline in the +local copy, people and companies read live from Attio, and each workspace's own attributes kept. That went into the +first release, `@wirecat/attio-cli` 0.1.0 (`0c33e81`), counted with the same command at `0c33e81`. + +| | first build (`003b809`) | 0.1.0 (`0c33e81`) | +|---|---:|---:| +| Config (`wirecat.config.ts`) | 64 | **126** | +| Hooks (`src/hooks.ts`) | 23 | **8** | +| Source written after `new`, hooks included | 42 | **75** | +| Source lines, comparable | 402 | 341 | +| Tests written after `new` | 98 | 391 | + +- **Deals cost 62 lines of config and no new hook.** The 23-line hook that kept an attribute's current value became + one config option in cli-core (`custom.typed`), and Attio's stage outcomes are the one hook left: a stage Attio + celebrates is won, one titled "Lost" is lost. cli-core [#3114](https://github.com/WireCatLabs/cli-core/pull/3114) + (0.700.0) added flat stages, typed attribute values and a live list for a searched kind. +- **The comparable source fell by 61 lines** because the template's own seeds took over code attio had written by + hand. The wiring grew from 19 to 67 lines: hosts, texts, the live lists of people and companies, and the update check. +- **Most new test lines are coverage, not mapping.** CI's test matrix asked for a test of every command and option + (540 in all); one api test and one CRM test cover them. +- **The live check passed on the owner's workspace**: 12 companies, 6 people, one pipeline with 4 stages, and live + people and companies answering. It held no deals, so deals were checked against the fake only. + ## What was not measured -- Deals, custom attributes and custom objects, writes, OAuth: attio-cli does not have them yet, so the comparison - covers people, companies and users only. hubspot-cli's numbers include deals, leads, activities, owners and - pipelines. -- The live check against a real Attio workspace: it is the owner's and had not run when this was written. +- The first build's comparison covers people, companies and users only; hubspot-cli's numbers include deals, leads, + activities, owners and pipelines. 0.1.0 adds deals and custom attributes, but still no custom objects, writes or + OAuth. +- Deals on real data: the owner's workspace has none, so the deal mapping is checked against the fake only. - Hours a person spent: the owner answered three questions. diff --git a/performance/generation/source-manifest/result.json b/performance/generation/source-manifest/result.json index 46d841e..18eca58 100644 --- a/performance/generation/source-manifest/result.json +++ b/performance/generation/source-manifest/result.json @@ -32,5 +32,20 @@ "test_helpers": 810, "generated": 0, "excluded_source": 2293 + }, + "attio_0_1_0": { + "commit": "0c33e81", + "source": 341, + "tests": 1102, + "test_helpers": 92, + "generated": 98069, + "excluded_source": 0, + "written_since_template": { + "source": 75, + "tests": 391, + "test_helpers": 3 + }, + "config_lines": 126, + "hook_lines": 8 } }