Skip to content

Correct the record: group_membership does export - #27

Merged
marcomoauro merged 1 commit into
mainfrom
fix/group-membership-is-exportable
Sep 3, 2026
Merged

Correct the record: group_membership does export#27
marcomoauro merged 1 commit into
mainfrom
fix/group-membership-is-exportable

Conversation

@marcomoauro

Copy link
Copy Markdown
Owner

Follow-up to #26, found while verifying it live.

CLAUDE.md, the tool's own description and a passing test all claimed two columns cannot be exported. Measured live 2026-09-03 against a real publication:

requested : 48
returned  : 47
missing   : ['tag_ids']
group_membership present: true      (value "None")

A narrower probe confirms it directly: requesting ['group_membership','tag_ids','activity_rating'] returns ['activity_rating','group_membership'] with missing_columns: ['tag_ids'].

Whether Substack changed or the original reading was wrong cannot be told apart after the fact. That is the argument for what the tool already does — diff the returned header against the requested list, rather than carry a static list of undeliverable columns. That mechanism is correct and untouched; only the claims about it were wrong, and the missing_columns guard is still needed for tag_ids.

Why the description mattered most

src/server.js is where a model reads whether the data it needs is obtainable at all, before it ever sees the schema. Telling it group_membership cannot be exported means it stops asking for a column that arrives fine — the same failure mode as the upload_image pitch that still said "from an http(s) URL" after path was added.

How it survived

The suite's CSV fixture omitted the Group membership header, so the false claim had a green test proving it. That is the same shape as the pending-poll mock in #26, and precisely what the fixture rule added there exists to catch:

A fixture that represents an API state must cite where that state was observed.

The fixture now carries the header, the value None as returned, and the date of the measurement.

Verification

  • Live measurement above, twice, at two different column widths.
  • Mutation-checked: removing Group membership from the fixture header again fails 3 tests (reports the columns it actually got back, names the requested columns that never came back, records each step of the flow). Confirmed the mutation landed before trusting it.
  • 753 pass / 0 fail on both supported runtimes.
  • tools/list re-read from the real entrypoint: the published description now ends "One column cannot be exported and is reported in missing_columns rather than failing: tag_ids."

🤖 Generated with Claude Code

`CLAUDE.md`, the tool's own description and a passing test all claimed that two
columns cannot be exported. Measured live 2026-09-03: all 48 columns requested
come back as **47**, the only missing one is `tag_ids`, and `group_membership`
carries the value `None`.

Whether Substack changed or the original reading was wrong cannot be told apart
after the fact, which is the argument for the tool diffing the returned header
against the requested list rather than carrying a list of undeliverable
columns. That mechanism was right and is untouched; only the claims about it
were wrong.

The description in the `tools` registry is the one that mattered most: it is
what a model reads to decide whether the column it needs is obtainable, and it
was telling every caller to stop asking for data that arrives.

Note how it survived: the suite's CSV fixture omitted the `Group membership`
header, so the false claim had a green test agreeing with it — the same shape as
the pending-poll mock in #26, and exactly what the fixture rule added there is
for. The fixture now carries the header and cites the measurement, and the
mutation check confirms three tests fail without it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@marcomoauro
marcomoauro merged commit 0fe8540 into main Sep 3, 2026
2 checks passed
@marcomoauro
marcomoauro deleted the fix/group-membership-is-exportable branch September 3, 2026 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant