test(web): fail the build when marketing copy drifts from shipped providers - #126
Merged
Conversation
…viders The last three PRs all fixed the same class of bug. /credential-sharing and README.md are hand-written copy; the providers they advertise are a real registry in @logicsrc/plugin-credential-sharing. Nothing connected the two, so the `team` provider shipped on 2026-07-13 and three weeks later both surfaces still described a five-provider tool with no mention of teams. The docs were right the whole time -- only the pages people actually land on had gone stale, which is worse, because it reads as "the product cannot do this" rather than as a documentation gap. Assert it instead. For every provider in the registry, the Credential Sharing section and the README must say something that counts as advertising it. The registry's own `name` cannot be the proof -- `env` is "Local .env file" and `team` is "LogicSRC Team Vault", neither of which is how the copy reads -- so each provider declares its own pattern, and a provider with no declaration fails too. That way adding a provider forces a deliberate answer about the customer-facing copy. Verified against the bug it is meant to catch: reverting the team copy reproduces "These providers ship but /credential-sharing never mentions them: team", and reverting the README line reproduces the same for sh1pt and team. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
vu1nz Security Review0 finding(s) in PR #? No security issues found. |
ThreatCrush Security Scan55 finding(s) HIGH/CRITICAL: 30 | MEDIUM: 25
…and 5 more. Full results in the Security tab. Snippets are redacted; ThreatCrush never prints matched credential material. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The structural fix behind #123, #124, and #125.
Why
Those three PRs all fixed one bug wearing different hats.
/credential-sharingandREADME.mdare hand-written copy; the providers they advertise are a real registry in@logicsrc/plugin-credential-sharing. Nothing connected the two.So the
teamprovider shipped 2026-07-13 and three weeks later both surfaces still described a five-provider tool that never said the word "team". The docs were correct the entire time — only the pages people actually land on had gone stale. That is the worse failure: a reader concludes the product cannot do this, not the docs are thin. It cost a real "does this even support teams yet?"Reviews do not catch this, because the diff that ships a provider looks complete on its own. So assert it.
What
For every provider in the registry, the Credential Sharing section and the README must contain something that counts as advertising it.
The registry's own
namecannot serve as the proof —envis"Local .env file"andteamis"LogicSRC Team Vault", neither of which is how marketing copy reads. So each provider declares its own pattern inMARKETING_PROOF, and a provider with no declaration fails too. Adding a provider therefore forces a deliberate answer about customer-facing copy rather than allowing silence.Three assertions:
Assertion 2 scopes the match to the
<section id="credential-sharing">band, so an unrelated mention elsewhere on the page cannot satisfy it.Verification
Confirmed against the actual bug, not just confirmed green. Reverting the copy to its pre-#123 state:
And removing a
MARKETING_PROOFentry, standing in for a newly added provider:Green state: 74 contract tests pass (71 + 3), and
next buildcompiles clean. The one Turbopack NFT warning is pre-existing — I verified it appears without this change too.Adds
@logicsrc/plugin-credential-sharingas a devDependency of the web app so the test reads the real registry rather than a regex over source. Lockfile diff is that one line.Scope
This covers credential providers, because that is where a machine-readable registry exists to check against. It is deliberately not a general "docs must match marketing" rule — that would be noisy and unenforceable. Other capabilities can be added the same way as registries appear.
🤖 Generated with Claude Code