test(mcp): opt-in live output schema conformance tests - #53
Conversation
Invoke real tools against the free Microlink endpoint and validate structuredContent against each tool's declared outputSchema. Catches contract drift between schemas and the real API that stubbed unit tests cannot (the null media/content/embed cases all came from live behavior). Skipped unless LIVE=1, so CI and local runs are unaffected. Run them with pnpm test:live. Signed-off-by: Kiko Beats <2096101+Kikobeats@users.noreply.github.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughThe MCP package adds a ChangesMCP live test execution
Priority: ⬇️ Low Estimated code review effort: 1 (Trivial) | ~5 minutes Change: Other Merge Risk: ⚪ Minimal · up to This adds an opt-in command for live schema tests without changing default tests or production behavior. No merge-blocking risk is evident. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (1 skipped: 1 unsupported.)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Coverage Report for CI Build 34702088029Warning No base build found for commit Coverage: 79.358%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsRequires a base build to compare against. How to fix this → Coverage Stats
💛 - Coveralls |
|
@coderabbitai review |
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
What
A new
test/live.test.jsthat invokes real tools against the free Microlink endpoint and validatesstructuredContentagainst each tool's declaredoutputSchema.Motivation: every output schema correction so far (
logo/video/audionull,markdown/html/textnull,embednull,lighthousestring output) came from real API behavior that stubbed unit tests could not catch. These tests make that drift visible on demand.Coverage
One tool per schema family: metadata, markdown, logo, links, screenshot, embed, technologies, extract. Skips slow/keyed tools (
search,function,lighthouse,pdf).Opt-in
Every test is skipped unless
LIVE=1, so CI and local runs are untouched (default suite: 101 pass, 8 skipped). New script:pnpm test:live.Verified just now against the live free endpoint: 8/8 pass.
Note
Low Risk
Test-only, opt-in changes with no production or CI behavior unless
LIVE=1is set.Overview
Adds opt-in live conformance tests for the MCP package so real Microlink API responses can be checked against each tool’s declared
outputSchema.A new
test/live.test.jsregisters all tools via the sametools()hook, invokes handlers against the free endpoint, and uses Zod to validatestructuredContent(with a 120s timeout). Eight cases cover one representative tool per schema family (metadata,markdown,logo,links,screenshot,embed,technologies,extract); everything is skipped unlessLIVE=1, so defaultpnpm testand CI stay unchanged.package.jsongainspnpm test:live(LIVE=1 node --test test/live.test.js) alongside the existing test script.Reviewed by Cursor Bugbot for commit 96c8881. Bugbot is set up for automated code reviews on this repo. Configure here.
Summary by CodeRabbit