ci: add MCP SDK compatibility workflow - #272
Conversation
Add a dedicated CI workflow (mcp-compat.yml) that tests MCP integration against both v1.x and v2.x SDK lines weekly and on manual dispatch, following the rosetta-compat.yml pattern with auto-issue filing on failure. Remove wall-clock elapsed assertions from timeout tests — the ErrorResult and "timed out" message checks already verify correctness without depending on execution speed, which caused flaky failures under load.
There was a problem hiding this comment.
LGTM 👍
Workflow (mcp-compat.yml):
- Matrix strategy with
fail-fast: falseensures both SDK lines get tested independently - Version resolution logic is solid — validates format, correctly skips mismatched matrix legs on manual dispatch
- Issue deduplication via
v${line}.xtitle match prevents spam; comment-on-existing is a nice touch - The v1 extra deps (
httpx>=0.28.1,websockets) handling looks correct
Test cleanup:
- Good call removing the
elapsed < 4.0assertions — wall-clock checks are inherently flaky under CI load - The remaining
ErrorResult+"timed out"checks are sufficient to verify timeout correctness
One minor note: the cron schedule 41 4 * * 3 (Wed 04:41 UTC) is fine, though if you want separation from rosetta-compat, maybe stagger by a few hours. Not blocking.
There was a problem hiding this comment.
Clean PR — workflow follows the rosetta-compat.yml pattern correctly, and the timeout test cleanup removes the right assertions.
Workflow (mcp-compat.yml):
- Cron timing, matrix with
fail-fast: false, version resolution with validation, artifact upload, and auto-issue deduplication all look solid. httpx>=0.28.1+websocketsfor v1 line only makes sense if v2 bundles these differently.
Test changes:
- Removing
elapsed < 4.0is correct. Wall-clock assertions are inherently flaky under CI load; theErrorResult+"timed out"checks already prove correctness.
Minor flags:
-
Issue title collision risk (L118-119) — dedup matches
v${line}.xsubstring in title. If someone manually edits the title to remove that substring, a second issue gets created on the next failure. Could add a hidden HTML comment marker in the body and match that instead, but this is an edge case. -
Version resolution when v2 not yet published —
mcp>=2.0.0,<3would fail if the v2 line doesn't exist on PyPI yet. If this is expected (v2 exists), no issue. If v2 is future-facing, the step should tolerate "no matching distribution" gracefully.
Neither is blocking — approve once CI is green.
Summary
.github/workflows/mcp-compat.yml— a dedicated CI workflow that tests MCP integration against both v1.x and v2.x SDK lines, with weekly schedule (Wednesday 04:41 UTC) and manual dispatch with optional version overriderosetta-compat.ymlpattern: matrix testing, per-leg artifact upload, auto-issue filing withmcp-compatlabel on failureelapsed < 4.0assertions from 9 timeout tests — theErrorResult+"timed out"checks already verify correctness without depending on execution speedTest plan
mcp-compatGitHub label createdgh workflow run mcp-compat.yml