Skip to content

ci: add MCP SDK compatibility workflow - #272

Merged
Oaklight merged 1 commit into
masterfrom
worktree-ci+mcp-compat
Sep 24, 2026
Merged

Oaklight merged 1 commit into
masterfrom
worktree-ci+mcp-compat

Conversation

@Oaklight

Copy link
Copy Markdown
Owner

Summary

  • Add .github/workflows/mcp-compat.yml — a dedicated CI workflow that tests MCP integration against both v1.x and v2.x SDK lines, with weekly schedule (Wednesday 04:41 UTC) and manual dispatch with optional version override
  • Follows the rosetta-compat.yml pattern: matrix testing, per-leg artifact upload, auto-issue filing with mcp-compat label on failure
  • Remove flaky wall-clock elapsed < 4.0 assertions from 9 timeout tests — the ErrorResult + "timed out" checks already verify correctness without depending on execution speed

Test plan

  • All 9 timeout tests pass locally after removing elapsed assertions
  • Full MCP test suite (113 tests) passes locally
  • mcp-compat GitHub label created
  • CI passes on this PR
  • Manual workflow dispatch works: gh workflow run mcp-compat.yml

Add a dedicated CI workflow (mcp-compat.yml) that tests MCP integration
against both v1.x and v2.x SDK lines weekly and on manual dispatch,
following the rosetta-compat.yml pattern with auto-issue filing on failure.

Remove wall-clock elapsed assertions from timeout tests — the ErrorResult
and "timed out" message checks already verify correctness without
depending on execution speed, which caused flaky failures under load.

@elena-oaklight elena-oaklight Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Workflow (mcp-compat.yml):

  • Matrix strategy with fail-fast: false ensures both SDK lines get tested independently
  • Version resolution logic is solid — validates format, correctly skips mismatched matrix legs on manual dispatch
  • Issue deduplication via v${line}.x title match prevents spam; comment-on-existing is a nice touch
  • The v1 extra deps (httpx>=0.28.1, websockets) handling looks correct

Test cleanup:

  • Good call removing the elapsed < 4.0 assertions — wall-clock checks are inherently flaky under CI load
  • The remaining ErrorResult + "timed out" checks are sufficient to verify timeout correctness

One minor note: the cron schedule 41 4 * * 3 (Wed 04:41 UTC) is fine, though if you want separation from rosetta-compat, maybe stagger by a few hours. Not blocking.

@clementine-oaklight clementine-oaklight Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean PR — workflow follows the rosetta-compat.yml pattern correctly, and the timeout test cleanup removes the right assertions.

Workflow (mcp-compat.yml):

  • Cron timing, matrix with fail-fast: false, version resolution with validation, artifact upload, and auto-issue deduplication all look solid.
  • httpx>=0.28.1 + websockets for v1 line only makes sense if v2 bundles these differently.

Test changes:

  • Removing elapsed < 4.0 is correct. Wall-clock assertions are inherently flaky under CI load; the ErrorResult + "timed out" checks already prove correctness.

Minor flags:

  1. Issue title collision risk (L118-119) — dedup matches v${line}.x substring in title. If someone manually edits the title to remove that substring, a second issue gets created on the next failure. Could add a hidden HTML comment marker in the body and match that instead, but this is an edge case.

  2. Version resolution when v2 not yet published — mcp>=2.0.0,<3 would fail if the v2 line doesn't exist on PyPI yet. If this is expected (v2 exists), no issue. If v2 is future-facing, the step should tolerate "no matching distribution" gracefully.

Neither is blocking — approve once CI is green.

@Oaklight
Oaklight merged commit 2956cab into master Sep 24, 2026
3 checks passed
@Oaklight
Oaklight deleted the worktree-ci+mcp-compat branch September 24, 2026 03:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant