Skip to content

fix(mcp): make BTrace MCP deployment work and test it end to end - #6

Merged
jbachorik merged 6 commits into
agent/jfr-perf-pluginsfrom
agent/move-btrace-mcp
Oct 2, 2026
Merged

jbachorik merged 6 commits into
agent/jfr-perf-pluginsfrom
agent/move-btrace-mcp

Conversation

@jbachorik

@jbachorik jbachorik commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

What & why

Running the BTrace MCP server against a real JVM showed that most operations were broken. The unit tests had missed this because they replace the BTrace client with fakes. This PR fixes the MCP-side defects, adds end-to-end tests that run the server against a live sample JVM (no LLM involved), runs those tests in CI against a BTrace build, and corrects the skill guidance.

Server fixes (5cba3ee)

  • Attach failed with No masked btrace.jar found. The BTrace client finds its agent through btrace.jar.path or a classpath entry named btrace.jar, but JBang supplies btrace-<version>.jar. The server now sets btrace.jar.path from the jar it has already located.
  • A successful deploy never returned. The client's submit runs the probe command loop until the probe exits. Deploys now submit on a daemon thread, stop buffering output after the status reply, and close the client when deployment times out or fails.
  • deploy_script rejected public classes. It named the source BTraceScript_<timestamp>.java, so public classes failed to compile and every other script fell back to BTrace short syntax, which allows no fields and no @Export. The file is now named after the first declared class.

Codex (212707d): the Codex manifest declared only skills, so Codex installs had no MCP server. Codex's plugin MCP format does not expand a plugin-root placeholder in args (checked in codex-rs plugin_config.rs), but it resolves a relative cwd against the plugin root, so the server is launched with cwd: ".". The file sits under .codex-plugin/ so Claude Code does not also load it as a root .mcp.json.

End-to-end tests (0037366, d0802cb): the new ./gradlew e2eTest drives every tool and prompt against a sample app.

  • No tool returns probe output after deployment, so the test checks probe effects through @Export counters read with jcmd: instrumentation fires, send_event runs its handler, a detached probe keeps running, and exit_probe stops it.
  • By default the server is launched through JBang, as the manifests do. With BTRACE_JAR set, it runs on a plain classpath against that jar.
  • The sample app lives outside io.btrace, which BTrace does not instrument, and the agent uses a free port.

CI (cb5b89b): the new MCP e2e workflow builds btraceio/btrace (default branch, or a ref given on manual dispatch) with :btrace-dist:btraceJar and runs e2eTest against the resulting jar. It never depends on the unpublished io.btrace:btrace:3.0.0-SNAPSHOT artifact.

Skills (9cff5dc): agents are told not to call list_probes (see limitations), and the skill no longer promises output inspection.

Rejected alternatives:

  • Codex $schema Agent Plugins MCP format: it does support ${PLUGIN_ROOT}, but it isn't needed, and the cwd form is verified.
  • Linking the e2e job to a published BTrace artifact: none exists. Building from source also tests against current BTrace.
  • Running the server's BTrace client in a separate process to avoid System.exit: deferred. The fix belongs in the BTrace client.

How verified

Gate Result
./gradlew clean check (mcp) BUILD SUCCESSFUL, 17 unit tests passed, 80% line-coverage rule passed
./gradlew e2eTest (JBang launch) 9 passed, 1 skipped (list_probes)
BTRACE_JAR=<fresh btrace develop ee8434e build> ./gradlew e2eTest 9 passed, 1 skipped, no JBang in server log
e2e with the jar-path fix disabled 5 failures (the suite catches the defect)
BTRACE_JAR=<directory> / blank BTRACE_JAR fails with BTrace JAR not found / falls back to JBang
codex mcp get btrace --json (isolated CODEX_HOME, plugin installed from this repo) cwd resolves to the installed plugin root; that command answers initialize and lists 7 tools
MCP e2e workflow on GitHub (first run, this PR) BTrace build, jar lookup and e2eTest steps all succeeded
actionlint .github/workflows/mcp-e2e.yml no findings
Workflow steps reproduced locally (fresh clone, :btrace-dist:btraceJar, jar lookup, e2eTest) all pass
scripts/validate-marketplace.sh on this branch merged into origin/main Repository consistency valid: 4 plugins, 22 skills, Marketplace validation passed.

Coverage notes

The e2e suite covers all 7 tools and 3 prompts on one sample JVM, including the session lifecycle: deploy, event, detach and exit. It does not cover a failed probe start, because the BTrace client still calls System.exit(1) there. It does not cover agents choosing these tools; LLM evals stay manual and are not part of CI.

Not verified / known limitations

  • Codex has not launched the server inside a real (logged-in) session. Only the resolved launch configuration was checked.
  • list_probes still terminates the server. Client.connectAndListProbes in BTrace calls System.exit(0), and submit calls System.exit(1) on failure. The fix belongs in btraceio/btrace; the e2e case is @Disabled and the skill warns against the tool until then.
  • The e2e sample app runs on Java 11, which BTrace now warns is deprecated for targets.
  • Workflow action pinning differs from validate.yml. The new workflow pins actions to commit SHAs (the same ones BTrace's CI uses), while validate.yml uses major tags.

Breaking changes / migration

None. deploy_script now uses the script's class name as the source file name; scripts that relied on short syntax still work.

Links

Depends on #5 (base of the stack). Retarget to main after #5 merges.

🤖 Generated with Claude Code

jbachorik and others added 6 commits October 2, 2026 22:51
The Codex manifest only declared skills, so Codex installs never got the
BTrace MCP server. Codex's plugin MCP format does not expand a plugin-root
placeholder in args but resolves a relative cwd against the installed
plugin root, so the server is launched with cwd "." and a relative script
path. The file lives under .codex-plugin/ so Claude Code does not also
auto-load it as a root .mcp.json.

Verified: `codex mcp get btrace --json` in an isolated CODEX_HOME resolves
cwd to the installed plugin root; running that command from there answers
initialize and lists all 7 tools; scripts/validate-marketplace.sh passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three defects made every deployment fail against a real JVM:

- Attach failed with "No masked btrace.jar found": the BTrace client finds
  its agent through btrace.jar.path or a classpath entry named btrace.jar,
  but JBang supplies btrace-<version>.jar. The server now exports the jar
  it already located as btrace.jar.path.
- A successful deploy never returned: the client's submit runs the probe
  command loop until the probe exits. Deploys now submit on a daemon
  thread, stop buffering output after the status reply, and close the
  client when deployment times out or fails.
- deploy_script named the source BTraceScript_<timestamp>.java, so public
  classes failed to compile and other scripts fell back to short syntax
  (no fields, no @export). The file is now named after the first declared
  class.

The fake test Client's submit now blocks until close, as the real one
does, so unit tests catch a blocking deploy.

Verified: ./gradlew clean check (17 tests, coverage rule) and
./gradlew e2eTest (9 passed, list_probes skipped) pass; with the jar-path
fix disabled, e2eTest fails 5 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unit tests replace the BTrace client with fakes, so attach, deployment and
lifecycle defects went unnoticed. The new e2eTest task launches the server
through JBang, as the plugin manifests do, against a sample app and
exercises every tool and prompt. No tool returns probe output after
deployment, so probe effects are checked through @export counters read
with jcmd: instrumentation fires, send_event runs the handler, a detached
probe keeps running, and exit_probe stops it.

The sample app lives outside io.btrace, which BTrace does not instrument,
and the agent uses a free port so other BTrace sessions on the host do
not interfere. list_probes is disabled because the BTrace client calls
System.exit after listing probes. The task needs jbang and a resolvable
io.btrace:btrace artifact, so it is not part of check.

Verified: ./gradlew clean check e2eTest → BUILD SUCCESSFUL, e2eTest 9
passed, 1 skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
list_probes terminates the MCP server (the BTrace client calls
System.exit after listing probes), dropping every open probe session. The
btrace-mcp-operations skill now tells agents not to call it and to track
deployments themselves. It also no longer promises output inspection:
tools return only a probe's initial output, so it points at @export
counters read with jcmd for ongoing results.

Verified: scripts/validate-marketplace.sh passes; the list_probes
behavior is reproduced by the disabled e2eTest case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The e2e suite could only launch the server through JBang, which resolves
io.btrace:btrace:3.0.0-SNAPSHOT; that artifact exists only in a local
Maven repository, so CI could not run the suite against its own BTrace
build. Setting BTRACE_JAR (or -PbtraceJar) to a masked btrace.jar now runs
the compiled server on a plain classpath with that JAR. Without it, the
JBang launch used by the plugin manifests is still exercised. A blank
value counts as unset, and a path that is not a file fails fast.

Verified: BTRACE_JAR=<btrace-dist btrace.jar> ./gradlew e2eTest → 9
passed, 1 skipped, no JBang in the server log; blank BTRACE_JAR → JBang
launch (fails without jbang on PATH, passes with it); directory path →
"BTrace JAR not found"; ./gradlew clean check --configuration-cache →
BUILD SUCCESSFUL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The MCP e2e suite needs a BTrace distribution, and the
io.btrace:btrace:3.0.0-SNAPSHOT artifact JBang resolves is not published.
The job builds btraceio/btrace (default branch, or a ref given on manual
dispatch) with :btrace-dist:btraceJar, which skips BTrace's own tests,
and runs ./gradlew e2eTest with BTRACE_JAR pointing at the resulting
masked jar, so it works with whatever version that build produces. No LLM
is involved. Logs and reports are uploaded on failure.

Verified: actionlint reports no findings; the job's steps reproduced
locally against a fresh clone of btrace develop (ee8434e):
:btrace-dist:btraceJar → BUILD SUCCESSFUL, jar lookup finds exactly one
jar, e2eTest → 9 passed, 1 skipped. Not yet run on a GitHub runner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jbachorik
jbachorik force-pushed the agent/move-btrace-mcp branch from fc07b1a to cb5b89b Compare October 2, 2026 20:52
@jbachorik
jbachorik marked this pull request as ready for review October 2, 2026 20:56
@jbachorik
jbachorik merged commit af7a147 into agent/jfr-perf-plugins Oct 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant