Repository navigation
Split releases per plane; publish each fat jar once; trim embedded cruft - #122
Merged
Merged
Conversation
The vanniktech maven-publish 0.35 plugin adds the shadow jar to the publication under the 'all' classifier, and each module's own afterEvaluate override then re-added the same jar as the main (classifier '') artifact. Both copies were uploaded, byte-identical: measured on Maven Central, 0.28.0 stored ~77 MiB per release of the same bytes twice (jafar-shell and jfr-mcp each ~80 MiB in total, mostly the duplicated fat jar). Central artifacts are immutable, so the already-published releases cannot be corrected going back. Verified: publishToMavenLocal for each module produces exactly one jar artifact (thin main jar, sources, javadoc, pom) and the artifact dump shows three jar entries instead of four; jar contents unchanged for jfr-mcp (38.2 MiB, still bundles the unimi/jetty stack - separate slimming piece).
…ugh the plugin loader The plugin installer downloaded exactly one jar per plugin and silently assumed that jar is dependency-free (true for the JFR backends, whose only dependency is the shell's own classpath). Any plugin that depends on a real library would fail at first use with NoClassDefFoundError, which is exactly what installing a thin LLM backend plugin needs. The installer now collects the dependency graph of the plugin's POM via aether - compile and runtime scopes (test/provided/system stay out of plugin storage), nearest-wins conflict resolution, BOM-managed versions honored - and copies every dependency jar under <pluginVersionDir>/deps/, which the plugin classloader already scans. Backend discovery for the LLM SPI follows the JFR backend registry's pattern: the plugin-aware classloader when the plugin system is initialized, the thread-context loader as the test fallback. The bundled plugin catalog is now actually consulted (PluginRegistry.get used to ignore it - the advertised offline fallback was dead data) and a no-network canInstallOffline exists for ask-time gating; local Maven scanning recognizes llm-* artifacts. Local plugin installs accept both backend SPIs; the PluginManager-singleton tests run in their own JVM (:shell-core:pluginSystemTest) with task-provided fixture directories. Verified: :shell-core:test and :shell-core:pluginSystemTest green (scope/conflict/BOM fixture test, installer end-to-end fixture test, classloader-scan test, discovery-through-plugin-loader test failing on the old discover()); runtime check of the real resolver shape returned 29 jars for com.anthropic:anthropic-java:2.34.0 from Maven Central.
…ing its SDK The jafar-shell release jar carried the whole Anthropic Java SDK plus okhttp/okio/kotlin: the SDK alone outweighed the rest of the jar (9.3 of 38.2 MiB is everything except the SDK). The backend now ships as its own artifact - a thin jar with a real POM (the installer resolves its dependency graph, the previous commit's machinery) and a self-contained jar under the 'all' classifier for air-gapped --install-plugin installs. It is cataloged as 'anthropic' in jfr-shell-plugins.json and published by the release workflow alongside the other artifacts. Selecting it explicitly (set llm.backend = anthropic) installs it on the spot at the first ask, mirroring the JFR backend selection flow; automatic selection never installs anything; llm status lists what is installed. The no-backend remedy no longer points at a module that has never existed (llm-core). Verified: built jafar-shell fat jar = 9.3 MiB with zero anthropic/okhttp/okio/kotlin entries; end-to-end through the built jar: ask with the backend unset installed the plugin (35 jars loaded), the ask then degraded to the credential remedy without any model call, and llm status listed the installed backend; LlmCommandsTest updated (the shell must NOT bundle the backend now) and green; all module tests match the pristine-main baseline failure set by name.
RELEASING.md and doc/agents/Release.md claimed jfr-shell is published to GitHub Packages and that the release workflow triggers a JitPack build - neither exists anywhere in the build or workflow history, and no release ever carried assets. The docs now state the actual channels (Maven Central artifacts + the plugin catalog committed to main) and the complete manual-release command list including llm-anthropic. doc/cli/LlmSetup.md documents the install-on-first-use behavior for the anthropic backend and the air-gapped -all.jar recipe; doc/agents/Llm.md and the changelog tell the same story. Verified: the referenced Gradle tasks exist (parser/tools/jfr-shell/jfr-mcp/llm-anthropic shadowJar + publishAllPublicationsToMavenCentralRepository ran against mavenLocal in the verification runs); doc changes are text-only against the state checked in the previous commits.
hdump-parser and hdump-shell ship 16,204 fastutil classes in every artifact that embeds them (the MCP server jar carried 19.5 MiB of packed fastutil - 51 percent of the jar) while their code uses nine types, all maps, sets and lists over long and int keys. it.unimi.dsi:fastutil-core is that exact subset, published by fastutil itself; for 8.5.12 it is 6.4 MB instead of 23 MB. Every consumer of these modules is unpublished or embeds them, so this only trims embedded copies; the heap parser's own tests plus the full four-format MCP end-to-end suite (heap open/summary/ query/report through the built jar) run green against the core artifact. Verified: :hdump-parser:test, :hdump-shell:test green; jfr-mcp fat jar 38.2 -> 21.1 MiB with all nine used fastutil types present; :jfr-mcp:endToEndTest green (real server, java -jar, all four formats).
…erver The PluginManager singleton built its Maven resolver, plugin registry, installer and update checker eagerly, so every backend discovery loaded the whole aether stack - and the MCP server, which never installs plugins, carried it in its jar (1.9 MiB packed of resolver, HTTP transports, commons and gson dependencies) merely because the backend registry reaches for the plugin classloader through this singleton. The collaborators now load on first registry/install/update use; the resolver stack stays out of the jfr-mcp fat jar, as does the shells' jline terminal stack (0.9 MiB) that the server never loads: the MCP tools use the engine classes - sessions, path evaluators, report generators - which import no jline, and the TUI adapters implementing ShellModule.getCompleter are never initialized there. The MCP jar drops from 38.2 MiB (0.26.0 through 0.29.0 releases) to 18.7 MiB. jline is not referenced from any class the server touches (verified by the shell-level import scan and jdeps over the server classes; the excluded commons-codec remnant is 30 KB). The plugin machinery's laziness is covered by the PluginManager-singleton tests and by the real-artifact flow: a fresh jafar-shell with no backends auto-installs the default JFR backend (5 jars), then an explicitly selected anthropic backend installs at first ask (35 jars), degrading to the credential remedy with no model call. Verified: :shell-core:test, :shell-core:pluginSystemTest, :jfr-mcp:test, :jfr-mcp:endToEndTest green; spotlessCheck green; real-artifact install flow re-run against mavenLocal succeeds.
…lly loads The fastutil-core swap left 16,204 shipped classes for the 17 reachable ones; the jdeps closure of the embedded code's references is the exact set that can ever load - fastutil ships pure collections with no reflection and the heap code invokes it through direct imports only - so the MCP fat jar embeds just that closure, derived at build time rather than from a checked-in list: the derivation re-runs on every classpath change, loads each kept class with initialization suppressed, and closes over superclasses, interfaces, field types and method signatures, so a newly referenced fastutil type can never ship missing; a name absent from the core jar fails the build, not hdump_open. The build-time jdeps walk found what a body-reference-only closure misses twice: jdeps does not report superclass chains at all (a class cannot load without them: NoClassDefFoundError in hdump_open) and does not resolve types appearing only in field or method descriptors (NoClassDefFoundError in hdump_summary). MCP fat jar: 21.1 -> 13.2 MiB; the embedded fastutil copy goes 16,204 classes / 14.6 MiB to the 169-class / 2.1-MiB closure. The E2E harness now includes the server's stderr in the tool-error failure message, matching the other assertions; it is what surfaces the failing class in a NoClassDefFoundError. Verified: :jfr-mcp:test, :jfr-mcp:endToEndTest (all four formats through the trimmed jar, heap workflow included), :shell-core:test, :shell-core:pluginSystemTest, :hdump-parser:test, :hdump-shell:test, :jfr-shell:test, :llm-anthropic:test green; full gate ./gradlew test shadowJar spotlessCheck --continue matches the pristine-main baseline failure set by name; spotlessCheck green.
Every published artifact's javadoc jar was ~90% javadoc's own bundled DejaVu web fonts: the jfr-mcp jar's 4.18 MiB held 3.91 MiB of woff/woff2 for a few hundred HTML pages of a handful of classes, and the 0.1-MB backend plugins carried matching four-miB docs. Generated documentation does not need the fonts shipped: --no-fonts (set for every subproject's Javadoc task; gradle prefixes option names with a single dash, so the name carries the dash of --no-fonts itself) keeps the HTML and the search index and drops the fonts. Per-release Maven Central cost, measured against mavenLocal republish of every artifact: 171.2 MiB -> 53.8 MiB (jafar-shell 80.9 -> 9.9, jfr-mcp 80.5 -> 13.6, backends 8.0 -> 0.3, llm-anthropic new at 29.0 for its self-contained air-gap jar, dedup from the earlier commits). Verified: publishToMavenLocal for jfr-shell, jfr-mcp, llm-anthropic, jfr-shell-jdk, jfr-shell-jafar, jafar-parser, jafar-tool; javadoc jars measured 0.1-0.4 MiB with zero woff entries; :jfr-mcp:javadoc runs the flag with zero woff files in build/docs/javadoc; spotlessCheck green.
…ndle both LLM backends llm-openai (the openai and ollama profiles, one small JDK-HttpClient adapter with no provider SDK) joins llm-anthropic as a standalone artifact; both shells drop their backend runtimeOnly lines, so the release jar bundles no LLM backend at all, just the SPI in shell-core. The catalog gains openai and ollama entries pointing at llm-openai (same artifact, two backends), and the release workflow publishes it next to llm-anthropic. The ask-time install flow is unchanged and now covers all three backend ids; a status with nothing installed tells the user how to install. Fat jars also record Embedded-Modules in their manifest now: the fat jar is a snapshot of its dependencies, so its own version does not say which parser or SPI it runs; the manifest's io.btrace module list (resolved at manifest-merge time, so nothing resolves during configuration) does. unzip -p jar META-INF/MANIFEST.MF | grep Embedded-Modules. Verified: the new shell artifact bundles zero backend classes and 32 SPI classes; real-artifact run: fresh plugin dir, set llm.backend = ollama, as-query prints 'Backend not found / Downloading from Maven repositories... done.' and installs llm-openai with its POM-resolved dependencies (2 jars), then llm status lists ollama READY against the loopback endpoint; a second run executed one completion against the configured Ollama Cloud endpoint (the user's own setting, one request - the query path works end to end); :jfr-shell:test green (tests updated for the no-backend-by-default world), :llm-openai:test, :llm-anthropic:test, :jfr-mcp:test, :jafar-shell:compileJava, :shell-core:pluginSystemTest green; spotlessCheck green.
One version line for the whole repo re-published every artifact at every tag - the LLM plugin's 29 MiB air-gap jar went to Maven Central for a parser-only fix too. The release now splits into planes, each with its own tag shape, version line and publishing scope. The shell plane keeps the original vX.Y.Z tags (its whole history); the other planes follow the Go module's component/vX.Y.Z shape: core/ (jafar-parser, jafar-tools, jafar-gradle-plugin), mcp/ (jfr-mcp), llm-anthropic/, llm-openai/. Coupling rules: self-contained planes release independently and their fat jars record Embedded-Modules in the manifest so the user knows which parser and SPI they got; a release-API breaking shell-core change tags every SPI-consuming plane the same day, each with its own number; the Go module versions with the core plane (the parser-parity rule). gradle/version-from-tag.gradle: the plane map and per-plane derivation are the single source of truth; a plane without tags of its own rides the newest bare vX.Y.Z number until its first tag lands. scripts/derive-version.sh gains --plane as the shell mirror. scripts/release.sh takes the plane as its argument; the shell keeps the release/X.Y._ stabilization-line policy (minor/major from the default branch, patches from the release branch), every other plane tags from the default branch - patches included, no stabilization lines for libraries. The release workflow derives the plane from the tag, publishes only that plane's artifacts, moves only that plane's plugin-catalog and JBang-catalog entries, and tags the Go module on core-plane releases; the release notes match the tag-shaped changelog section with a fallback to the bare version of the pre-split layout. The dead jar-glob in create-release is gone: the checkout builds nothing, so it silently matched nothing on every release ever, and the published channel is Maven Central. Verified: scratch plane tags on a detached head derive per-plane versions (gradle + script); exact-on-tag builds report the release version; release.sh --dry-run rejects wrong-branch combos and derives the right tag per plane; the success paths in release.sh NOT VERIFIED: running them needs the main checkout (the branch is checked out in the user's tree) - the tag/push/release-create flow runs for real on the next release. The full gate (./gradlew test shadowJar spotlessCheck --continue) matches the pristine-main baseline failure set by name; every module's version derives green with the scratch tags deleted
README.md gains a Releasing section: the plane table (tag shapes, what each publishes, the cost that motivates the split), the release command, the Embedded-Modules manifest line, and the version-derivation one-liner; the stale 'v0.11.0' status line becomes the 0.x line it is. doc/agents/Release.md is rewritten as the agent-facing procedure: the plane table, the release steps the agent owes (changelog section keyed by tag, dry-run, release), the coupling rules that matter when an agent edits code (self-contained planes release independently; breaking SPI changes tag every consuming plane; the Go module moves with core; never mass-bump or downgrade catalog entries), the post-release nothing-to-do, the verification owed, and the emergency per-plane manual publish. RELEASING.md's quick steps, changelog example, the automated-behavior list and the catalog-version rule all match the implemented machinery now. Verified: derive-version.sh --plane and release.sh --dry-run run as documented (branch guards fire off-branch as designed, version derivation returns per-plane numbers); docs contain no references to the never-existing GitHub Packages publish or the JitPack trigger; the AGENTS.md map already routes release questions to doc/agents/Release.md
Contributor
Combined JUnit Test Report
HTML Test ReportsRun artifacts: https://github.com/btraceio/jafar/actions/runs/37202374413
|
Groovy cannot pick between the ClassLoader and AccessControlContext overloads when the parent argument is an untyped null on the CI's JVM; the trim task failed there while it ran locally. Cast the parent argument to ClassLoader.
The --no-fonts javadoc flag failed on CI: parser-codegen's javadoc rejects it (mode-dependent; release-8 javadoc runs cannot take it), which broke the mavenLocal publish the plugin tests depend on. Excluding resource-files/fonts/** from javadoc-classed Jar tasks instead keeps the effect (no fonts shipped in any module's documentation) regardless of how javadoc was invoked; the javadoc invocation itself stays untouched. Verified: :parser-codegen:javadoc, :parser-core:javadoc, :jfr-mcp:javadoc, :tools:javadoc green with zero woff entries in the packaged jars; spotlessCheck green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Maven Central storage showed ~180 MiB per release across the month's three releases. Measured root causes, each fixed:
Every shadow-jarred module published its fat jar twice. The vanniktech maven-publish plugin adds the shadow jar under the
allclassifier while each module's ownafterEvaluateoverride re-added the same file as the main artifact - byte-identical copies landed twice per artifact (verified by Central md5). ~77 MiB pure waste per release. Publication now carries exactly one jar per module: thin main + sources + javadoc; theallclassifier survives only where it is a genuinely different artifact.The shell jar carried the whole Anthropic Java SDK (46 of 96 MiB unpacked, dragging okhttp/okio/kotlin). The Anthropic backend now ships as its own artifact, cataloged (
anthropic) and installed at first explicit use - the same "Downloading from Maven repositories..." flow the JFR backends already have;autonever installs anything.jafar-shell: 38.2 -> 9.3 MiB.The OpenAI-compatible backend (openai/ollama, no SDK) also moved out and publishes as
llm-openai; both catalog entries point at it.The MCP server jar was 51% fastutil (16k classes for 9 used types). The heap tools now depend on
fastutil-coreand the embedded copy is trimmed to the exact static closure derived at build time: jdeps over the runtime classpath, then a loading fixpoint (every kept class is loaded and its super/interfaces/field/method types walked) - jdeps alone misses supertypes and descriptor-only types, which two runtime NoClassDefFoundErrors proved. No keep-list ships; the derivation re-runs on every classpath change and fails the build on a missing class. Fat jar: 38.2 -> 13.2 MiB. jline (never referenced by anything the server loads) and the Maven-resolver stack (the plugin machinery now lazy-initializes; a server that never installs plugins never loads it) are excluded too.Javadoc jars were ~90% javadoc's bundled web fonts (3.91 of 4.18 MiB for jfr-mcp).
--no-fontsfor every subproject. Backends went 3.96 -> ~0.13 MiB, which were mostly docs.Releases split into planes, each with its own tag shape, version line and publish set:
vX.Y.Z= shell (jafar-shell + the jfr-shell-* backends),core/= parser+tools+gradle-plugin (also tagsgo-parser/vX.Y.Z),mcp/= jfr-mcp,llm-anthropic/,llm-openai/. Version derivation (gradle/version-from-tag.gradle) is per-plane and stays the single source of truth (root build, all subprojects, the included gradle-plugin build);scripts/derive-version.sh --planeis the shell mirror;scripts/release.sh [plane] <type>applies per-plane branch policy (the shell keeps itsrelease/X.Y._stabilization lines; the other planes tag from the default branch). The release workflow derives the plane from the tag, publishes only that plane's artifacts, moves only that plane'sjfr-shell-plugins.jsonentries (per-plugin jq, never a mass-bump) and JBang-catalog aliases, and tags the Go module on core releases. Fat jars recordEmbedded-Modulesin their manifest so an app's version says which parser/SPI it embedded. Docs: README + doc/agents/Release.md + RELEASING.md describe the whole story.Per-release Maven Central cost, measured: 171.2 -> ~54 MiB.
How verified
./gradlew test shadowJar spotlessCheck --continueat HEADpublishToMavenLocal(throwaway key) for all 7 published artifactsallon llm-anthropic); no duplicatesset llm.backend=ollama,as-queryllm status= READY (loopback):jfr-mcp:endToEndTest./gradlewper module;derive-version.sh --plane;release.sh --dry-run-SNAPSHOT; branch guards fireCoverage notes
The e2e suite exercises all four formats through the real jar (the net for the fastutil trim); the plugin-system tests cover installer dep-resolution, catalog fallback usage, classloader scanning, and discovery through the plugin loader (a test that fails on the old code). The javadoc flag is verified by the generated docs having zero woff entries.
Not verified / known limitations
release.shsuccess paths (only dry-runs; the branch guards require a main/release-branch checkout) | the next real release.jfr-mcp's remaining 13.2 MiB: jetty (the--attachSSE bridge) + the MCP SDK's jackson/reactor/schema stack are by design; gson stays (installed.json parsing is load-bearing).LlmConfigTest's env leak, flakyMcpJfrTransportTestport test - same names on pristine main.Breaking changes / migration
askinstalls a backend at first explicit use (documented in doc/cli/LlmSetup.md, air-gapped recipe included). Users must allow a Maven download on firstask, or install from files.io.btrace:*by exact version: planes version independently now;core/vX.Y.Zno longer implies a same-numbered shell release.scripts/release.shtakes an optional plane argument (shell default preserves old behavior).