test(docs): compile published Java snippets to catch API drift#266
Merged
Conversation
DocumentationExamplesTest renders hand-kept Java copies of examples, but nothing compiles the literal docs/*.md fences, so a published snippet can drift from the API while the build stays green. DocumentationSnippetCompileTest walks docs/**/*.md and, for each java fence carrying an opt-in marker comment <!-- doc-example: id=... mode=... -->, wraps it per mode (method / members / class) and compiles it in-memory via ToolProvider.getSystemJavaCompiler() against the test classpath. Compiler errors fail the test and name the snippet id + file; a second test guards marker well-formedness, and the run asserts at least one snippet is found so a moved docs folder cannot make the guard pass vacuously. Markers are HTML comments, invisible on GitHub; teaching fragments stay unmarked. Marks the five self-contained snippets in first-document.md (2) and business-templates.md (3). Complements DocumentationExamplesTest, which renders example copies; this guard pins the published markdown to the API.
Make DocumentationSnippetCompileTest verify its own failure path so a regression in the parse/wrap/compile pipeline cannot pass vacuously: - compilerReportsErrorForBrokenSnippet feeds a snippet that references a missing symbol and asserts the error is surfaced and attributed to its id. - knownCanonicalTypeResolvesOnTestClasspath compiles a trivial canonical call, so a classpath failure is distinguishable from a documentation break. Also harden the mechanics: attribute diagnostics to snippets by file-object identity instead of a substring match; lift only the leading run of imports (an import-shaped line inside a body stays put); match the java fence exactly (no `javascript`); create the compiler output dir inside the try; guard sanitized unit-name uniqueness. Drop the unused class mode.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
DocumentationExamplesTestrenders hand-kept Java copies of representative examples, but nothing compiles the literaldocs/*.mdfences. A published snippet can therefore drift from the API — a renamed or removed method silently breaks the documented example while the build stays green.What changed
DocumentationSnippetCompileTest— walksdocs/**/*.mdand, for eachjavafence carrying an opt-in marker comment<!-- doc-example: id=… mode=method|members|class -->, wraps it permodeand compiles it in-memory (ToolProvider.getSystemJavaCompiler()) against the test classpath. Compiler errors fail the test and name the snippetid+ file; warnings are ignored. A second test guards marker well-formedness (unique id, supported mode, a fence actually follows), and the run asserts it found at least one snippet so a moved docs folder can't make the guard pass vacuously.invoicevariable) carry no marker and are left untouched, keeping the guard free of false positives.docs/first-document.md(2) anddocs/templates/business-templates.md(3).Complements
DocumentationExamplesTest(which renders example copies); this guard pins the published markdown itself to the API.Verification
./mvnw -B -ntp test -Dtest='DocumentationSnippetCompileTest,DocumentationExamplesTest,DocumentationCoverageTest,CanonicalSurfaceGuardTest,PackageMapGuardTest,VersionConsistencyGuardTest' -pl .→ 36 tests, 0 failures.[business-invoice — docs/templates/business-templates.md] cannot find symbol: method …, then reverted.Full
verifyruns in CI.