Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,13 @@ Entries land here as they merge.

### Internal

- **Benchmark suite cleanup (not shipped).** Removed three redundant
benchmark mains: `FullCvBenchmark` (superseded by the JMH
`TemplateCvJmhBenchmark`), `GraphComposeBenchmark` (early-engine relic
duplicating `CurrentSpeedBenchmark`'s `engine-simple` scenario), and
`ScalabilityBenchmark` (its thread-scaling sweep folded into
`CurrentSpeedBenchmark`'s full-profile throughput run, now `1,2,4,8,16`).
Dropped the matching `run-benchmarks.ps1` steps and doc entries.
- **Removed the `java.awt.*` / `java.util.*` co-wildcard in four files.**
`InvoiceTemplateComposer`, `ProposalTemplateComposer`,
`WeeklyScheduleTemplateComposer`, and the engine `PdfRenderingSystemECS`
Expand Down
6 changes: 1 addition & 5 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,15 +62,11 @@
| File | Role |
|---|---|
| `CurrentSpeedBenchmark` | Default scenario runner — what CI's `perf-smoke` job exercises. Takes a `-Dgraphcompose.benchmark.profile=smoke\|full\|stress` switch. |
| `ComparativeBenchmark` | Renders the same fixtures through GraphCompose, iText, openHTMLToPDF, JasperReports. **Rough local comparison only** — see "When not to use" above. |
| `FullCvBenchmark`, `ScalabilityBenchmark` | Fixture-specific runners for CV and table-heavy scenarios. |
| `CanonicalBenchmarkSupport`, `BenchmarkSupport` | Shared fixture builders + measurement helpers. |
| `ComparativeBenchmark` | Renders the same fixtures through GraphCompose, iText, openHTMLToPDF, JasperReports. **Rough local comparison only** — see "When not to use" above. || `CanonicalBenchmarkSupport`, `BenchmarkSupport` | Shared fixture builders + measurement helpers. |
| `BenchmarkReportWriter` | Writes JSON / CSV / text reports under `benchmarks/target/benchmarks/`. |
| `BenchmarkDiffTool` | Compares two JSON reports and prints a delta table. Useful for pre/post comparisons. |
| `BenchmarkMedianTool` | Median + dispersion across N runs of the same scenario. |
| `GraphComposeStressTest`, `EnduranceTest` | Long-running stress / endurance harnesses. |
| `GraphComposeBenchmark` | Legacy entry point preserved for one downstream caller. New work should target `CurrentSpeedBenchmark`. |

## Running

From the repo root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,9 @@ public final class CurrentSpeedBenchmark {
private static final int DEFAULT_FULL_WARMUP_ITERATIONS = 12;
private static final int DEFAULT_FULL_MEASUREMENT_ITERATIONS = 40;
private static final int DEFAULT_FULL_DOCS_PER_THREAD = 12;
private static final String DEFAULT_FULL_THREAD_COUNTS = "1,2,4,8";
// The 16-thread tier is absorbed from the removed ScalabilityBenchmark so the
// full profile keeps a thread-scaling data point (smoke runs no throughput).
private static final String DEFAULT_FULL_THREAD_COUNTS = "1,2,4,8,16";
// Bumped from 2/5 to 30/100 so smoke runs reach a steady JIT state and the
// p95 calculation actually has enough samples to interpolate rather than
// collapsing to the maximum observed time. The smoke profile remains the
Expand Down

This file was deleted.

This file was deleted.

This file was deleted.

9 changes: 2 additions & 7 deletions docs/operations/benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,10 @@ The script prints numbered sections so you can map console output to the pipelin
1. `01-build-classpath`
Builds the test classpath once and writes `target/benchmark.classpath`.
2. `02-current-speed`
Runs `CurrentSpeedBenchmark` in the selected profile.
Runs `CurrentSpeedBenchmark` in the selected profile. The full profile also
runs the thread-scaling throughput sweep (1 → 16 threads).
3. `03-comparative`
Runs the GraphCompose canonical vs iText 5 vs JasperReports comparison.
4. `04-core-engine`
Runs `GraphComposeBenchmark`.
5. `05-full-cv`
Runs `FullCvBenchmark`.
6. `06-scalability`
Runs the thread-scaling throughput benchmark.
7. `07-stress`
Runs the concurrent stability stress test.
8. `08-endurance`
Expand Down
7 changes: 2 additions & 5 deletions scripts/run-benchmarks.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ Runs the local GraphCompose benchmark pipeline and stores timestamped logs and r

.DESCRIPTION
The wrapper performs a staged local run:
01 build classpath, 02 current-speed, 03 comparative, 04 core engine, 05 full CV, 06 scalability,
07 stress, optional 08 endurance, then 09/10 diff steps.
01 build classpath, 02 current-speed, 03 comparative, 07 stress,
optional 08 endurance, then 09/10 diff steps.

Current-speed diffs are profile-aware. The wrapper only compares reports
from the same current-speed profile (`smoke` or `full`) and skips the
Expand Down Expand Up @@ -368,9 +368,6 @@ try {
-InputPaths $comparativeRuns | Out-Null
}

Invoke-JavaMain -Name "04-core-engine" -Classpath $javaClasspath -MainClass "com.demcha.compose.GraphComposeBenchmark"
Invoke-JavaMain -Name "05-full-cv" -Classpath $javaClasspath -MainClass "com.demcha.compose.FullCvBenchmark"
Invoke-JavaMain -Name "06-scalability" -Classpath $javaClasspath -MainClass "com.demcha.compose.ScalabilityBenchmark"
Invoke-JavaMain -Name "07-stress" -Classpath $javaClasspath -MainClass "com.demcha.compose.GraphComposeStressTest"

if ($IncludeEndurance) {
Expand Down