Enhance gas benchmark gates and dashboard#406
Open
TUPM96 wants to merge 1 commit into
Open
Conversation
|
@TUPM96 is attempting to deploy a commit to the smartdevs17's projects Team on Vercel. A member of the Team first needs to authorize it. |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a gas benchmark reporting + gating layer (coverage, regression, history) and integrates it into local tooling and CI.
Changes:
- Introduces a Python report generator that validates results, produces a Markdown dashboard, and appends historical trend data.
- Adds a required-operations coverage list and a seed history log for long-term trend tracking.
- Updates the local runner script and GitHub Actions workflow to generate/upload the dashboard and enforce coverage/regression gates.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
stellar-lend/benchmarks/public-functions.json |
Defines the required benchmark operations (“coverage”) used as a CI gate. |
stellar-lend/benchmarks/history.jsonl |
Seeds an initial historical trend entry. |
stellar-lend/benchmarks/README.md |
Documents coverage + regression gates and new dashboard/history artifacts. |
scripts/gas_benchmark_report.py |
New validator/dashboard generator for coverage/budget/regressions + history output. |
run-benchmarks.sh |
Updates local runner to generate dashboard/history and optionally enforce regression gate. |
.github/workflows/gas-benchmarks.yml |
Updates CI to run the report script, publish dashboard to step summary, and upload new artifacts. |
.gitattributes |
Enforces LF line endings for common repo file types. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+118
to
+125
| def cross_contract_results(results: list[dict[str, Any]]) -> list[dict[str, Any]]: | ||
| markers = ("cross", "bridge", "amm", "flash_loan", "callback", "auto_swap") | ||
| selected = [] | ||
| for item in results: | ||
| haystack = " ".join([item["operation"], item["description"], *item["tags"]]).lower() | ||
| if any(marker in haystack for marker in markers): | ||
| selected.append(item) | ||
| return selected |
Comment on lines
+15
to
+19
| def load_json(path: Path, default: Any) -> Any: | ||
| if not path or not path.exists(): | ||
| return default | ||
| with path.open("r", encoding="utf-8") as handle: | ||
| return json.load(handle) |
Comment on lines
+128
to
+135
| def history_entry(payload: dict[str, Any], results: list[dict[str, Any]]) -> dict[str, Any]: | ||
| instructions = [item["instructions"] for item in results] | ||
| return { | ||
| "timestamp": payload.get("timestamp") or datetime.now(timezone.utc).isoformat(), | ||
| "source": "gas-benchmark-report", | ||
| "total_benchmarks": len(results), | ||
| "passed": len(results) - len(budget_failures(results)), | ||
| "failed": len(budget_failures(results)), |
Comment on lines
+134
to
+135
| "passed": len(results) - len(budget_failures(results)), | ||
| "failed": len(budget_failures(results)), |
Comment on lines
+36
to
+38
| BENCHMARK_RESULTS: stellar-lend/benchmark-results.json | ||
| BENCHMARK_DASHBOARD: stellar-lend/benchmark-dashboard.md | ||
| BENCHMARK_HISTORY: stellar-lend/benchmark-history.jsonl |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #350
Summary
scripts/gas_benchmark_report.pyto generate a Markdown gas dashboard, track historical trend entries, summarize storage read/write usage, highlight cross-contract/integration calls, enforce hard gas budgets, and fail on >10% regressions against the baseline.run-benchmarks.shrunner and benchmark docs so the same gates can be run outside CI.Validation
python -m py_compile scripts/gas_benchmark_report.pybash -n run-benchmarks.shpython scripts/gas_benchmark_report.py --results stellar-lend/benchmarks/gas-baseline.json --dashboard <temp-dashboard> --history stellar-lend/benchmarks/history.jsonl --history-out <temp-history> --max-regression-pct 10public-functions.jsonand verified coverage/budget/regression gates pass against it.git diff --cached --checkNote
cargo check -p stellarlend-benchmarkscurrently fails before reaching this patch because the base contract crates do not compile cleanly: lending is missing module exports forinterest_rate/risk_monitor, and hello-world has duplicate definitions plus stale test/client references. This PR keeps the scope on the benchmark framework and CI gate requested in #350.