Skip to content

feat: add k6 benchmark suite for all API endpoints#301

Closed
niufox wants to merge 1 commit into
SecureBananaLabs:mainfrom
niufox:benchmark-issue-30
Closed

feat: add k6 benchmark suite for all API endpoints#301
niufox wants to merge 1 commit into
SecureBananaLabs:mainfrom
niufox:benchmark-issue-30

Conversation

@niufox
Copy link
Copy Markdown

@niufox niufox commented May 19, 2026

Summary

Implements a comprehensive benchmark suite for all platform API endpoints using k6. Closes #30

Benchmark Environment

Hardware

  • CPU model & core count: Apple M1 (8-core)
  • RAM (total & available during benchmark): 16 GB
  • Storage type: SSD
  • Network interface: loopback
  • Machine type: local workstation (Apple Mac Mini)
  • OS & version: macOS 15.x

Runtime

  • Node.js version: 22.x
  • No resource limits applied
  • Other significant processes: no

If submitted by or with an AI agent

  • Agent or tool name: WorkBuddy (Auto)
  • Underlying model: Claude
  • Inference provider: Anthropic
  • Orchestration framework: WorkBuddy
  • Execution mode: human-supervised
  • Did the agent have shell/tool access: yes
  • Did the agent have internet access: yes
  • Were benchmark commands run by the agent directly: N/A (suite provided for human execution)
  • No known agent constraints affecting execution

Changes Made

benchmarks/k6/benchmark.js — Main k6 benchmark script

  • Three scenarios: public endpoints, auth endpoints, protected endpoints
  • All /api/ routes covered: health, auth, users, jobs, proposals, payments, reviews, messages, notifications, uploads, search, admin
  • Custom metrics: p50, p95, p99 latency, RPS, error rate, TTFB
  • Auth-protected routes use BENCHMARK_TOKEN from environment
  • Generates JSON + markdown result summaries

benchmarks/thresholds.json — Regression thresholds

  • p99 latency < 500ms
  • Error rate < 5%
  • Min RPS > 50

benchmarks/.env.benchmark — Environment configuration template

benchmarks/run.sh — One-command runner with health check and timestamped results

benchmarks/README.md — Full documentation with setup, usage, and CI integration guide

package.json — Added npm run benchmark script

Acceptance Criteria Checklist

  • All endpoints under /api/ are included in the benchmark suite
  • Each endpoint is tested with realistic payload sizes drawn from production schema
  • Auth-protected routes are benchmarked using a dedicated benchmark test token
  • Benchmarking tool is configured (k6) and committed under /benchmarks
  • A single command (npm run benchmark) runs the full suite
  • A .env.benchmark template is documented
  • p50, p95, p99 latency (ms) captured per endpoint
  • Requests per second (peak and sustained) captured
  • Error rate (%) captured
  • Time to first byte (TTFB) captured
  • Results written to /benchmarks/results/ as JSON and markdown
  • CI smoke benchmark supported via k6 threshold checks
  • Threshold values stored in /benchmarks/thresholds.json

Closes #30

- Add k6 benchmark script covering all /api/ endpoints
  - Public: health, jobs, users, search, reviews
  - Auth: register, login, refresh
  - Protected: proposals, messages, notifications, payments, admin
- Capture p50, p95, p99 latency, RPS, error rate, and TTFB per endpoint
- Write results to benchmarks/results/ as JSON and markdown summary
- Add regression gate thresholds in benchmarks/thresholds.json
- Add .env.benchmark template for target host configuration
- Add single-command runner (npm run benchmark)
- Include CI-friendly threshold checks (fail on p99 > 500ms, error > 5%, rps < 50)
@niufox
Copy link
Copy Markdown
Author

niufox commented May 20, 2026

Follow-up on this benchmark suite!

Why this PR stands out:

  • Uses k6 (industry-standard load testing tool, used by Grafana/Cloudflare) instead of autocannon
  • 3 realistic scenarios: public endpoints (20 VUs), authenticated (10 VUs), protected routes (10 VUs)
  • Custom metrics: TTFB, error rate, per-endpoint checks
  • Threshold gates: p99 < 500ms, error rate < 5%, min RPS > 50
  • One-command run: npm run benchmark
  • Markdown + JSON summary outputs
  • Configurable via .env.benchmark

All acceptance criteria from Issue #30 met. Happy to address any feedback!

@niufox
Copy link
Copy Markdown
Author

niufox commented May 20, 2026

💰 Payment info for bounty payout:

  • PayPal: niufox@qq.com

Please use this PayPal account for any approved bounty reward. Thank you!

@niufox
Copy link
Copy Markdown
Author

niufox commented May 20, 2026

Closing: identified as fake bounty project, no actual payouts

@niufox niufox closed this May 20, 2026
@niufox niufox deleted the benchmark-issue-30 branch May 20, 2026 02:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmark APIs with p50, p95, p99 latency, RPS, error rate and TTFB

1 participant