Skip to content

Benchmark the Go prototype against Rust - #1854

Merged
JacobStephens2 merged 2 commits into
issue-1830from
issue-1835
Nov 7, 2026
Merged

JacobStephens2 merged 2 commits into
issue-1830from
issue-1835

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

Summary

Add a repeatable benchmark command and an empirical report for the Go Ticket/Delivery/Pickup slice and its Rust owners. Record cold and edited builds in fresh Worktrees, test compilation/execution, static binary sizes, source manifests and ADR-0017 checks. Explain the feature and model differences; ADR-0001 stays unchanged.

prototype/go/benchmark.py
  fresh detached Worktree + private caches per sample
  Go / Rust builds, compiled tests, model checks, static-link verification
  results.json + command logs
docs/research/go-vs-rust-prototype.md
  measured observations + subprocess, exhaustiveness and verification tradeoffs

Closes #1835

Evidence

Execution evidence

  • Before: python3 -m unittest discover -s prototype/go -p benchmark_test.py -v failed: python3: can't open file '/home/jacob/repos/thirdshift-issue-1835/prototype/go/benchmark.py': [Errno 2] No such file or directory.
    After: the same command passed all four tests in 2.917 seconds.
  • Before: python3 -m unittest discover -s prototype/go -p benchmark_test.py -k PublishedReport -v failed with FileNotFoundError: [Errno 2] No such file or directory: '/home/jacob/repos/thirdshift-issue-1835/docs/research/go-vs-rust-prototype.md'.
    After: the same command passed the report completeness/numeric agreement test.
  • Real experiment: PATH=/usr/local/go/bin:$PATH python3 prototype/go/benchmark.py --ref 82cc4dc2 --samples 2 --jobs 2 --output /tmp/thirdshift-1835-benchmark.PgYbxF completed both samples. Cold builds: Go 9.659–10.197s, Rust 160.736–166.977s; edited builds: Go 0.601–0.631s, Rust 8.887–9.292s. Static binaries verified: Go 3,289,272 bytes; Rust 21,210,280 bytes. Report and scope limitations.
  • Final local selection: PATH=/usr/local/go/bin:$PATH CARGO_BUILD_JOBS=2 RUST_TEST_THREADS=3 /usr/bin/time -p scripts/affected-tests da63d90d5482abcd6de148d579eaaf7f072bf11c passed. Go: 154.72s; full Rust cargo test: 2963.85s, including compilation. Total wall time: 3118.73s. Full logs remain under .affected-tests/.
  • go vet ./..., gofmt -l benchmark_test.go (empty), and python3 bundle-skills.py --check passed from prototype/go with Go on PATH.
  • python3 -m unittest discover -s scripts -p 'test_*.py' passed all 37 tests in 11.639s.
  • ADR-0001: git diff da63d90d5482abcd6de148d579eaaf7f072bf11c -- docs/adr/0001-rust-binary-with-embedded-skills.md is empty.

Test seams

  • Benchmark CLI in a temporary Git repository: external tool subprocesses, JSON/log artifacts, static verification, isolation and cleanup on success/failure.
  • Published report and measured artifact: required comparison topics and every published timing, source count and binary size.

Review coverage

Reviewed against the fetched issue-1830 tip, da63d90d5482abcd6de148d579eaaf7f072bf11c. The local issue-1830 branch is checked out in another Worktree at an older commit, so the fixed point is the captured upstream tip rather than changing that other checkout.

Standards: 0 findings. Spec: 0 findings. Each independent reviewer read all ten changed files and its supporting context. No generated-output exclusions; no changed file was left unread under either axis. Reports with each axis's final files-read list and the count summary are retained in .thirdshift-review-9GMu5v/ beside the Session trace.

Unaddressed findings

Standards

None: the reviewer raised no findings.

Spec

None: the reviewer raised no findings.

Security

None: the reviewer raised no findings.

Security review outcome

No unaddressed introduced findings.

Merge Danger

Door: two-way

Blast Radius: prototype

Adds opt-in benchmark tooling, evidence and tests. Rust build/size results include the larger production executable, the test workloads differ, and the two bounded model graphs differ; the report makes those limits explicit. No production language decision is made here.

Built with codex codex-cli 0.164.0 · gpt-6.1-sol · xhigh

@JacobStephens2
JacobStephens2 merged commit 72a77a1 into issue-1830 Nov 7, 2026
5 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1835 branch November 7, 2026 01:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant