Skip to content

Restore the GitHits A/B rehearsal and the pilot rehearsal - #178

Draft
sykosyber wants to merge 1 commit into
mainfrom
claude/restore-githits-ab-rehearsal
Draft

sykosyber wants to merge 1 commit into
mainfrom
claude/restore-githits-ab-rehearsal

Conversation

@sykosyber

Copy link
Copy Markdown
Member

Admission

Class: R (reversible/internal)
Agent-platform: claude
Reviewed-by: none recorded
Admitted-by: pending, held as a draft until Mateo has looked at it
Evidence: tested

Objective

Bring back examples/githits_ab.py and examples/pilot_rehearsal.py, which #166 removed, ahead of the GitHits pilot runs.

Before: main had no GitHits A/B rehearsal or pilot rehearsal, while docs/EXPERIMENTS.md, docs/GITHITS.md, docs/PAID_PROBLEM.md and docs/ROADMAP_STATUS.md still told readers to run them.

After: both scripts are back on main with the tests that covered them, and the docs point at files that exist again.

How: the two scripts and tests/test_pilot_rehearsal.py are checked out from the parent of #166, and the Rehearsal case (with its imports) is put back in tests/test_experiment_githits_api.py by reversing that file's hunk from #166. Nothing else in #166 is reverted.

What changed

What did not change

Evidence

PYTHONPATH=. python3 -m unittest tests.test_pilot_rehearsal tests.test_experiment_githits_api
Ran 15 tests in 13.443s
OK

PYTHONPATH=. python3 examples/githits_ab.py
Rule (accepted and reviewed = delivered): not met. Delivered 6 on vs 5 off ... (needs_lookup: met)

PYTHONPATH=. python3 examples/pilot_rehearsal.py
pilot rehearsal customer=None price=None license=None role=actor_role_missing ... chain=True

The full suite did not finish inside a 10 minute local limit; CI runs it on this PR.

Known limitations

  • tests/durations.json has no entries for the restored tests, so the shard runner uses the median for them.

Generated by Claude Code

…166

examples/githits_ab.py and examples/pilot_rehearsal.py come back with the tests that covered them
(tests/test_pilot_rehearsal.py and the Rehearsal case in tests/test_experiment_githits_api.py).
docs/EXPERIMENTS.md, docs/GITHITS.md, docs/PAID_PROBLEM.md and docs/ROADMAP_STATUS.md still point at both.
The counted GitHits pilot runs from the pinned commit b298cb6, which already has both files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants