Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,10 @@ run time, so the same tasks compare any of them.
steps, each a fresh agent session given the orchestrator's own message, then the validator step,
whose final findings a rubric judge grades. One trajectory per step, so you can see each
reviewer come in and out.
- `evals/team-tasks/vo-x1`, `vo-x4`, `vo-x7`, `vo-x8`, `vo-x17` and `vo-x20`: six more voiceover
changes, run the same way. Each key is the pull request's first bot review (Codex or CodeRabbit),
keeping only the findings the reviewed commit confirms on reading. Five of them (all but
vo-x20) were the study's tuning changes, so they are not blind like the six scored ones.
- `evals/cases/cr-self-01`: this repository's own history, the first commit of the CI action and
the 18 confirmed bugs it contained. Can the review tool find the bugs in itself?

Expand Down
34 changes: 32 additions & 2 deletions evals/team-tasks/vo-t01/steps/validator/tests/finish.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 32 additions & 2 deletions evals/team-tasks/vo-t02/steps/validator/tests/finish.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 32 additions & 2 deletions evals/team-tasks/vo-t03/steps/validator/tests/finish.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 32 additions & 2 deletions evals/team-tasks/vo-t04/steps/validator/tests/finish.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 32 additions & 2 deletions evals/team-tasks/vo-t05/steps/validator/tests/finish.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading