Repository navigation
Release 0.26.0 - #1950
Merged
Merged
Release 0.26.0#1950
Conversation
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
0.26.0 adds process Experiments: compare Built-in or Custom Ticket processes on merged work, grade their final commits, and report paired quality, cost and time Verdicts (#1915).
thirdshift triage <Issue URL or number> [base <branch>]investigates and updates an existing issue in place (Support targeted in-place issue triage #1911). With terminal input or empty stdin at EOF,thirdshift triage [base <branch>]selects one eligibleneeds-triageissue, including from cron (Run unattended Triage Passes from terminal or EOF input #1918).thirdshift evals propose-process <repository>drafts tasks from merged Tickets.evals approve process <task>andevals prepare process <task> <destination>approve and screen them: tests must fail at base and pass at merge; preparation refuses changed inline Rust tests and produces a base-only Trial repository (Build and screen process Eval tasks from merged Tickets #1926).process_grading(Grade process Trials with the reference fix's tests #1927, Spec: process Trials and a code-graded implement Eval suite #1915).--pricessupplies frozen, dated token prices for Session, Trial, role and Arm accounting, with paired median cost/time ratios. Missing usage or unpriced Models make a Trial Void (Price Trial sessions by role and report paired cost and time #1932). Opt-in--cost-ceilingand--time-ceilingconstrain Candidate/Baseline ratios; Verdicts add cheaper/faster annotations, and unknown measurements cannot pass a configured ceiling (Add cost and time ceilings and cheaper/faster Experiment Verdicts #1938).ready-for-humanunless permitted, and cleans up Claims and reservations across success, failure, interruption and Stop (Support targeted in-place issue triage #1911, Unattended triage of existing needs-triage issues #1917). Passes honor Command/Failure limits and configured issue order, retaining failed Triage in failure accounting (Run unattended Triage Passes from terminal or EOF input #1918). Sessions receive bounded cached images from trusted GitHub attachments (Pre-download trusted issue images for triage #1922); Report forms honor the selected base branch, and private Security-finding evidence stays out of email (Unattended triage of existing needs-triage issues #1917).Also: site and docs updates (#1917, #1915), plus the 0.25.0 release PR (#1937).
Version diff