release: v0.3.8 runtime reliability - #31
Conversation
📝 WalkthroughWalkthroughThe release adds run-level workflow token budgets, paginated subagent results, steering-input recovery, counted compaction replay, locked event-sequence reservations, expanded TUI status reporting, and v0.3.8 release metadata. ChangesWorkflow token budgets
Runtime persistence and paginated results
TUI task and workflow reporting
v0.3.8 release surface
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant WorkflowDraftAction
participant WorkflowRunner
participant WorkflowExecutionGate
participant ChildAgent
participant WorkflowOutput
WorkflowDraftAction->>WorkflowRunner: launch with tokenBudget
WorkflowRunner->>WorkflowExecutionGate: initialize budget and resumed spend
WorkflowRunner->>WorkflowExecutionGate: reserve child-agent capacity
WorkflowExecutionGate->>ChildAgent: grant capped token allowance
ChildAgent-->>WorkflowExecutionGate: settle token usage
WorkflowRunner->>WorkflowOutput: report total, spent, and remaining
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (1)
site/src/shared.ts (1)
7-14: 🎯 Functional Correctness | 🔵 TrivialVerify release sequencing before publishing the
latestentry.The review date is August 6, 2026, but Line 12 declares August 7, 2026 and Line 13 links to the v0.3.8 release. That release currently returns 404, while the public changelog still lists v0.2.50 as latest. ()
If the site deploys before the GitHub tag exists, users will see a future-dated latest entry with a broken release-notes link. Create the release first, or gate the site metadata, sitemap, and roadmap deployment on the same release step.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@site/src/shared.ts` around lines 7 - 14, Ensure the v0.3.8 GitHub release and tag exist before publishing the metadata in releaseVersion and releases. If the release cannot be created first, gate the site metadata, sitemap, and roadmap deployment on that release step so the latest entry is neither future-dated nor linked to an unavailable URL.Source: MCP tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/orca-runtime/src/workflow/runner.rs`:
- Around line 474-478: Update the token budget snapshot construction in
execute_prepared to use the resumed run’s recorded prior child usage instead of
hardcoding spent as 0, while preserving the existing behavior for fresh
launches. Add a test covering a resumed background launch that verifies the
reported spent amount and remaining capacity reflect prior usage before new
agents are started.
- Around line 297-304: Update begin_agent and the concurrent child execution
paths to reserve bounded token capacity against token_budget before starting
each child, preventing concurrent starts from exceeding the remaining budget.
Reconcile the reservation with actual settled usage when each child completes,
including the related checks at the other execution sites. If pre-start
reservation cannot be implemented, change the behavior and wording to explicitly
describe a post-settlement cutoff rather than a hard budget.
- Around line 1787-1797: Update the warning calculation in the budget_line
closure to compare the 80% threshold using widened arithmetic instead of
saturating u64 multiplication. Preserve the existing conditions, including
budget.remaining > 0, while ensuring large valid budgets are evaluated by their
actual spent-to-total ratio.
In `@crates/orca-runtime/tests/runtime_host.rs`:
- Around line 2050-2053: Update the assertions in the runtime host recovery test
around thread.snapshot() to require exactly one matching user message rather
than merely any match, and additionally verify that neither generation consumed
the queued steer input. Preserve the existing snapshot validation while
explicitly asserting the queued steer-input state after both generations.
In `@docs/agent-workflow-benchmark.md`:
- Around line 299-309: Renumber the later “Resume/fork stress coverage”
follow-up entry from item 13 to item 14, preserving its existing content and
leaving the agent teams item as 13.
---
Nitpick comments:
In `@site/src/shared.ts`:
- Around line 7-14: Ensure the v0.3.8 GitHub release and tag exist before
publishing the metadata in releaseVersion and releases. If the release cannot be
created first, gate the site metadata, sitemap, and roadmap deployment on that
release step so the latest entry is neither future-dated nor linked to an
unavailable URL.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 8bf8e8d0-deb6-4606-8beb-a9a762eaf12d
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (25)
Cargo.tomlcrates/orca-core/src/workflow_types.rscrates/orca-runtime/src/controller.rscrates/orca-runtime/src/history.rscrates/orca-runtime/src/runtime_host.rscrates/orca-runtime/src/runtime_special.rscrates/orca-runtime/src/system_prompt.rscrates/orca-runtime/src/thread_store/writer.rscrates/orca-runtime/src/workflow/command.rscrates/orca-runtime/src/workflow/runner.rscrates/orca-runtime/src/workflow_execution.rscrates/orca-runtime/tests/runtime_host.rscrates/orca-tools/src/registry.rscrates/orca-tui/src/hosted_runtime.rscrates/orca-tui/src/ui.rsdocs/agent-workflow-benchmark.mddocs/production-roadmap.mddocs/releases/v0.3.8.mddocs/tools-comparison.mdnpm/orca/package.jsonscripts/validate-runtime-surface-contract.mjssite/public/sitemap.xmlsite/src/changelog/Changelog.tsxsite/src/shared.tstests/workflow_types_contract.rs
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/pages.yml:
- Around line 38-45: Update the release lookup in the Pages workflow so only a
confirmed missing or unpublished release writes ready=false; use structured gh
release view output such as publishedAt to determine availability. Capture
lookup failures, distinguish the missing-release status from authentication,
rate-limit, network, and API errors, and fail the release gate for all other
errors.
In @.github/workflows/release.yml:
- Around line 526-536: Update the pages workflow dispatch in the release job to
pass github.ref_name as a workflow_dispatch input, and update the pages workflow
gate to require that input matches the version read from site/src/shared.ts
before building or deploying. Use the dispatched tag rather than the main branch
ref to ensure Pages deploys the verified release.
In `@crates/orca-runtime/src/workflow/runner.rs`:
- Around line 1262-1273: Update the child execution path around
run_child_agent_call so the permit’s budget_cap is enforced by the child runtime
during execution, not only validated after the call returns. Pass the
reservation through the child execution-limit mechanism while preserving the
existing settle_usage handling, and add coverage for a child reporting usage
above its reservation.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 15be9c78-afad-46f6-b0eb-d4d4f9e79a8a
📒 Files selected for processing (8)
.github/workflows/pages.yml.github/workflows/release.ymlcrates/orca-runtime/src/system_prompt.rscrates/orca-runtime/src/workflow/runner.rscrates/orca-runtime/tests/runtime_host.rscrates/orca-tools/src/registry.rsdocs/agent-workflow-benchmark.mddocs/releases/v0.3.8.md
🚧 Files skipped from review as they are similar to previous changes (5)
- docs/releases/v0.3.8.md
- crates/orca-runtime/src/system_prompt.rs
- crates/orca-tools/src/registry.rs
- docs/agent-workflow-benchmark.md
- crates/orca-runtime/tests/runtime_host.rs
| version="$(sed -nE 's/^export const releaseVersion = "([^"]+)";$/\1/p' site/src/shared.ts)" | ||
| test -n "$version" | ||
| if gh release view "$version" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then | ||
| echo "ready=true" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "GitHub Release $version is not published yet; defer Pages deployment." | ||
| echo "ready=false" >> "$GITHUB_OUTPUT" | ||
| fi |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'Files matching pages workflow:\n'
fd -a 'pages\.yml$' . | sed 's#^\./##'
printf '\nRelevant workflow excerpt:\n'
if [ -f .github/workflows/pages.yml ]; then
nl -ba .github/workflows/pages.yml | sed -n '1,90p'
fi
printf '\nSearch for release gate usages:\n'
rg -n "ready=|pages-build-deployment|releaseVersion|gh release view|site/src/shared.ts" .github site 2>/dev/null || true
printf '\nShared release version declaration:\n'
if [ -f site/src/shared.ts ]; then
nl -ba site/src/shared.ts | sed -n '1,120p'
fiRepository: echoVic/orca-agent
Length of output: 255
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'Files matching pages workflow:\n'
git ls-files '.github/workflows/pages.yml' | sed 's#^\./##' || true
printf '\nRelevant workflow excerpt:\n'
if [ -f .github/workflows/pages.yml ]; then
awk '{printf "%6d\t%s\n", NR, $0}' .github/workflows/pages.yml | sed -n '1,100p'
fi
printf '\nSearch for release gate usages:\n'
if git ls-files | grep -E '(^|/)(\.github/workflows/pages\.yml|site/src/shared\.ts)$' >/tmp/fileslist; then
grep -n -H "ready=\|pages-build-deployment\|releaseVersion\|gh release view\|site/src/shared.ts" "$(cat /tmp/fileslist)" 2>/dev/null || true
fi
printf '\nShared release version declaration:\n'
if [ -f site/src/shared.ts ]; then
awk '{printf "%6d\t%s\n", NR, $0}' site/src/shared.ts | sed -n '1,140p'
fiRepository: echoVic/orca-agent
Length of output: 7938
🌐 Web query:
GitHub CLI gh release view --json exit code 404 documentation
💡 Result:
The GitHub CLI (gh) does not return a specific exit code 404 when a release is not found [1]. Instead, when the gh release view command fails to locate the specified release, it returns exit code 1 [1][2]. In the GitHub CLI, exit code 1 is the standard convention for general errors, including cases where a requested resource (like a release or repository) cannot be found [1]. While some manual pages for the GitHub CLI document specific exit codes like 4 for "Authentication required" [3][4][5], they do not list 404 as a specific exit code for gh release view. You should expect an exit code of 1 for "not found" scenarios [1][2].
Citations:
- 1: Document exit codes for
gh releasecli/cli#6024 - 2: christaylorcodes/ConnectWiseAutomateAgent@bf28e3c
- 3: https://man.freebsd.org/cgi/man.cgi?apropos=0&manpath=FreeBSD+15.1-RELEASE+and+Ports.quarterly&query=gh-release-view&sektion=1
- 4: https://man.archlinux.org/man/gh-release-view.1.en
- 5: https://man.archlinux.org/man/gh-release-view.1.en.txt
🌐 Web query:
GitHub CLI gh release view structured JSON output fields exit-code 4
💡 Result:
In the GitHub CLI (gh), the command gh release view supports structured JSON output and adheres to established exit code conventions. JSON Output To obtain structured JSON output, use the --json flag followed by the specific fields you wish to retrieve [1][2]. The available fields include: apiUrl, assets, author, body, createdAt, databaseId, id, isDraft, isImmutable, isPrerelease, name, publishedAt, tagName, tarballUrl, targetCommitish, uploadUrl, url, and zipballUrl [1][3]. You can further refine or format this output using the following flags: - --jq: Filters the JSON output using a jq expression [1][4]. - --template: Formats the JSON output using a Go template [1][4]. Exit Code 4 The exit code 4 in the context of the GitHub CLI signifies that authentication is required to execute the command [3][5]. If you encounter this exit code, ensure that you are authenticated with GitHub (e.g., by running gh auth login) and that your credentials have the necessary permissions for the repository [2][3]. Standard exit codes for the GitHub CLI generally follow these conventions: 0: Successful execution [3][5] 1: Error (e.g., resource not found, network issues) [3][6] 2: Command canceled [3][5] 4: Authentication required [3][5] Note that while these codes are widely used across the CLI, specific commands may occasionally return additional, command-specific exit codes [3][5].
Citations:
- 1: https://cli.github.com/manual/gh_release_view
- 2: https://latchkey.dev/learn/command-reference/gh-release-view
- 3: https://man.archlinux.org/man/gh-release-view.1.en
- 4: https://cli.github.com/manual/gh_help_reference
- 5: https://www.mankier.com/1/gh-release-view
- 6: Document exit codes for
gh releasecli/cli#6024
🌐 Web query:
gh release view GitHub CLI 404 error message not found
💡 Result:
Receiving a "404 Not Found" or "release not found" error when using gh release view typically indicates that the GitHub CLI cannot locate the specified release based on your current request, permissions, or the state of the repository [1][2]. Common causes and solutions include: 1. No "Latest" Release Exists: When gh release view is executed without a specific tag argument, it defaults to searching for the "latest" release [3][4]. GitHub defines a "latest" release specifically as a full (stable) release [3]. If your repository only contains pre-releases or drafts, the command will fail with a 404 error because no stable "latest" release exists [3]. Solution: Explicitly provide the tag name (e.g., gh release view <tag_name>) to view a pre-release or specific version [3]. 2. Replication Lag in CI/CD: In automated workflows, such as GitHub Actions, there can be a slight delay (a few seconds) between the time a release is created (e.g., via gh release create) and when it becomes discoverable by subsequent gh commands [5][6]. Solution: Add a brief pause (e.g., sleep 5) after creating the release before attempting to view or modify it [5][6]. Alternatively, use a single command to perform operations (like uploading assets) during the creation step itself [6]. 3. Insufficient Permissions or Authentication: If the repository is private, the GH_TOKEN or credentials provided to the CLI may not have the necessary permissions to access the release [1][7]. In GitHub Actions, the default GITHUB_TOKEN is scoped only to the repository where the workflow is running; it cannot access other repositories unless specifically configured with a wider-scoped token, such as a Personal Access Token (PAT) [7]. Solution: Ensure your authentication token has sufficient read permissions for the target repository [1][7]. 4. Incorrect Tag Name: The gh release commands expect a Git tag name, not necessarily the display name of the GitHub release [2]. Solution: Use gh release list to verify the exact tag names available in the repository, and use the correct tag when running gh release view <tag> [2]. 5. Draft Releases: gh release view may struggle to find draft releases if they have not been properly initialized or if the authentication lacks the required scopes to see drafts [1]. If you are creating a draft, ensure the process completes successfully before accessing it [5].
Citations:
- 1: https://latchkey.dev/learn/command-reference/gh-release-view
- 2: https://stackoverflow.com/questions/73780525/release-not-found-by-github-cli-when-attempting-to-download-a-named-release-by
- 3: Does "gh release view/download" fail if there are only prerelease releases available? cli/cli#5131
- 4: https://cli.github.com/manual/gh_release_view
- 5:
gh release createsuccessfully returns before release is available to subsequent calls cli/cli#6599 - 6: Uploading an artificat to a draft release doesn't work (not found) cli/cli#6198
- 7: https://stackoverflow.com/questions/71068476/accessing-another-repository-with-github-cli-in-github-actions
Fail the release gate when release lookup fails for reasons other than a missing release.
Lines 40-45 write ready=false for every gh release view failure. Authentication failures, rate limits, network failures, and GitHub API errors can also return non-zero status, so the build and Pages deploy are skipped even though the advertised release should be ready. Treat only a missing release as ready=false; fail the gate for other lookup errors. Use structured release output such as --json publishedAt or --jq .publishedAt != null when querying availability.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/pages.yml around lines 38 - 45, Update the release lookup
in the Pages workflow so only a confirmed missing or unpublished release writes
ready=false; use structured gh release view output such as publishedAt to
determine availability. Capture lookup failures, distinguish the missing-release
status from authentication, rate-limit, network, and API errors, and fail the
release gate for all other errors.
| pages: | ||
| if: ${{ github.ref_type == 'tag' && needs.verify.result == 'success' && needs.verify-windows.result == 'success' }} | ||
| needs: [verify, verify-windows] | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| actions: write | ||
| steps: | ||
| - name: Dispatch Pages after public release verification | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| run: gh workflow run pages.yml --repo "${{ github.repository }}" --ref main |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Files matching release/pages/shared:"
git ls-files | rg '(^|/)(release\.yml|pages\.ya?ml|shared\.ts)$|site/src/shared\.ts|pages' || true
echo
echo "Release workflow relevant dispatch:"
sed -n '510,545p' .github/workflows/release.yml 2>/dev/null || true
echo
echo "Pages workflows:"
fd -t f '(^pages.*\.ya?ml$\|release\.ya?ml$)' .github/workflows -x sh -c 'echo "--- $1"; sed -n "1,220p" "$1"' sh {}
echo
echo "site/src/shared.ts if present:"
if [ -f site/src/shared.ts ]; then
sed -n '1,220p' site/src/shared.ts
fi
echo
echo "Search releaseVersion/advertisement/version inputs:"
rg -n "releaseVersion|release_version|ref_name|workflow_dispatch|workflow_dispatch:" .github site || trueRepository: echoVic/orca-agent
Length of output: 9562
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Pages workflow:"
cat -n .github/workflows/pages.yml
echo
echo "Relevant release workflow sections:"
sed -n '230,275p' .github/workflows/release.yml | cat -n
sed -n '330,375p' .github/workflows/release.yml | cat -n
sed -n '530,540p' .github/workflows/release.yml | cat -n
echo
echo "Workflow run contract tests:"
cat -n tests/pages_workflow_contract.test.mjs
echo
echo "Static invariant checks:"
python3 - <<'PY'
from pathlib import Path
release = Path(".github/workflows/release.yml").read_text()
pages = Path(".github/workflows/pages.yml").read_text()
shared = Path("site/src/shared.ts").read_text()
dispatch = "gh workflow run pages.yml --repo \"${{ github.repository }}\" --ref main"
page_input_defs = pages.splitlines()[:80]
has_release_input = any("release_version" in line or "release-version" in line or "required" in line for line in page_input_defs)
extracts_shared = "sed -nE 's/^export const releaseVersion = \"([^\"]+)\";$/\\1/p' site/src/shared.ts" in pages
print("release workflow contains dispatch without inputs:", dispatch in release)
print("pages workflow exposes release_version input:", has_release_input)
print("pages workflow compares release input with site/src/shared.ts:", "compare" in pages.lower() and "release" in pages.lower() and "releaseVersion" in pages)
print("pages workflow extracts site/src/shared.ts releaseVersion:", extracts_shared)
PYRepository: echoVic/orca-agent
Length of output: 8803
Pass the verified release tag to the Pages workflow gate.
The release workflow dispatches pages.yml --ref main, but pages.yml then reads the advertised version from the checked-out main tree with sed ... site/src/shared.ts. This means a different tag or a main change can determine the deployment instead of the release that just passed verification. Pass ${{ github.ref_name }} into pages.yml as a workflow_dispatch input and require that input to match site/src/shared.ts before building or deploying.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/release.yml around lines 526 - 536, Update the pages
workflow dispatch in the release job to pass github.ref_name as a
workflow_dispatch input, and update the pages workflow gate to require that
input matches the version read from site/src/shared.ts before building or
deploying. Use the dispatched tag rather than the main branch ref to ensure
Pages deploys the verified release.
| let mut child_execution_policy = execution_policy.clone(); | ||
| if let Some(budget_cap) = permit.budget_cap() { | ||
| child_execution_policy.max_agent_tokens = Some(budget_cap); | ||
| } | ||
| match self.run_child_agent_call( | ||
| &call, | ||
| workflow_ipc, | ||
| &child_execution_policy, | ||
| workflow_cancel, | ||
| ) { | ||
| Ok(child_output) => { | ||
| permit.settle_usage(child_output.usage.total_tokens())?; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Enforce the reservation in the child runtime.
Lines 1262-1265 set max_agent_tokens to the reservation. run_child_agent_call only checks this value after the child returns at Line 1603. A child can consume more than its reservation before Lines 1273 and 1360 settle the usage. The run can then exceed its configured token budget.
Pass the reservation to the child execution limit, not only to post-execution validation. Add a test where a child reports usage above its reservation.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/orca-runtime/src/workflow/runner.rs` around lines 1262 - 1273, Update
the child execution path around run_child_agent_call so the permit’s budget_cap
is enforced by the child runtime during execution, not only validated after the
call returns. Pass the reservation through the child execution-limit mechanism
while preserving the existing settle_usage handling, and add coverage for a
child reporting usage above its reservation.
1 participant
Summary
Verification
Summary by CodeRabbit