diff --git a/cli.md b/cli.md index a74b070..bacf5d9 100644 --- a/cli.md +++ b/cli.md @@ -692,11 +692,14 @@ In the `ag-ui` format the same contract holds one level up: one `TEXT_MESSAGE_ST **Exit codes:** -| Code | Meaning | -| ---- | --------------------------- | -| 0 | Task completed successfully | -| 1 | Task failed | -| 2 | Max turns exhausted | +| Code | Meaning | +| ---- | ---------------------------------------------------- | +| 0 | Task completed successfully | +| 1 | Task failed | +| 2 | Max turns exhausted (`agent.max_turns`) | +| 3 | Time budget exhausted (`agent.max_duration_minutes`) | + +Codes `2` and `3` are both "stopped early, not broken" - a harness can tell out of turns from out of time and salvage the work in progress instead of treating the run as a failure. **Headless behavior:** @@ -2846,6 +2849,7 @@ logging: - System prompt (`prompts.agent.system_prompt`) and per-mode adjustments (`mode_adjustment_plan` / `mode_adjustment_auto`, delivered by the [mode-change reminder](#how-mode-instructions-are-delivered)) - System reminders interval - Max turns and tokens +- `agent.max_duration_minutes` (`INFER_AGENT_MAX_DURATION_MINUTES`): wall-clock budget for one run in minutes, default `0` (no budget). Once spent, the run ends **after the current tool call** rather than mid-call, so the work so far stays on disk and `infer headless` exits `3`. It is the clock-based companion to `agent.max_turns` - a slow run (a flash-tier model, browser use) can burn an hour inside its turn budget, and only a wall-clock limit bounds that. Pair it with the [`elapsed_minutes`](#trigger-catalog) reminder trigger to have the agent start wrapping up before the budget runs out - Parallel tool execution (default: 5 concurrent) - Project instructions: `agent.agents_md.*` injects the root `AGENTS.md` and points at nested ones (see [How AGENTS.md reaches the model](#how-agents-md-reaches-the-model)) @@ -2936,7 +2940,7 @@ reminders: | `hook` | string | When the reminder fires. One of `pre_tool` (before each tool call) or `post_tool` (after each tool call completes). | | `trigger` | string | Condition under which the reminder fires. See [Trigger catalog](#trigger-catalog) below. | | `text` | string | The reminder text injected into the conversation. Supports `os.ExpandEnv` environment variable interpolation (`$VAR` or `${VAR}`). | -| `threshold` | integer | `on_repeated_failure` only (required, > 0). Number of consecutive failures of the same call before the reminder fires. | +| `threshold` | integer | Required (> 0) for `on_repeated_failure` - consecutive failures of the same call before the reminder fires - and for `elapsed_minutes`, where it is the number of minutes. | | `guidance` | map | `on_mode_change` only. Maps a mode key (`standard`, `plan`, `auto`) to the text substituted for the `{guidance}` placeholder in `text`. Omitted keys keep their built-in defaults. | #### Trigger catalog @@ -2945,6 +2949,7 @@ reminders: | --------------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `always` | Any | Fire on every hook invocation. | | `on_failure` | `post_tool` | Fire only when the tool call that just ran failed (returned an error). Requires `hook: post_tool`; validation rejects other hooks. | +| `elapsed_minutes` | Any | Fire once, when the run has been going `threshold` wall-clock minutes. Requires `threshold` > 0. The clock-based twin of `turns_before_max` - set it below [`agent.max_duration_minutes`](#configuration) so the agent is told to wrap up while it still has budget left. | | `on_mode_change` | Any | Fire when the [agent mode](#agent-modes) changes (Shift+Tab). Used by the built-in `mode-change-reminder`, which is the **sole carrier** of mode-specific instructions - see [How mode instructions are delivered](#how-mode-instructions-are-delivered). | | `on_repeated_failure` | `post_tool` | Fire when the same tool call has failed `threshold` times in a row (built-in default `3`). Matching is on **normalized** arguments, not the raw string: `offset`, `limit`, key order and whitespace are ignored, so retries that only change pagination or re-serialize the same arguments still count toward the threshold. `Read` matches on the `file_path` alone. A success with the same key resets the counter. Requires `hook: post_tool` and `threshold` > 0. | @@ -3149,14 +3154,15 @@ infer config set gateway.timeout 300 **Hot keys** - these change the running session immediately: -| Key | Effect on reload | -| ------------------------ | --------------------------------------------------------------- | -| `gateway.timeout` | The next request uses the new timeout | -| `agent.model` | Switches the active model, like [`/model`](#built-in-commands) | -| `agent.max_turns` | Applies from the next turn | -| `agent.max_tokens` | Applies from the next turn | -| `agent.reasoning_effort` | Applies from the next turn, and the status bar follows | -| `chat.status_bar.*` | The whole section - indicators hide or show on the next repaint | +| Key | Effect on reload | +| ---------------------------- | --------------------------------------------------------------- | +| `gateway.timeout` | The next request uses the new timeout | +| `agent.model` | Switches the active model, like [`/model`](#built-in-commands) | +| `agent.max_turns` | Applies from the next turn | +| `agent.max_duration_minutes` | Applies from the next turn | +| `agent.max_tokens` | Applies from the next turn | +| `agent.reasoning_effort` | Applies from the next turn, and the status bar follows | +| `chat.status_bar.*` | The whole section - indicators hide or show on the next repaint | **Everything else needs a restart.** Any other changed key is named in the status line as "restart to apply" rather than silently ignored - `Reloaded agent.model ยท restart to apply gateway.url`. `gateway.url` and `gateway.api_key` are restart keys, and [tool policy](#tool-configuration), the [file sandbox](#file-sandbox) and approval policy never change mid-session, by design: a running agent keeps the permissions it started with. diff --git a/github-action.md b/github-action.md index 822dc0d..7709ba4 100644 --- a/github-action.md +++ b/github-action.md @@ -69,6 +69,7 @@ The action composes a default set of [CLI system reminders](/cli/#system-reminde - **Periodic context nudge** - a `pre_stream` reminder on an interval that keeps the agent focused on the task. - **Turns-before-max wrap-up** - a `turns_before_max` reminder that fires near the turn limit, prompting the agent to wrap up. +- **Time-budget wrap-up** - when [`time-budget-minutes`](#time-budget) is set, an `elapsed_minutes` reminder that fires at 80% of the budget, prompting the agent to wrap up before the clock runs out. Absent when the input is empty. - **Post-tool failure nudge** - a `post_tool` / `on_failure` reminder that fires only after a failed tool call on writable runs, reminding the agent to retry or ask the user. - **Memory nudges** - when `memory-repo` is set, the CLI's built-in `memory-consult` and `memory-hygiene` reminders are re-emitted. - **Demo recording pointer** - when [`record-demo`](#demo-recordings) is `true`, a `pre_stream` / `once` reminder named `infer-action-record-demo` fires on the first turn, telling the agent to read and follow the CLI's built-in `demo` skill for the recording procedure. @@ -87,7 +88,35 @@ To override the composed default and supply your own full set of reminders, use text: "A failed call means the change did not happen" ``` -See the [CLI reminders documentation](/cli/#system-reminders) for the full YAML schema, trigger catalog (`always`, `interval`, `turns_before_max`, `once`, `on_failure`), and the `INFER_REMINDERS_CONFIG` / `--reminders-file` / `on_failure` trigger reference. +See the [CLI reminders documentation](/cli/#system-reminders) for the full YAML schema, trigger catalog (`always`, `interval`, `turns_before_max`, `elapsed_minutes`, `once`, `on_failure`), and the `INFER_REMINDERS_CONFIG` / `--reminders-file` / `on_failure` trigger reference. + +### Time budget + +`max-turns` is counted in turns, not minutes. A slow run - a flash-tier model, heavy browser use - can spend the job's `timeout-minutes` long before its wrap-up turn arrives, and GitHub then kills the job mid-tool-call: the salvage step never runs, and the spinner comment is never replaced. + +`time-budget-minutes` adds the clock. It takes whole minutes and sets the CLI's [`agent.max_duration_minutes`](/cli/#configuration): + +```yaml +jobs: + infer: + runs-on: ubuntu-latest + timeout-minutes: 60 + steps: + - uses: inference-gateway/infer-action@main + with: + max-turns: 150 + time-budget-minutes: 45 +``` + +- **At 80%** the composed wrap-up reminder fires on the CLI's `elapsed_minutes` trigger, telling the agent to finish and commit what it has. +- **At 100%** the CLI ends the run after the current tool call and exits `3`. Because the job itself is still alive, the salvage and report steps run as usual - the work in progress becomes a draft PR and the result comment is posted. +- The run is reported like the turn limit (`exit-code` `2`): stopped early, the exit code passed through, `timed-out` set to `true`, and a warning rather than a failed job. + +Guidance: + +- Keep the budget **under the job's `timeout-minutes`**, with room for the salvage and report steps - otherwise GitHub kills the job first and the budget never gets a chance to fire. +- Keep it **under 60** when `github-token` comes from `create-github-app-token`. That token lives one hour, and the salvage and report steps need it to push the branch and post the comment. +- Empty (the default) leaves today's behaviour unchanged. Requires CLI >= v0.232.0. ### System prompt override @@ -158,6 +187,7 @@ The table below covers the most commonly used inputs. The complete, always-curre | `apt` | No | `''` | Newline- or space-separated list of apt packages to install before the agent runs. Installed via `DEBIAN_FRONTEND=noninteractive sudo apt-get install -y`. Debian/Ubuntu runners only - on other OS families the step prints an error and fails. Packages are cached across runs (keyed on OS, architecture, and the package list); a cache failure falls back to a plain install. A plain `run:` step before the action is the portable alternative. | | `languages` | No | `''` | Newline- or space-separated list of languages whose toolchains to install before the agent runs. Supported: `go`, `rust`, `node` (alias `typescript`), `python`. Empty = no language setup steps. See [Language toolchains](#language-toolchains). | | `max-turns` | No | `150` | Maximum agent iterations - acts as a runaway-cost guard. | +| `time-budget-minutes` | No | `''` | Wall-clock budget for the agent run, in whole minutes. Sets the CLI's `agent.max_duration_minutes` (`INFER_AGENT_MAX_DURATION_MINUTES`). At 80% of the budget a wrap-up reminder fires on the CLI's `elapsed_minutes` trigger; at 100% the CLI ends the run after the current tool call and exits `3`, so the salvage and report steps still run inside the job. Empty = off. Requires CLI >= v0.232.0. See [Time budget](#time-budget). | | `custom-instructions` | No | `''` | Extra instructions appended to the default system prompt (does **not** replace the defaults). | | `system-prompt-issue` | No | `''` | Overrides the action's bundled system prompt for issue-driven runs. Substitutes `{{issueNumber}}`. See [System prompt override](#system-prompt-override). | | `system-prompt-pr` | No | `''` | Overrides the action's bundled system prompt for PR-driven runs (non-fork). Substitutes `{{prNumber}}`, `{{headRef}}`. See [System prompt override](#system-prompt-override). | @@ -264,16 +294,16 @@ Requires a reasonably recent `inference-gateway/infer-action` release; referenci ## Outputs -| Output | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `result` | Human-readable summary of the agent execution. | -| `exit-code` | Exit code returned by `infer` - non-zero means the agent failed. | -| `pr-url` | URL of the pull request the agent opened (empty if none). Populated for direct-prompt runs and any run that opens a PR. | -| `run-duration-ms` | Wall-clock duration of the agent run in milliseconds (0 if unavailable). | -| `stopped-early` | `true` when the agent stopped before finishing its plan (unfinished todos, uncommitted work left behind, or a job-timeout stop). | -| `timed-out` | `true` when the agent was stopped because the job hit its timeout; the remaining work is recovered into a draft PR and reported as a warning, not a failure. | -| `failed-tool-calls-count` | Number of failed tool calls detected in the agent output. | -| `total-tool-calls-count` | Total number of tool calls the agent made during the run. | +| Output | Description | +| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `result` | Human-readable summary of the agent execution. | +| `exit-code` | Exit code returned by `infer`. `0` is success, `2` means the turn limit was reached, `3` means the [time budget](#time-budget) was spent, any other non-zero means the agent failed. | +| `pr-url` | URL of the pull request the agent opened (empty if none). Populated for direct-prompt runs and any run that opens a PR. | +| `run-duration-ms` | Wall-clock duration of the agent run in milliseconds (0 if unavailable). | +| `stopped-early` | `true` when the agent stopped before finishing its plan (unfinished todos, uncommitted work left behind, or a job-timeout stop). | +| `timed-out` | `true` when the agent ran out of time - the job hit its own timeout, or the [time budget](#time-budget) was spent (`exit-code` `3`). The remaining work is recovered into a draft PR and reported as a warning, not a failure. | +| `failed-tool-calls-count` | Number of failed tool calls detected in the agent output. | +| `total-tool-calls-count` | Total number of tool calls the agent made during the run. | Reference outputs in downstream steps via ${{ steps.<id>.outputs.result }}. @@ -814,7 +844,7 @@ Three principles to follow: Additional hardening: -- Cap `max-turns` to prevent runaway loops and bound cost. The default is `150`; `30-50` covers most issues. +- Cap `max-turns` to prevent runaway loops and bound cost. The default is `150`; `30-50` covers most issues. Add [`time-budget-minutes`](#time-budget) to bound the run in wall-clock time too - turns alone do not stop a slow run from eating the whole job timeout. - Set `enable-git-operations: false` whenever the agent only needs to read and comment. - Keep the bash allow-list narrow - only add commands you trust the agent to invoke. - Reference `inference-gateway/infer-action@main` so every run uses the latest action; if you need reproducible runs, pin to a tagged release from the [releases page](https://github.com/inference-gateway/infer-action/releases).