Skip to content

feat: adapt subagent timeouts to observed duration - #7195

Open
rubencu wants to merge 2 commits into
kirodotdev:mainfrom
rubencu:feat/adaptive-subagent-timeouts
Open

feat: adapt subagent timeouts to observed duration#7195
rubencu wants to merge 2 commits into
kirodotdev:mainfrom
rubencu:feat/adaptive-subagent-timeouts

Conversation

@rubencu

@rubencu rubencu commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

Subagents have a fixed 30-minute default execution timeout. Real workloads that legitimately need longer are terminated unless an operator notices the pattern and manually increases agent.subagent_timeout_secs.

Why it matters

Long-running delegated work can lose progress at a predictable deadline even when it is healthy. Operators should not need to repeatedly tune a global timeout just because observed subagent workloads have grown beyond the initial default.

What changed (motivation → approach → change)

  • Treat agent.subagent_timeout_secs as the operator-controlled floor and enable adaptive growth by default with agent.subagent_timeout_auto.
  • Raise future deadlines after a timeout or a successful run that consumes at least 80% of its captured deadline. Growth is at least 30 minutes and approximately 1.5x, rounded to five minutes: 30m → 60m → 90m → 120m.
  • Add agent.subagent_timeout_max_secs, defaulting to two hours and validated from 30 minutes through 24 hours. A manually configured floor above that ceiling remains authoritative.
  • Capture each run's timeout when it starts, so learning never changes an in-flight deadline. Stale observations from concurrent runs cannot compound the same increase, and the reaper uses the captured per-run value.
  • Persist only the learned timeout, reason, and timestamp in an owner-only atomic state file. Reads reject links, hardlinks, non-regular files, out-of-root paths, and oversized state.
  • Defer state restoration until after gateway readiness and run all state reads and writes on the maintenance executor, keeping filesystem I/O off the constructor, boot path, and event loop.
  • Expose the validated settings through the existing configuration surfaces and update user, configuration, subagent, and resource-protection documentation.

Tests

  • Added adaptive-policy coverage for growth, ceiling/floor behavior, opt-out, stale/concurrent observations, corrupt state, and monotonic delayed restoration.
  • Added persistence security coverage for symlink/hardlink rejection, redirected-parent refusal, bounded reads, and atomic round trips.
  • Added manager/run/reaper integration coverage for deferred executor I/O, scheduled persistence, near-limit success, timeout learning, immutable run deadlines, and reaping races.
  • Validation completed:
    • 56 focused adaptive-timeout, configurable-timeout, reaper, and conservative-shutdown tests passed.
    • 734 subagent and gateway regression tests passed.
    • 70 configuration schema, bounds, and baseline tests passed.
    • Black, isort, flake8, mypy, docs lint, public scrub, brand, harness-parity, lock, changelog, focus-cue, and vendor-manifest gates passed.

Manual verification

N/A — policy, persistence, lifecycle, race, and configuration behavior are covered by focused and regression tests; no external service or rendered UI path changes.

Screenshots / video

Why no screenshot: backend timeout policy and configuration only; no rendered UI changes.

Related Issues

no linked issue: reported directly by a user

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@rubencu
rubencu requested a review from a team as a code owner August 31, 2026 03:28
@rubencu
rubencu requested a review from pepmach August 31, 2026 03:28
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Aug 31, 2026
@rubencu

rubencu commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author
  • Reaper diagnostic may display the mutable manager default — rebutted as disproportionate to this PR.

The advisory holds only for the diagnostic number; enforcement, persistence, and captured deadlines remain correct. The owning upstream module currently contains an unbaselined backend import, so any edit fails the shrink-only SDK-boundary gate. Correcting the string therefore requires a separate process-supervisor boundary migration rather than widening this timeout-policy PR.

@rubencu

rubencu commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author
  • Concurrent user stop could be learned as a timeout — fixed in 6fb9c2b3b.

The reaper now observes timeout usage only after execution started and only when user_stopped remains false after force-reap returns. Parameterized one-sweep coverage locks normal timeout growth, concurrent-stop neutrality, and pre-execution neutrality; the focused verifier returned PASS.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Aug 31, 2026
@bolichen97

Copy link
Copy Markdown
Collaborator

Open PR relationship audit

This is a consolidated, point-in-time code-level audit note. It compares complete merge-base diffs and current/merged code; it does not treat a shared topic as duplication or partial coverage as completion.

Relationship findings

  • PR #5277 is OVERLAPPING relative to this PR. The goals differ or the implementations can complement each other; this is not a duplicate claim. Recommended action for PR #5277: MERGE_DISCUSSION. Independent goals in the same constructor; sequence them rather than choose between them. Files: src/kiro_crew/subagent.py. The two independent directions used different labels; the matrix conservatively retains OVERLAPPING for coordination.
  • PR #6713 is OVERLAPPING relative to this PR. The goals differ or the implementations can complement each other; this is not a duplicate claim. Recommended action for PR #6713: KEEP. Different user goals with no behavioral collision; only ordinary rebase friction in run.py and subagent.md. Files: src/kiro_crew/subagent_manager/run.py, docs/system-specs/modules/subagent.md.
  • This PR is OVERLAPPING with PR #7304. The goals differ or the implementations can complement each other; this is not a duplicate claim. Recommended action for PR #7195: REBASE. 7304 merged at 2026-09-02T03:57:34Z and the merge conflict label was applied to 7195 twenty seconds later at 2026-09-02T03:57:54Z. This is a pure relocation conflict in one file, not a behavior conflict. Files: src/kiro_crew/config/loader.py.
  • This PR is OVERLAPPING with PR #8003. The goals differ or the implementations can complement each other; this is not a duplicate claim. Recommended action for PR #7195: REBASE. Different user goals — an enforced adaptive deadline versus credit-usage reporting — that complement each other. Only textual conflict in shared hunks; whichever merges second rebases. Files: src/kiro_crew/subagent.py, src/kiro_crew/slack/gateway.py, docs/system-specs/modules/subagent.md.

No PR, Issue, label, branch, or review state was changed by the relationship-note portion of this audit.

@dwu96 dwu96 added the needs-pr-triage PR scanner: awaiting automated triage label Sep 7, 2026
@NicholasRBowers NicholasRBowers added drive-to-green PR claimed by drive-to-green pipeline and removed needs-pr-triage PR scanner: awaiting automated triage labels Sep 7, 2026
@NicholasRBowers

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]: This PR has been inactive for 7+ days with failing CI. I've assessed the blockers and they appear resolvable — I'll push fixes directly to this branch as a co-author.

Assessment: The maintainer audit already mapped the rebase file-by-file (pure relocation conflict in config/loader.py from #7304, textual hunks in subagent.py/slack/gateway.py/docs from #8003, judged non-behavioral). The stale-base CI reds are in files this diff never touches and clear on rebase; the fresh head then lets the skipped review lanes stamp. Prior GPT/Opus findings are already dispositioned.

If you'd prefer I don't touch this PR, add the pr-no-autofix label.

@bolichen97

Copy link
Copy Markdown
Collaborator

Hi @rubencu, thanks for this one. Two things before it can land.

Still worth doing, but the numbers need reworking. The motivation, a fixed 30-minute deadline killing healthy long runs, is already handled on main by a different mechanism: merged #8891 raised the static default in src/kiro_crew/constants.py to SUBAGENT_TIMEOUT_SECS = 10800 with a 60..86400 load-time clamp. The adaptive part is genuinely absent from main: there is no src/kiro_crew/subagent_timeout.py, no subagent_timeout_auto or subagent_timeout_max_secs, and both enforcement sites still read the single mutable manager value (src/kiro_crew/subagent_manager/run.py, src/kiro_crew/subagent_manager/monitoring.py). So the learned per-run deadline is the remaining scope, and it is not a rebase-only change. As written it would regress: your 1800s floor and 7200s learning ceiling both sit below main's 10800s default. Please rebase on current main, derive the floor from constants.SUBAGENT_TIMEOUT_SECS and put the ceiling above it, and move the config hunks from src/kiro_crew/config/loader.py to src/kiro_crew/config/sections.py plus constants.py, where AgentConfig now lives. One more thing to declare: main documents agent.subagent_timeout_secs as deliberately not dashboard-settable in docs/architecture/resource-protection.md, and your _EDITABLE_CONFIG change in src/kiro_crew/dashboard/handlers/core.py makes both new keys editable.

Overlap with #8003 (credit usage reporting, cc @welikoiwanenko). Shared files: src/kiro_crew/subagent.py (both add fields to SubagentInfo), src/kiro_crew/subagent_manager/run.py, src/kiro_crew/subagent_manager/monitoring.py, src/kiro_crew/slack/gateway.py, and the same run-lifecycle bullet in docs/system-specs/modules/subagent.md. The goals do not collide, the conflict is textual. #8003 is already based on current main, so let it land first and rebase this PR on top.

Posted from the 2026-09-08 open-PR relationship audit (read-only, one auditor per PR); reply here if any of this is wrong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

drive-to-green PR claimed by drive-to-green pipeline fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants