Bound stalled provider model streams

[ross0x01]

Closed without merging

This PR was closed after verifying the current OpenRouter and AI SDK behavior more closely.

All HackerAI model traffic already goes through OpenRouter, which enforces provider read deadlines and enables provider fallback by default. HackerAI also supplies explicit fallback model lists to OpenRouter. OpenRouter can retry before any tokens are delivered and intentionally does not replay after partial output; mid-stream provider timeouts are returned as in-stream errors. This already covers the primary provider-stall and safe-fallback behavior targeted here.

The additional five-minute application timer would duplicate that responsibility and introduce a correctness risk. OpenRouter sends SSE comments such as : OPENROUTER PROCESSING while a long-running model request is still alive, but the installed AI SDK parser consumes those comments without surfacing them through onChunk. The proposed guard could therefore classify a live request as inactive and abort it after five minutes.

OpenRouter does not publish its exact read-deadline duration, so an outer HackerAI deadline could still be useful if production evidence shows the OpenRouter connection itself can remain stuck. That should be reconsidered with observed failure data and a design that accounts for transport keepalives, rather than shipping a second timer based only on parsed model chunks.

Summary

• add a shared five-minute provider inactivity guard to Agent streamText execution
• cover both a missing first chunk and a stall between provider chunks
• pause the guard before tool execution so long terminal and sandbox work are not timed as model inactivity
• classify internal timeouts separately from explicit user cancellation and reuse the existing provider timeout observability/refund path
• allow the existing bounded fallback only when the current output-safety rules say the failed leg is safe to discard

Root cause

The existing run-level caps are evaluated around completed model steps, so a provider stream that stops making progress can prevent the Agent loop from reaching those checks. The pinned AI SDK exposes totalMs, stepMs, and chunkMs, but its current chunkMs implementation is first armed only after the SDK receives a stream event and can stay armed while tool execution is in progress. Using it directly would miss a never-starting response and could abort a legitimate long tool step.

The new guard follows the shared Agent provider lifecycle instead: it arms immediately before each provider request, resets on provider chunks, pauses before tool execution, and clears on step completion. Both the Next.js and Trigger Agent paths inherit the same behavior from agent-stream-runner.

Behavior and risk

• Timeout aborts carry ProviderStreamTimeoutError with PROVIDER_STREAM_TIMEOUT, first_chunk or between_chunks, and the bounded duration.
• Provider timeout errors continue through the existing timeout category, usage tracking, refund checks, and terminal Trigger failure handling.
• Explicit user aborts dispose the guard without creating a provider timeout.
• Fallback is still limited to the existing safe-output policy; visible partial text, completed tool calls, and tool output are not replayed.
• Existing run duration, budget, spend-cap, served-model accounting, and settlement behavior remain unchanged.

Validation

• corepack pnpm test --runInBand — 297 suites / 2,931 tests passed
• focused provider, Agent runner, retry, billing/spend-cap, PTY, active-runtime, error-classification, and Trigger contract suites — 11 suites / 185 tests passed
• five-minute policy follow-up — 2 suites / 20 tests passed
• corepack pnpm typecheck
• targeted ESLint for all changed files
• corepack pnpm lint — 0 errors; 5 existing warnings in unrelated UI files
• Prettier check for all changed files

Manual Agent-mode verification

1. On the preview deployment, select Agent mode with Auto and use a non-production provider fault injection that withholds the first response chunk for more than five minutes. Confirm the primary leg ends as a provider timeout and fallback starts only if no useful output was emitted.
2. Repeat with one visible text chunk followed by more than five minutes of provider silence. Confirm the partial output is preserved and no fallback replay starts.
3. Run a long terminal or interactive tool operation spanning more than five minutes. Confirm it is not interrupted by the provider inactivity guard and the next provider step resumes normally.
4. Click Stop during a provider response. Confirm the run remains a user cancellation and is not classified or logged as a provider timeout.

Summary by CodeRabbit

• New Features
  • Added provider stream inactivity detection with separate “first chunk” vs “between chunks” phases and detailed timeout errors.
  • Provider inactivity timeout now defaults to 5 minutes.
• Bug Fixes
  • Streaming/fallback now treats provider timeouts as a distinct terminal error, updating stop/finish behavior and suppressing success reporting.
  • Retry logic uses a specific provider_timeout reason, and abort handling preserves timeout stop state.
• Tests
  • Added/extended Jest coverage for timeout guard behavior, runner lifecycle handling, multi-compaction aborts, and retry decisions.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hackerai	Ready	Preview, Comment	Jul 24, 2026 3:54pm

[coderabbitai]

📝 Walkthrough

Walkthrough

Changes

The PR adds provider-stream inactivity detection with first-chunk and between-chunk timeout phases. Agent streaming tracks provider activity, pauses during tool execution, propagates timeout state, records errors, and preserves terminal outcomes through fallback retry and analytics.

Provider timeout handling

Layer / File(s)	Summary
Timeout guard contract and lifecycle lib/api/provider-stream-timeout.ts, lib/api/__tests__/provider-stream-timeout.test.ts	Defines timeout errors, phases, guard lifecycle methods, timer behavior, external-abort handling, tool-execution pauses, disposal, and tests.
Stream runner timeout integration lib/api/agent-stream-runner.ts, lib/api/__tests__/agent-stream-runner-multi-compaction.test.ts, lib/api/__tests__/agent-long-contracts.test.ts	Wires timeout tracking into stream callbacks, abort propagation, error recording, finish reasons, cleanup, and coverage.
Fallback retry and terminal outcome handling lib/api/chat-handler.ts, trigger/agent-long.ts, lib/chat/__tests__/agent-long-provider-retry.test.ts	Adds provider-timeout retry state and reasons, resets timeout state for fallback runs, prioritizes terminal errors, and suppresses success analytics for terminal provider errors.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Provider
  participant AgentStreamRunner
  participant TimeoutGuard
  participant ChatHandler
  participant AgentLongRetry
  Provider->>AgentStreamRunner: Start provider step and emit chunks
  AgentStreamRunner->>TimeoutGuard: Start, record activity, pause, finish
  TimeoutGuard->>AgentStreamRunner: Emit timeout error and abort signal
  AgentStreamRunner->>ChatHandler: Record provider timeout state
  ChatHandler->>AgentLongRetry: Schedule fallback with provider_timeout reason
  AgentLongRetry->>ChatHandler: Reconcile terminal outcome

Loading

Possibly related PRs

• hackerai-tech/hackerai#635: Modifies the same agent-long fallback retry logic for terminal provider stream failures.
• hackerai-tech/hackerai#904: Modifies the same agent stream runner lifecycle hooks used for provider step and tool-call handling.
• hackerai-tech/hackerai#950: Modifies the same runner lifecycle and agent-long contract coverage around stream completion callbacks.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding a guard around stalled provider model streams.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/provider-stream-stall-guard

---

_{Comment @coderabbitai help to get the list of available commands.}

[ross0x01]

Closing without merging after verifying the existing request path.

All HackerAI model traffic already uses OpenRouter, which applies provider read deadlines and fallback routing. It can transparently retry before output and does not replay after partial output, so the proposed guard largely duplicates the existing stall and fallback boundary.

The five-minute timer also watches parsed AI SDK chunks, while OpenRouter keepalive comments are consumed by the SSE parser without reaching onChunk. That creates a risk of aborting a legitimate long-running request that OpenRouter still considers active.

If production evidence later shows the OpenRouter connection itself can remain stuck, we can revisit an outer deadline designed around raw transport activity and an explicit HackerAI SLA.