Skip to content

Codex host guidance and honest connection failures; halve the configuration docs - #197

Merged
tobert merged 6 commits into
mainfrom
codex-integration
Sep 15, 2026
Merged

tobert merged 6 commits into
mainfrom
codex-integration

Conversation

@tobert

@tobert tobert commented Sep 13, 2026

Copy link
Copy Markdown
Owner

A Codex MCP call can reach a provider while the same kaibo CLI call fails inside Codex's command sandbox. kaibo used to report that connection failure as a provider rejection. This PR classifies it as a connection failure, points the caller at the checks to run, adds a short host-agents section to the configuration reference, and halves the configuration docs.

What changed

  • Connection failure class (src/server/render.rs). rig flattens reqwest's chain to HttpError … error sending request; that now reads as "no response from the provider, not a rejection", names the endpoint/DNS/proxy/TLS/network checks, and points at Host agents in kaibo://config/guide. Timeouts and resets stay TransientProvider (the transient markers are checked first, and the test now asserts the classification, not just the wording).
  • Every failure message tells the calling agent to ask the user what to do — retry, use another cast, or continue without the consultation. Main's messages told the agent to "proceed without the consultation", and the first draft here swapped that for "within the user's authorized scope". Amy: "I don't think we should tell an agent to proceed without consultation, it should ask the user what to do." A new test takes one body per failure kind, asserts its classification first, and requires "Ask the user whether" with no "proceed without"; it failed before the messages changed. The kaibo-side message no longer says "please report it". The transient message states TRANSIENT_RETRIES and that a timeout or reset is not retried, and the README failure paragraph no longer claims kaibo never retries.
  • kaibo config-guide prints the embedded reference without loading config or telemetry, so it works when the config is broken. The test runs the binary with a cleared environment and a malformed config and compares stdout byte for byte.
  • Host agents, not a separate Codex guide. Amy: "I feel like codex.md should merge into the config guide, which should have a short section for per-agent quirks." So docs/codex.md, the kaibo://config/codex resource, and the config-guide codex topic are gone; ## Host agents / ### Codex at the end of docs/config.md keeps what changes setup: the launch snippet with an approval default, separate MCP and shell network access, forwarding api_key_env variables, the 60 s tool timeout and the async tools, store write access for shell-run kaibo. Copies of Codex's own auto-review, elicitation, persistent-approval, and permissions-profile documentation were cut: they drift with Codex, the approval UI was unverified, and kaibo sends no elicitation requests.
  • The resident instructions lead says "codebase review and second opinions from another model family" and discloses that hosted casts send source to providers. It no longer names a Codex resource, which every host would pay for each session.
  • Configuration reference and template: 23,669 → 11,223 words, including the new Host agents section. A Sonnet audit compared every config key, KAIBO_* variable, and flag against the structs and found none lost. Restored from the cut: a blocked disable_builtins call exits 126 and kaibo serve refuses an unknown name, a write past scratch_limit_bytes is refused, a scratch-directory allow_paths example, and the configure prompt's OpenRouter public-catalog query. Two audit items were stale rather than lost and stay out: the "no default root" case (the inferred cwd is additive now, resolver.rs) and a bare $TMPDIR example (an unset variable fails loading).
  • Doc errors fixed while there: --config beats $KAIBO_CONFIG (the old block had it backwards, inherited from main); reasoning.effort on the Responses wire is sent only for gpt-5* ids; telemetry has no CLI flags.
  • Merged main (A slow model call shows in job_get and warns the caller #196): slow_chat_secs is one line in the short template.

Not in this PR

  • classify_failure has no marker for a bare 500/502 body, so a call completion_retry just retried four times reads "retrying is unlikely to help". Pre-existing; its own PR with a failing-first test.
  • MCP tool annotations, a planned follow-up PR (Amy: "we can do a followup PR with tool annotations, that probably will help modern claude code too"). kaibo sets none today, so Codex's approval_mode = "writes" prompts for every kaibo tool; readOnlyHint would let it auto-approve paid calls, so that PR weighs it against openWorldHint.
  • A Claude Code subsection under Host agents.
  • [sandbox] disable_builtins names are validated only in kaibo serve (main.rs:201); a CLI run with a misspelled name silently leaves the builtin enabled, although sandbox.rs says it was validated loudly. The template now says kaibo serve; the CLI gap is its own fix.

Validation

  • Offline suite: 1,377 passed, 0 failed, 23 ignored (live provider tests); cargo clippy --all-targets -D warnings clean.
  • The new ## Host agents heading guard fails when the heading is renamed (negative control run, then restored byte for byte).
  • every_failure_asks_the_user_what_to_do failed on the old messages (exit 101) before they changed.
  • Codex CLI 0.154.0 probes (first commit): MCP connectivity versus the sandboxed CLI, and a command-domain policy that reached an allowed provider host and refused a denied one.

Review

  • Codex (GPT-6 Astra) wrote the first three commits; DeepSeek Flash reviewed them through kaibo and caught the connection classifier absorbing timeouts.
  • Claude Opus 5 reviewed the PR for what helps Codex use kaibo, then made the fold and the fixes above. DeepSeek Flash (kaibo deepseek cast) verified the doc corrections and the Codex guide's kaibo-side claims against the code, and reviewed the final revision.

🤖 Generated with Claude Code

tobert and others added 4 commits September 13, 2026 11:39
…ejection

A Codex MCP call can reach a provider while the same kaibo CLI call fails
inside the command sandbox. Report that connection failure accurately and
point to focused host-access and paid-call approval guidance.

Expose the guide as kaibo://config/codex and kaibo config-guide codex;
config-guide bypasses config and telemetry loading so recovery guidance
remains available with broken setup. State hosted data flow in the MCP
handshake and preserve Codex-owned consent decisions.

Validated with failing-first transport/resource tests, CLI broken-config
coverage, the offline suite, and a no-model Codex handshake/network probe.

Co-Authored-By: Codex (GPT-6 Astra) <noreply@openai.com>
Keep the template focused on settings and the reference focused on semantics.
Reduce their combined word count from 23,669 to 10,718. Preserve active TOML
values except the example preamble, and correct stale key-resolution, retry,
path-inference, telemetry, and OpenAI-compatible effort guidance.

Use existing user preferences in configure and ask only for missing choices;
configured credentials do not themselves authorize paid calls.

Validation: offline suite (1,367 passed; 23 live tests ignored), clippy with
warnings denied, guide CLI test after final prose edits, TLS dependency guard.
DeepSeek Flash reviewed the template against key, path, scope, and failure
sections and found no concrete contradictions.

Co-Authored-By: Codex (GPT-6 Astra) <noreply@openai.com>
Co-Authored-By: DeepSeek Flash <noreply@deepseek.com>
DeepSeek review caught the new generic connection classifier absorbing known
timeouts. Strengthen the existing transient regression test to assert the
classification, prove it fails, then preserve the more specific diagnosis.
Pin the diagnostic guide URI and repair the misplaced provenance doc comment.

Describe persistent approval conditionally: kaibo neither stores approval nor
promises a host control whose UI behavior has not been verified.

Validation: failing-first timeout regression; all 1,367 offline tests pass;
clippy --all-targets with warnings denied; TLS dependency guard.

Co-Authored-By: Codex (GPT-6 Astra) <noreply@openai.com>
Co-Authored-By: DeepSeek Flash <noreply@deepseek.com>
# Conflicts:
#	CHANGELOG.md
#	docs/config.example.toml
@tobert tobert changed the title Guide Codex access and paid-call approval; shorten configuration docs Codex host guidance and honest connection failures; halve the configuration docs Sep 15, 2026
tobert and others added 2 commits September 15, 2026 11:30
Amy judged the Codex guide by what helps Codex use kaibo, and asked
that it merge into the config guide as a short section for per-agent
quirks. docs/codex.md, the kaibo://config/codex resource, and the
`config-guide codex` topic are gone; `## Host agents` / `### Codex`
keeps the launch snippet, the separate MCP and shell network access,
env key forwarding, the 60 s tool timeout and the async tools, and
store write access. Copies of Codex's own approval, auto-review, and
elicitation docs were cut: they drift with Codex, and kaibo sends no
elicitation requests.

Connection and transient failures end in "or proceed without the
consultation" again, since kaibo is augmentation; "within the user's
authorized scope" is gone from both. The transient message states
TRANSIENT_RETRIES and that timeouts and resets are not retried. The
resident instructions lead no longer names a Codex resource.

Restored from the doc cut: disable_builtins exit 126 and serve's
unknown-name refusal, a scratch-dir allow_paths example, and the
configure prompt's OpenRouter catalog query. Fixed: --config beats
$KAIBO_CONFIG, Responses effort is gpt-5* only, telemetry has no CLI
flags, and configure names the state directory rather than the db file.
A test pins the Host agents heading that three published strings name;
renaming it was shown to fail the test.

DeepSeek Flash (kaibo, deepseek cast) verified the doc corrections
against the code and reviewed this revision; its findings on the serve-
only builtin check, job_wait's 60 s default, the state directory grant,
and the missing heading guard are addressed here.

Reviewed-by: DeepSeek Flash <noreply@deepseek.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Amy, reading the PR: "I don't think we should tell an agent to proceed
without consultation, it should ask the user what to do." An agent
that decides alone to go on without kaibo hides a failure the user may
want to fix, so every classified failure now ends with "Ask the user
whether to" retry, use another cast, or continue without the
consultation. That covers main's rejection and kaibo-side messages
too, and the kaibo-side message no longer asks anyone to "report it".

A new test takes one error body per FailureKind, asserts the
classification before the wording so a drifted body cannot pass on
another arm's text, and requires the ask-the-user sentence with no
"proceed without". It failed on the old messages before they changed.

The README failure paragraph said kaibo never retries; it now says
overloads, rate limits, and 500/502/503 are retried up to four times,
a fumbled tool call twice, and timeouts and resets not at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tobert
tobert marked this pull request as ready for review September 15, 2026 16:17
@tobert
tobert merged commit 3c03b0e into main Sep 15, 2026
1 check passed
@tobert
tobert deleted the codex-integration branch September 15, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant