Skip to content

docs(verify): map maintenance 2026-11-20 - #7031

Merged
enyst merged 8 commits into
mainfrom
cloud/nifty-ride-1jbmi6
Nov 21, 2026
Merged

enyst merged 8 commits into
mainfrom
cloud/nifty-ride-1jbmi6

Conversation

@enyst

@enyst enyst commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

HUMAN:

Engel Nyst (@enyst) set up a daily automation that keeps the verify-agent-server and verify-sdk feature maps current. This PR is its run for main@722e5d4, after PR #7030.


AGENT:

I'm an AI agent helping Engel Nyst (@enyst) with project work.

Why

This is the daily delta pass over both feature maps for the commits merged since each map's baseline:

  • Agent-server map: main@0275e714..main@722e5d4, 48 commits. Its baseline did not move yesterday.
  • SDK map: main@14142392..main@722e5d4, 19 commits.

Every family the owners commands listed was replayed live on a fresh run. Bullets whose fixes merged in the range were updated, and every edited family was re-run by an independent agent.

As yesterday, this machine had no DeepSeek key, only a LiteLLM proxy serving deepseek-v4.1-flash. Recipes that read $DEEPSEEK_API_KEY or assert the native deepseek/deepseek-flash model fail here for that reason alone. F30–F34 were also driven through a local stand-in for the DeepSeek API that forwards to the proxy.

Summary

Agent-server map: delta pass, partial (baseline stays at 0275e71). BASE..TARGET 0275e71..722e5d4.

SDK map: delta pass (baseline moves to 722e5d4). BASE..TARGET 1414239..722e5d4.

Both maps

Issue Number

#5751

How to Test

uv sync --dev
export PATH="$PWD/.agents/skills/verify-agent-server/scripts:$PWD/.agents/skills/verify-sdk/scripts:$PATH"
control-agent-server map check && control-agent-server map coverage && sdk-map check
control-agent-server map run --file F19 --only F19.conversation-mcp-add-restart --fresh --record
control-agent-server map run --file Fnn --fresh --record      # each of the 30 families listed above
sdk-map run --file S09 --only S09.runtime-add-resume
sdk-map run --file S15 --only S15.tabs                        # needs Chromium
sdk-map run --file S13 --only S13.secret-registry             # known failure #7002
sdk-map run --file Snn                                        # each of the 17 families listed above

uv run pytest tests/cross/test_verify_agent_server_skill.py tests/cross/test_verify_sdk_skill.py → 96 passed.

Evidence (README):

  • [Agent-server report](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/agent-server-report-main.md): every replayed bullet, with re-runs. Also the [stand-in runs](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/agent-server-report-shim.md) and the [F19 edit](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/agent-server-report-edited.md).
  • [SDK report](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/sdk-report-main.md): the 17 families. Also the [independent re-run of the edited families](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/sdk-report-edited.md).
  • F19.conversation-mcp-add-restart: the [tool call after the restart](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/edited/F19/F19.conversation-mcp-add-restart), which returns QA_MCP_ECHO:two.
  • S09.runtime-add-resume: [transcript](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/edited/sdk/B2-S09-resume), pass.
  • S15.tabs: [transcript](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/edited/sdk/B3-S15-tabs-after), pass.
  • S21.ask-agent: [transcript](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/edited/sdk/B3-S21-ask), pass.
  • S13 rotated secret, Security: SecretRegistry leaks rotated credentials in cleartext after update_secrets() or LookupSecret re-export (and drops masked history across sub-agents) #7002: [SDK transcript](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/edited/sdk/final-S13/S13-security-confirmation-04.log) and [agent-server observations](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/repro/7002-server/obs.json).
  • F07 Gotcha, Session socket drops the UI websocket with 1013 'frame_too_large' on events > 4 MiB (MAX_FRAME_BYTES hardcoded) #6934: [live-socket capture](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/repro/6934/live-big.json), which closes with 1013 frame_too_large, and the [reconnect past it](https://github.com/enyst/openhands-evidence/blob/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/repro/6934/after3.json).

Video/Screenshots

The agent-server change is #7016, the factories experimental flag. The Canvas feature is F30 (Settings → Experimental), on Canvas 428b3dc against an Agent Server built from this checkout.

[F30.open: the Factories switch, off by default](https://raw.githubusercontent.com/enyst/openhands-evidence/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/ui/factories.png)

[F30.toggle: Factories switched on and still on after a reload; there is no sidebar entry yet (OpenHands/OpenHands#19800)](https://raw.githubusercontent.com/enyst/openhands-evidence/3723e9c6bf25dd053c3b3ace90adc57885a1cb2f/software-agent-sdk/map-maintenance/2026-11-20/ui/factories-on.png)

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

The branch carries the pass's work-in-progress commits. Squash on merge.


Generated by Claude Code


🐳 Agent Server images for this PR — GHCR package, pull/run commands, and all pushed tags (click to expand)

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python-slim amd64, arm64 python-node-runtime Link
python-minimal amd64, arm64 python-node-runtime Link
python amd64, arm64 python-node-runtime Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:944ae10-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-944ae10-python \
  ghcr.io/openhands/agent-server:944ae10-python

All tags pushed for this build

ghcr.io/openhands/agent-server:944ae10-golang-amd64
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-golang-amd64
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-golang-amd64
ghcr.io/openhands/agent-server:944ae10-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:944ae10-golang-arm64
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-golang-arm64
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-golang-arm64
ghcr.io/openhands/agent-server:944ae10-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:944ae10-java-amd64
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-java-amd64
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-java-amd64
ghcr.io/openhands/agent-server:944ae10-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:944ae10-java-arm64
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-java-arm64
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-java-arm64
ghcr.io/openhands/agent-server:944ae10-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:944ae10-python-amd64
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-python-amd64
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-python-amd64
ghcr.io/openhands/agent-server:944ae10-python-node-runtime-amd64
ghcr.io/openhands/agent-server:944ae10-python-arm64
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-python-arm64
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-python-arm64
ghcr.io/openhands/agent-server:944ae10-python-node-runtime-arm64
ghcr.io/openhands/agent-server:944ae10-python-minimal-amd64
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-python-minimal-amd64
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-python-minimal-amd64
ghcr.io/openhands/agent-server:944ae10-python-node-runtime-minimal-amd64
ghcr.io/openhands/agent-server:944ae10-python-minimal-arm64
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-python-minimal-arm64
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-python-minimal-arm64
ghcr.io/openhands/agent-server:944ae10-python-node-runtime-minimal-arm64
ghcr.io/openhands/agent-server:944ae10-python-slim-amd64
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-python-slim-amd64
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-python-slim-amd64
ghcr.io/openhands/agent-server:944ae10-python-node-runtime-slim-amd64
ghcr.io/openhands/agent-server:944ae10-python-slim-arm64
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-python-slim-arm64
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-python-slim-arm64
ghcr.io/openhands/agent-server:944ae10-python-node-runtime-slim-arm64
ghcr.io/openhands/agent-server:944ae10-golang
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-golang
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-golang
ghcr.io/openhands/agent-server:944ae10-golang_tag_1.21-bookworm
ghcr.io/openhands/agent-server:944ae10-java
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-java
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-java
ghcr.io/openhands/agent-server:944ae10-eclipse-temurin_tag_17-jdk
ghcr.io/openhands/agent-server:944ae10-python-minimal
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-python-minimal
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-python-minimal
ghcr.io/openhands/agent-server:944ae10-python-node-runtime-minimal
ghcr.io/openhands/agent-server:944ae10-python-slim
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-python-slim
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-python-slim
ghcr.io/openhands/agent-server:944ae10-python-node-runtime-slim
ghcr.io/openhands/agent-server:944ae10-python
ghcr.io/openhands/agent-server:944ae10b38fab5199dfa12b21fd5f7c29d7b8397-python
ghcr.io/openhands/agent-server:cloud-nifty-ride-1jbmi6-python
ghcr.io/openhands/agent-server:944ae10-python-node-runtime

About Multi-Architecture Support

  • Each variant tag (e.g., 944ae10-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., 944ae10-python-amd64) are also available if needed

enyst and others added 8 commits November 20, 2026 08:27
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwM8dBzVQQUM8UxPqdS
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwM8dBzVQQUM8UxPqdS
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwM8dBzVQQUM8UxPqdS
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwM8dBzVQQUM8UxPqdS
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwM8dBzVQQUM8UxPqdS
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwM8dBzVQQUM8UxPqdS
SDK map baseline moves to main@722e5d40; agent-server map keeps its baseline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwM8dBzVQQUM8UxPqdS
@github-actions

Copy link
Copy Markdown
Contributor

REST API breakage checks (OpenAPI) — ✅ PASSED

Result: ✅ PASSED

Action log

@enyst
enyst requested a review from all-hands-bot November 20, 2026 10:18

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review was posted by an AI agent (OpenHands).Scope gate: passes. The PR touches only .agents/skills/verify-agent-server/ and .agents/skills/verify-sdk/ — this repository's own verification skills and feature maps, with dedicated cross tests (tests/cross/test_verify_*.py). It is the established daily map-maintenance pass (previous run merged as #7006), so it belongs here and needs no product/architecture decision.

Technical review — no material findings.

Verified against the current head (944ae10, all required CI checks green):

  • References check out. The proposed SDK baseline main@722e5d403 exists on main (Release v1.63.1, #7030). Every fix the edited bullets credit is present in main history: #6929 (7d6ef6f, runtime MCP tool names), #6751 (1a73a00, close executors on rejected response_schema), #6786 (3d638f2, close_tab waits for tab-list drop), #6666 (7ccc5e4, fork usage billing), #6678 (8fb5ba0, orphan tool-message classification), #6811 (c763226, LiteLLM 1.108.3). The LiteLLM version notes updated from 1.105.1 to 1.108.3 match that bump.
  • Issue states match the map edits. #6927, #6785 and #6748 are closed (their known-bug markers are correctly dropped or rewritten as current behavior); #7002 (rotated-secret masking), #6934 (4 MiB frame cap on live sockets) and #6695 are open, so keeping them as known failures/Gotchas is accurate.
  • README counts are arithmetically consistent. Family rows still sum to 504 sub-features with untested 36 → 35, and the S09 row changes 23/1 → 23/0, matching the claimed "S09 untested 1 → 0". Only that row changed, as described.
  • The new verification program is sound. programs/S13/secret_rotation.py compiles, uses the documented sdk_verify helpers and the public SDK API (SecretSource.get_value, Conversation.update_secrets), follows the same pattern as the existing S13 program, and its documented failure mode (single-value _exported_values cache, resolve_uncached_values skipping cached names, masking limited to exported values) is consistent with the actual openhands/sdk/conversation/secret_registry.py on main. Its synthetic token values are clearly test fixtures, not secrets.
  • Focused tests pass. uv run pytest tests/cross/test_verify_agent_server_skill.py tests/cross/test_verify_sdk_skill.py → 96 passed in the workspace on this head, matching the PR's claim and the green cross-tests CI job.

The agent-server side adds two Gotchas (#6934 live-stream frame cap, #7002 secret-rotation masking) and rewrites the F19 runtime-MCP bullet to describe post-#6929 behavior including runtime_mcp_servers, which exists in openhands/sdk/conversation/state.py as described. All documentation-level, with live evidence linked in the description.

No blocking or non-blocking findings; nothing needs a maintainer decision.

✅ APPROVED

@enyst
enyst merged commit 3a893a1 into main Nov 21, 2026
53 checks passed
@enyst
enyst deleted the cloud/nifty-ride-1jbmi6 branch November 21, 2026 12:34
@enyst enyst mentioned this pull request Nov 21, 2026
1 of 5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants