Skip to content

Upstream sync: v1.0.0-beta.10 round 6 (schema 1.0.56-1, cloud-no-id, multitenancy)#113

Draft
krukow wants to merge 6 commits into
mainfrom
upstream-sync/v1.0.0-beta.10-round-6
Draft

Upstream sync: v1.0.0-beta.10 round 6 (schema 1.0.56-1, cloud-no-id, multitenancy)#113
krukow wants to merge 6 commits into
mainfrom
upstream-sync/v1.0.0-beta.10-round-6

Conversation

@krukow
Copy link
Copy Markdown
Collaborator

@krukow krukow commented May 30, 2026

Summary

Syncs the Clojure SDK with upstream github/copilot-sdk for tags v1.0.0-beta.9 and v1.0.0-beta.10. Schema pin advances 1.0.55-11.0.56-1. Closes round 6 of the rolling upstream-sync project.

This is a non-breaking addition-only sync. All new config keys and message options are optional, and the cloud-no-id path activates only when :cloud is set and :session-id is omitted (an explicit caller-supplied id keeps the prior behaviour exactly).

Session plan: ~/.copilot/session-state/b794e17b-2c5a-42ed-90fe-63b823312966/plan.md (local; key decisions captured in this PR description).

Ported upstream PRs

Upstream PR What Where
#1326 mcpOAuthTokenStorage config client.clj, specs.clj, API.md
#1438 agentMode on MessageOptions session.clj, specs.clj, API.md
#1470 displayPrompt on MessageOptions session.clj, specs.clj, API.md
#1474 7 multitenancy flags + embeddingCacheStorage client.clj, specs.clj, API.md
#1479 Server-assigned sessionId for cloud sessions protocol.clj, client.clj, helpers, API.md
#1482 pluginDirectories config client.clj, specs.clj, API.md

Parity gaps closed (existed in SessionConfigBase prior to this window):

  • reasoningSummary (#{:none :concise :detailed})
  • contextTier (#{:default :long-context}, wire "default" / "long_context")
  • largeOutput now forwarded on session.resume (was already accepted on create)

Three new session events (auto-picked-up by the regenerated wire spec; added to the public event-types set; curated data specs for the two with hand-shaped payloads):

  • :copilot/hook.progress
  • :copilot/session.autopilot_objective_changed
  • :copilot/session.permissions_changed

Deferred (will be addressed in follow-up rounds)

  • PR #1428 — Multitenancy Client Mode: substantial new public surface (mode = "empty" | "copilot-cli", ToolSet, toolFilterPrecedence, ambient flags via session.options.update). Per User input in the planning phase, this gets a dedicated future plan and sync round of its own.
  • configDirconfigDirectory and outputDiroutputDirectory rename (PR #1482 tail): wire keys stay the same; deferring the Clojure-side option-key rename to a coordinated breaking-rename release alongside other rename PRs.
  • Canvas runtime, MCP Apps enableMcpApps: continue to defer as experimental coupled surfaces.

Cloud-no-id design notes (PR #1479)

The upstream Node.js SDK omits sessionId from session.create when the caller doesn't supply one and :cloud is set, then registers the session under the server-returned id. This is necessary because the server is the source of truth for cloud session ids.

The Clojure port faces an ordering challenge: any session-scoped notifications that arrive immediately after the response must find the session registered. The protocol's reader thread processes responses synchronously, then continues reading. The implementation adds an inline-response callback option to protocol/send-request ({:on-response-inline (fn [result])}) that runs in the reader thread before the result is delivered downstream. The client's callback:

  1. Validates the returned sessionId is a non-blank string (errors otherwise).
  2. Pre-registers the session under the server-assigned id.
  3. Registers transform callbacks.
  4. Installs the session-fs handler if applicable.

If anything throws, the callback unwinds any partial registration via a tracked registered-id atom before delivering an error result. The session-fs factory is validated upfront (ensure-session-fs-handler-factory!) before the RPC, so a missing factory cannot reach the reader-thread callback. The callback body is wrapped in try/catch in the protocol so reader-thread health is never compromised by a misbehaving client callback.

Three helpers (ensure-session-fs-handler-factory!, install-session-fs-handler!, make-create-session-inline-callback) are shared between the sync and async create/resume paths so the same cleanup guarantees apply everywhere.

Wire-format gotchas (verified)

  • :mcp-oauth-token-storage would camelCase to mcpOauthTokenStorage (lowercase o), which the CLI does not accept. Build-params bypasses the default converter with the literal string wire key "mcpOAuthTokenStorage" (the conversion layer preserves non-keyword keys).
  • :in-memory value → "in-memory" via (name kw), not csk (which would mangle to "inMemory").
  • :context-tier :long-context"long_context" (underscore) via an explicit case mapping.
  • :config-directory and :output-directory Clojure-side options round-trip to the legacy wire keys configDir / outputDir (deferred Clojure-side rename, see Deferred section).

Validation

  • bb test (unit + integration): 300 tests / 1439 assertions / 0 failures / 0 errors
  • bb ci (no E2E): passes
  • bb ci:full (with E2E): one pre-existing flake in test-e2e-blob-attachment (30 s LLM timeout on the vision model) — reproduces on clean main, not a regression
  • ./run-all-examples.sh: all examples pass
  • bb validate-docs: clean

Code review

Two parallel multi-model reviews (Claude Opus 4.7-high and GPT-5.5) were run on the full change set with focused review areas (cloud-no-id correctness, wire conversion accuracy, spec completeness, API parity, test coverage, concurrency, DRY).

Finding Severity Status
::join-session-config missing ::large-output in :opt-un despite :large-output being in its closed-keys set (Opus) Low Fixed before push
::context-tier (keyword spec) lifted into ::session.resume-data curated spec, but inbound events carry wire string "long_context"; no coerce entry — fails the curated spec (GPT-5.5) Medium Fixed before push — removed ::context-tier from curated ::session.resume-data to match the round-5 pattern (::session.model_change-data deliberately did not lift context-tier into the curated layer). The wire-level generated spec still covers it.
Earlier rubber-duck pass: sessionFs factory on reader thread; partial-registration cleanup High → addressed in design Fixed via upfront validation + registered-id cleanup atom (in the same commit)

Both reviewers' final recommendations after fixes: cloud-no-id flow is structurally sound, all wire conversions verified correct, spec coverage matches API parity, tests assert wire-format at the right level.

Commits

  1. chore(schema): bump copilot CLI schema 1.0.55-1 → 1.0.56-1
  2. feat(events): expose 3 new round-6 event types on the public API
  3. feat(protocol): add inline-response callback on send-request
  4. feat(session): forward :agent-mode and :display-prompt on send!
  5. feat(client): port round 6 session config + cloud-no-id (PR #1479)
  6. docs: round 6 changelog and API reference

Each commit individually builds and passes tests.


Generated via Copilot on behalf of @krukow

krukow and others added 6 commits May 30, 2026 13:05
Regenerated wire layer (event_specs.clj, coerce.clj) for upstream tags
v1.0.0-beta.9 and v1.0.0-beta.10. Schema diff surfaces:

- contextTier ("long_context" | "default" | nil) on session.start,
  session.resume, session.model_change event data.
- workingDirectory on external_tool.requested event data.
- Three new session event types: hook.progress,
  session.autopilot_objective_changed, session.permissions_changed.
- Autopilot objective status enum widened to include "active",
  "paused", "cap_reached", "completed".
- Many new SessionConfigBase fields (mcpOAuthTokenStorage,
  embeddingCacheStorage, pluginDirectories, multitenancy flags,
  reasoningSummary, contextTier, displayPrompt, agentMode, etc.) —
  exposed on the public API in a follow-up commit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- :copilot/hook.progress (ephemeral progress from long-running hooks)
- :copilot/session.autopilot_objective_changed
- :copilot/session.permissions_changed

Added to the public event-types and session-events sets, plus fixture
entries in codegen_test for the two with hand-written curated data
specs (added in the follow-up specs commit). Curated specs land in
specs.clj so they're not re-generated when the schema is regenerated.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds an optional {:on-response-inline (fn [result])} option to
send-request and send-request!. The callback is invoked synchronously
in the JSON-RPC reader thread, before the response is delivered to the
result channel and before the next inbound message is dispatched.

This is the building block for upstream PR #1479 (server-assigned
sessionId for cloud sessions): the SDK uses this callback to register
the session under the server-returned id atomically with respect to
session-scoped notifications that may arrive immediately after the
response on the wire.

The callback runs in a try/catch so any exception is logged and the
result is still delivered to the caller. Callers must keep the callback
fast and non-blocking — it executes on the single reader thread.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- :agent-mode — keyword in #{:interactive :plan :autopilot :shell},
  wire-encoded as agentMode. Per-message agent mode (upstream PR #1438).
- :display-prompt — string shown in the timeline UI instead of the
  model-facing :prompt. Wire-encoded as displayPrompt
  (upstream PR #1470).

Applied to both send! and <send-async* so blocking and core.async
call paths behave identically.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Session config additions, all optional, accepted on both create and
resume unless noted (closes pre-existing parity gaps from earlier
upstream releases as well):

- :mcp-oauth-token-storage — #{:persistent :in-memory}.
  Wire-encoded as the string key "mcpOAuthTokenStorage" (bypassing the
  default kebab→camel converter, which would lower-case OAuth).
  (upstream PR #1326)
- :embedding-cache-storage, :skip-embedding-retrieval,
  :organization-custom-instructions,
  :enable-on-demand-instruction-discovery, :enable-file-hooks,
  :enable-host-git-operations, :enable-session-store, :enable-skills
  — per-session multitenancy granular flags. (upstream PR #1474)
- :plugin-directories — extra plugin dirs loaded even when
  :enable-config-discovery is false. (upstream PR #1482)
- :reasoning-summary — #{:none :concise :detailed} (parity gap).
- :context-tier — #{:default :long-context}, wire-encoded as
  contextTier with values "default"/"long_context" via an explicit
  case table (parity gap).
- :large-output on resume — was already accepted on create; now also
  forwarded on resume to match upstream client.ts:1308 (parity gap).

PR #1479 — server-assigned sessionId for cloud sessions:

When :cloud is set and :session-id is omitted from create-session /
<create-session, the SDK now omits sessionId from session.create and
captures the server-assigned id from the response. Registration runs
inside an inline-response callback on the protocol reader thread so
that any session-scoped notification arriving immediately after the
response is correctly routed to the freshly-registered session.

Extracted three helpers used by both sync and async create/resume:
- ensure-session-fs-handler-factory! validates the factory is present
  BEFORE the RPC (fail-fast, prevents deadlock from the reader-thread
  callback throwing).
- install-session-fs-handler! is the post-RPC factory invocation.
- make-create-session-inline-callback builds the cloud-no-id reader
  callback with full partial-registration cleanup.

Curated event-data specs for the two round-6 event types with
hand-shaped payloads:
- ::hook.progress-data — :message string.
- ::session.permissions_changed-data — :allow-all-permissions and
  :disable-permissions booleans.

23 new integration tests cover all of the above, including wire-format
assertions for the camelCase / kebab / underscore boundaries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CHANGELOG: round 6 [Unreleased] entries for MessageOptions additions
(agentMode, displayPrompt), mcpOAuthTokenStorage, multitenancy flags,
pluginDirectories, cloud-no-id (PR #1479), reasoningSummary / contextTier /
resume largeOutput parity gaps, three new event types, and the schema
bump. Deferred items called out: PR #1428 (multitenancy client mode),
the configDir → configDirectory rename (PR #1482), Canvas runtime,
MCP Apps enableMcpApps.

doc/reference/API.md: new session-config options table entries with
wire-key annotations and upstream PR references; new event types in
the event-type table; cloud-no-id behaviour noted on :cloud; resume
table mentions :large-output forwarded on session.resume; send! options
table gains :agent-mode and :display-prompt.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 30, 2026 11:07
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR syncs the Clojure SDK with upstream Copilot SDK schema 1.0.56-1, adding new session config/message options, new schema event types, and cloud session server-assigned ID handling.

Changes:

  • Adds round 6 config/message options such as storage modes, multitenancy flags, plugin directories, :agent-mode, and :display-prompt.
  • Implements cloud session.create without caller-supplied sessionId using an inline JSON-RPC response callback.
  • Regenerates schema/event specs and updates tests, API reference, and changelog for new upstream events and options.
Show a summary per file
File Description
.copilot-schema-version Advances pinned schema version to 1.0.56-1.
CHANGELOG.md Documents round 6 additions and deferred work.
doc/reference/API.md Updates API reference for new options and event types.
schemas/README.md Updates documented schema pin.
schemas/api.schema.json Updates upstream API schema snapshot.
schemas/session-events.schema.json Updates upstream session event schema snapshot.
src/github/copilot_sdk.clj Adds new public event types and session event grouping entries.
src/github/copilot_sdk/client.clj Adds config wire encoding and cloud server-assigned session ID flow.
src/github/copilot_sdk/generated/event_specs.clj Regenerates event specs for schema 1.0.56-1.
src/github/copilot_sdk/protocol.clj Adds inline response callback support for JSON-RPC requests.
src/github/copilot_sdk/session.clj Forwards new per-message send options.
src/github/copilot_sdk/specs.clj Adds specs for new config keys, send options, and event data.
test/github/copilot_sdk/codegen_test.clj Adds schema fixture coverage for new event data.
test/github/copilot_sdk/integration_test.clj Adds integration/spec tests for new wire options and cloud session ID behavior.

Copilot's findings

  • Files reviewed: 13/14 changed files
  • Comments generated: 6


;; Reasoning summary mode (upstream PR #813 - pre-existing parity gap).
;; Wire enum: "none" | "concise" | "detailed". Mirrors upstream's ReasoningSummary type.
(s/def ::reasoning-summary #{"none" "concise" "detailed"})
Comment thread doc/reference/API.md
Comment on lines +1398 to +1399
| `:copilot/session.autopilot_objective_changed` | Autopilot objective added/updated/removed; data: `{:operation "..." :objective {...}}` (upstream schema 1.0.56). The `:status` enum for autopilot objectives is widened to include `"active"`, `"paused"`, `"cap_reached"`, `"completed"`. |
| `:copilot/session.permissions_changed` | Per-session permission flags changed; data: `{:allow-all-permissions boolean :disable-permissions boolean}` (upstream schema 1.0.56). |
Comment thread CHANGELOG.md
Comment on lines +67 to +77
- `:copilot/hook.progress` — ephemeral progress updates from
long-running hooks. Curated `::hook.progress-data` spec exposes
`:message` (string) plus the standard `:session-id`/`:timestamp`.
- `:copilot/session.autopilot_objective_changed` — autopilot
objective updates. Generated spec carries `:operation` and
`:objective` data; the `:status` enum is widened to include
`"active"`, `"paused"`, `"cap_reached"`, `"completed"`.
- `:copilot/session.permissions_changed` — emitted when per-session
permission flags change. Curated `::session.permissions_changed-data`
spec exposes `:allow-all-permissions` and `:disable-permissions`
booleans.
Comment on lines +245 to +251
;; Round 6 additions (upstream schema 1.0.56-1).
"session.permissions_changed"
{:allow-all-permissions true
:previous-allow-all-permissions false}

"hook.progress"
{:message "extracting..."}})
Comment on lines +5263 to +5268
(deftest test-spec-session-resume-data-context-tier
(testing "::session.resume-data accepts :context-tier (round-5 consistency)"
(is (s/valid? :github.copilot-sdk.specs/session.resume-data
{:resume-time (java.time.Instant/parse "2026-05-01T00:00:00Z")
:event-count 0
:context-tier :long-context}))))
Comment thread CHANGELOG.md
Comment on lines +90 to +93
- **`configDir` → `configDirectory` and `outputDir` → `outputDirectory`
rename (upstream PR #1482)** — Wire keys stay `configDir`/`outputDir`.
Renaming the Clojure-side option keys would be breaking; tracked
alongside the other rename PRs (#1357 etc.) for a coordinated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants