Bring back async API, allow API connection to LSP process #2620

andrewbranch · 2026-01-30T20:51:03Z

Brings back the async, JSON-RPC-based API client that existed in intermediate commits of Scaffold IPC-based API #711 but was ultimately deleted.
Adds a proper JSON-RPC API server for said async client (rather than abusing the LSP server as the prototype in Scaffold IPC-based API #711 did).
The new API server stack is less tightly coupled between method handlers and transport/protocol, so now a single implementation of API handlers and session management (api.Session) can be used over either STDIO or UDS / named pipe, and communicate over either JSON-RPC or the custom sync protocol we built on top of MessagePack.
The LSP now supports a custom command that initializes an (async) API server listening on a UDS / named pipe. In this mode, the data backing the API server is the Snapshot owned by the LSP session. Our VS Code extension exposes a command that calls this API initialization method and returns the API connection, which will allow third party extensions and LSP servers to query TypeScript’s LSP state.

The API surface area is still early prototype quality and very incomplete; this PR doesn’t add any methods. But the infrastructure provides the beginnings of a path toward replacing TS Server plugins, and will hopefully make it less cumbersome to maintain both a sync and async version of the API. (It’s still pretty annoying in the TS client code, thanks to async function coloring in JavaScript. Possibly future codegen can help us out a little there.)

Copilot

Pull request overview

This PR reintroduces the async JSON-RPC-based API that was prototyped in #711 and adds infrastructure for LSP-based API connections. The implementation refactors the API server stack to support both synchronous (MessagePack) and asynchronous (JSON-RPC) protocols, allowing a single api.Session to work over either STDIO or Unix domain socket/named pipe transports.

Changes:

Introduces a generic jsonrpc package that extracts common JSON-RPC types from lsproto, enabling code reuse
Adds api.Session with protocol abstraction supporting both sync (MessagePack) and async (JSON-RPC) connections
Implements LSP custom command custom/initializeAPISession to expose API connections to VS Code extensions
Splits TypeScript API client into sync and async variants with shared base interfaces

Reviewed changes

Copilot reviewed 46 out of 48 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
internal/jsonrpc/	New package with common JSON-RPC types and base protocol handling
internal/lsproto/	Refactored to use jsonrpc types, reducing code duplication
internal/api/session.go	Core API session managing requests, registries, and lifecycle
internal/api/conn_*.go	Connection implementations for sync and async request handling
internal/api/protocol_*.go	Protocol adapters for MessagePack and JSON-RPC
internal/api/transport*.go	Platform-specific transport for pipes/sockets
internal/api/callbackfs.go	Virtual filesystem with client-side callbacks
internal/lsp/server.go	LSP integration for API session initialization
_packages/api/src/async/	New async TypeScript client using vscode-jsonrpc
_packages/api/src/base/	Shared types and object registry for sync/async clients
_packages/api/test/	Split tests into sync and async variants
_extension/	VS Code command to expose API connection paths

Copilot · 2026-01-31T00:02:09Z

_packages/api/src/async/client.ts

+        this.process = spawn(options.tsserverPath, args, {
+            stdio: ["pipe", "pipe", "inherit"],
+        });
+
+        const reader = new StreamMessageReader(this.process.stdout!);
+        const writer = new StreamMessageWriter(this.process.stdin!);
+        this.connection = createMessageConnection(reader, writer);
+        this.connection.listen();
+        this.connected = true;


The connectViaSpawn method doesn't check if the spawned process fails to start or exits prematurely. If spawn fails, process.stdout or process.stdin could be null, leading to errors. Consider adding error handling for the spawned process, including listening to the 'error' and 'exit' events on the child process.

Suggested change

this.process = spawn(options.tsserverPath, args, {

stdio: ["pipe", "pipe", "inherit"],

});

const reader = new StreamMessageReader(this.process.stdout!);

const writer = new StreamMessageWriter(this.process.stdin!);

this.connection = createMessageConnection(reader, writer);

this.connection.listen();

this.connected = true;

const child = spawn(options.tsserverPath, args, {

stdio: ["pipe", "pipe", "inherit"],

});

this.process = child;

return new Promise((resolve, reject) => {

let settled = false;

const fail = (error: Error) => {

if (settled) return;

settled = true;

reject(error);

};

child.on("error", error => {

fail(new Error(`Failed to start tsgo process: ${error.message}`));

});

child.on("exit", (code, signal) => {

if (this.connected || settled) {

return;

}

const reason =

code !== null

? `exit code ${code}`

: signal !== null

? `signal ${signal}`

: "unknown reason";

fail(new Error(`tsgo process exited prematurely (${reason})`));

});

if (!child.stdout || !child.stdin) {

child.kill();

fail(new Error("Failed to establish communication with tsgo process: stdin/stdout not available"));

return;

}

const reader = new StreamMessageReader(child.stdout);

const writer = new StreamMessageWriter(child.stdin);

this.connection = createMessageConnection(reader, writer);

this.connection.listen();

this.connected = true;

if (!settled) {

settled = true;

resolve();

}

});

Copilot · 2026-01-31T00:02:09Z

_packages/api/test/api.bench.ts

-import { Bench } from "tinybench";
-import ts from "typescript";
+import { runBenchmarks as runAsyncBenchmarks } from "./api.async.bench.ts";
+import { runBenchmarks as runSyncBenchmarks } from "./api.async.bench.ts";


Line 3 imports runSyncBenchmarks from api.async.bench.ts instead of api.sync.bench.ts. This should import from the sync benchmark file.

Suggested change

import { runBenchmarks as runSyncBenchmarks } from "./api.async.bench.ts";

import { runBenchmarks as runSyncBenchmarks } from "./api.sync.bench.ts";

Copilot · 2026-01-31T00:02:09Z

internal/api/conn_sync.go

+	select {
+	case <-ctx.Done():
+		return nil, ctx.Err()
+	case <-c.closedChan:
+		return nil, ErrConnClosed
+	default:
+		// Non-blocking check - continue to read response
+	}
+


The context check on lines 150-157 uses a non-blocking select with default, which doesn't actually wait for context cancellation or closed channel. This means if the context is canceled or the connection is closed after the check but before ReadMessage, the operation continues anyway. The check should be removed since it doesn't provide the intended protection, and ReadMessage should be made interruptible via context if needed.

Suggested change

select {

case <-ctx.Done():

return nil, ctx.Err()

case <-c.closedChan:

return nil, ErrConnClosed

default:

// Non-blocking check - continue to read response

}

Copilot · 2026-01-31T00:02:10Z

internal/api/transport_unix.go

+func GeneratePipePath(name string) string {
+	return "/tmp/" + name + ".sock"


Using /tmp/ as the socket directory on Unix systems can be problematic on multi-user systems or in containerized environments. Consider using a user-specific directory like os.TempDir() which respects platform conventions and environment variables like $TMPDIR, or create sockets in $XDG_RUNTIME_DIR if available for better security and cleanup guarantees.

Copilot · 2026-01-31T00:02:10Z

internal/api/callbackfs.go

+func (fs *CallbackFS) SetConnection(ctx context.Context, conn Conn) {
+	fs.ctx = ctx
+	fs.conn = conn
+}


The SetConnection method on lines 54-57 lacks any synchronization, yet conn and ctx can be accessed concurrently by filesystem operations from multiple goroutines. This creates a data race. Consider adding a mutex or using atomic operations to ensure safe concurrent access.

Copilot · 2026-01-31T00:02:10Z

internal/api/proto.go

 }

+func parseProjectHandle(handle Handle[project.Project]) tspath.Path {
+	return tspath.Path(handle[2:])


The parseProjectHandle function on lines 74-76 assumes the handle always starts with "p." (prefix + dot), but doesn't validate this. If a malformed handle is passed, this could lead to incorrect behavior. Consider adding validation to ensure the handle has the correct prefix and format before slicing.

Suggested change

return tspath.Path(handle[2:])

s := string(handle)

if len(s) < 3 || s[0] != byte(handlePrefixProject) || s[1] != '.' {

// Return the zero value for tspath.Path on invalid handles.

return ""

}

return tspath.Path(s[2:])

Copilot · 2026-01-31T00:02:10Z

internal/api/conn_async.go

+		for id, ch := range c.pending {
+			delete(c.pending, id)
+			close(ch)


In the Close method, when canceling pending requests by closing their channels (line 206), there's a potential race condition. If a response arrives at the same time Close is called, handleResponse might try to send to the channel after it's been closed, causing a panic. Consider checking if the connection is closed in handleResponse before sending, or use a different signaling mechanism.

Suggested change

for id, ch := range c.pending {

delete(c.pending, id)

close(ch)

for id := range c.pending {

delete(c.pending, id)

Copilot · 2026-01-31T00:02:11Z

internal/lsp/server.go

+	go func() {
+		if apiErr := apiSession.Run(ctx, transport); apiErr != nil {
+			s.logger.Errorf("API session %s: %v", apiSession.ID(), apiErr)
+		}
+	}()


The goroutine started on line 1274 uses the context passed to handleInitializeAPISession, but this context is likely bound to the LSP request lifetime. When the request completes, the context may be canceled, which would terminate the API session prematurely. Consider using a background context or a context tied to the server's lifecycle instead.

Copilot · 2026-01-31T00:02:11Z

internal/api/session.go

+	s.symbolRegistryMu.Lock()
+	s.symbolRegistry = nil
+	s.symbolRegistryMu.Unlock()
+
+	s.typeRegistryMu.Lock()
+	s.typeRegistry = nil
+	s.typeRegistryMu.Unlock()


Setting the registries to nil on lines 591 and 595 instead of clearing them creates a potential race condition. If concurrent requests are still accessing these registries during shutdown, they will panic with nil pointer dereference. Consider setting a closed flag and checking it in registry access methods, or using clear() to empty the maps while keeping them non-nil.

Copilot · 2026-01-31T00:02:11Z

_packages/api/src/client.ts

-        this.channel = new SyncRpcChannel(options.tsserverPath, [
+        const args = [
            "--api",
            "-cwd",


The command-line arguments on lines 17-21 use -cwd (single dash) but this is non-standard. Most command-line tools use double dashes for full-word options (--cwd) or single dashes for single-letter options. The --api flag already uses double dashes. Consider using --cwd for consistency, or verify this matches what the Go server expects.

Suggested change

"-cwd",

"--cwd",

andrewbranch added 15 commits December 18, 2025 10:08

POC server-to-server communication via extension

79fdb2f

Improve examples

6d851f2

Merge branch 'main' into api

44b76d2

Delete example extension

ed66793

Clean up

c764bc3

Migrate sync STDIO API to use new stack

fd8c677

Fix stuff and clean up

e02f31e

Remove API package watch from extension run

bbf4793

Fix custom LSP method prefix in VS Code client

ce91da9

Restore overloads that Copilot rudely deleted

9f46886

Enable spawning async API and using over STDIO

9590b3d

Clean up SyncConn

458465d

Remove unused package.json script

e0ae116

Format

79a656f

Generate

854bcb0

andrewbranch marked this pull request as ready for review January 30, 2026 23:56

Copilot AI review requested due to automatic review settings January 30, 2026 23:56

Copilot started reviewing on behalf of andrewbranch January 30, 2026 23:57 View session

andrewbranch requested a review from jakebailey January 30, 2026 23:58

Copilot AI reviewed Jan 31, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bring back async API, allow API connection to LSP process #2620

Bring back async API, allow API connection to LSP process #2620

andrewbranch commented Jan 30, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

-        this.process = spawn(options.tsserverPath, args, {
-            stdio: ["pipe", "pipe", "inherit"],
-        });
-        const reader = new StreamMessageReader(this.process.stdout!);
-        const writer = new StreamMessageWriter(this.process.stdin!);
-        this.connection = createMessageConnection(reader, writer);
-        this.connection.listen();
-        this.connected = true;
+        const child = spawn(options.tsserverPath, args, {
+            stdio: ["pipe", "pipe", "inherit"],
+        });
+        this.process = child;
+        return new Promise((resolve, reject) => {
+            let settled = false;
+            const fail = (error: Error) => {
+                if (settled) return;
+                settled = true;
+                reject(error);
+            };
+            child.on("error", error => {
+                fail(new Error(`Failed to start tsgo process: ${error.message}`));
+            });
+            child.on("exit", (code, signal) => {
+                if (this.connected || settled) {
+                    return;
+                }
+                const reason =
+                    code !== null
+                        ? `exit code ${code}`
+                        : signal !== null
+                        ? `signal ${signal}`
+                        : "unknown reason";
+                fail(new Error(`tsgo process exited prematurely (${reason})`));
+            });
+            if (!child.stdout || !child.stdin) {
+                child.kill();
+                fail(new Error("Failed to establish communication with tsgo process: stdin/stdout not available"));
+                return;
+            }
+            const reader = new StreamMessageReader(child.stdout);
+            const writer = new StreamMessageWriter(child.stdin);
+            this.connection = createMessageConnection(reader, writer);
+            this.connection.listen();
+            this.connected = true;
+            if (!settled) {
+                settled = true;
+                resolve();
+            }
+        });

	import { runBenchmarks as runSyncBenchmarks } from "./api.async.bench.ts";
	import { runBenchmarks as runSyncBenchmarks } from "./api.sync.bench.ts";

		func GeneratePipePath(name string) string {
		return "/tmp/" + name + ".sock"

-	return tspath.Path(handle[2:])
+	s := string(handle)
+	if len(s) < 3 || s[0] != byte(handlePrefixProject) || s[1] != '.' {
+		// Return the zero value for tspath.Path on invalid handles.
+		return ""
+	}
+	return tspath.Path(s[2:])

Bring back async API, allow API connection to LSP process #2620

Are you sure you want to change the base?

Bring back async API, allow API connection to LSP process #2620

Conversation

andrewbranch commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

andrewbranch commented Jan 30, 2026 •

edited

Loading