Skip to content

Preserve original stack traces in cross-project panic handling - #3726

Open
Daniel Rosenwasser (DanielRosenwasser) with Copilot wants to merge 6 commits into
mainfrom
copilot/preserve-context-in-stack-traces
Open

Preserve original stack traces in cross-project panic handling#3726
Daniel Rosenwasser (DanielRosenwasser) with Copilot wants to merge 6 commits into
mainfrom
copilot/preserve-context-in-stack-traces

Conversation

Copilot AI commented May 6, 2026

Copy link
Copy Markdown
Contributor

Fixes #3725

handleCrossProject catches panics in worker goroutines and re-panics after RunAndWait, but the re-panic creates a new stack starting at crossproject.go:210. The actual crash site stack is lost from logs and telemetry.

Changes:

  • Add core.PanicWithStack struct to wrap a recovered panic value with its original stack trace
  • Add core.NewPanicWithStack() constructor that captures the stack and strips recovery infrastructure frames (debug.Stack, deferred recovery function, panic runtime) to avoid duplicating frames in the output
  • Store *PanicWithStack in cross-project goroutine recovery instead of formatted strings
  • Re-panic with the wrapped struct so the original stack propagates
  • In Server.recover, unwrap PanicWithStack to use the original stack for logging and telemetry reporting
  • Update sanitizeStackTrace to handle trimmed stacks that no longer start with runtime/debug.Stack()
  • Add unit tests for trimPanicRecoveryFrames
  • Add a stack sanitizer baseline test (TestSanitizedCrossProjectPanicStackTrace) demonstrating what the sanitized telemetry output looks like for cross-project panics

Before: telemetry/logs show stack starting at the re-panic site (handleCrossProject).
After: telemetry/logs show only the frames from the actual crash site onward, without duplicating recovery infrastructure:

typescript-go|>internal|>checker.(*Checker).checkExpression()
	typescript-go|>internal|>checker|>checker.go:5000
typescript-go|>internal|>ls.(*LanguageService).provideSymbolsAndEntries()
	typescript-go|>internal|>ls|>findallreferences.go:100
typescript-go|>internal|>ls.handleCrossProject[...].func1()
	typescript-go|>internal|>ls|>crossproject.go:105
typescript-go|>internal|>core.(*WorkGroup).worker()
	typescript-go|>internal|>core|>workgroup.go:50
typescript-go|>internal|>core.(*WorkGroup).Queue
	typescript-go|>internal|>core|>workgroup.go:35

Copilot AI and others added 2 commits May 6, 2026 18:29
When a panic occurs in a cross-project goroutine, wrap the recovered
panic value and its stack trace in a core.PanicWithStack struct. When
re-panicking, use this struct so the server's recover method can extract
and report the original stack trace instead of the re-panic site's stack.

Agent-Logs-Url: https://github.com/microsoft/typescript-go/sessions/3e8c5719-fcb6-448b-b817-c628dd2c0bc6

Co-authored-by: DanielRosenwasser <972891+DanielRosenwasser@users.noreply.github.com>
Agent-Logs-Url: https://github.com/microsoft/typescript-go/sessions/3e8c5719-fcb6-448b-b817-c628dd2c0bc6

Co-authored-by: DanielRosenwasser <972891+DanielRosenwasser@users.noreply.github.com>
Copilot AI changed the title [WIP] Preserve more context in cross-project stack traces Preserve original stack traces in cross-project panic handling May 6, 2026
@DanielRosenwasser

Copy link
Copy Markdown
Member

Copilot the problem is includes what's reported in telemetry.

  1. Intentionally trigger a new panic to see what a new error/stack trace would look like
  2. Report back in a comment what that stack error will show up like.
  3. Make sure the new stack is reported in telemetry.
  4. Add a test for the stack sanitizer.

Adds a test demonstrating what the sanitized telemetry stack trace
looks like when a panic occurs in a cross-project worker goroutine
and is preserved via PanicWithStack. The sanitized output shows the
actual crash site (e.g., checker.checkExpression) and full call chain
through provideSymbolsAndEntries and handleCrossProject, rather than
just showing the re-panic location.

Agent-Logs-Url: https://github.com/microsoft/typescript-go/sessions/a16961db-d2a6-42e8-b21a-c00c664561b0

Co-authored-by: DanielRosenwasser <972891+DanielRosenwasser@users.noreply.github.com>

Copilot AI commented May 6, 2026

Copy link
Copy Markdown
Contributor Author

Copilot the problem is includes what's reported in telemetry.

  1. Intentionally trigger a new panic to see what a new error/stack trace would look like
    ...

Done in 5cdf72f. Here's what the telemetry stack trace will look like with the fix:

Before (re-panic site only, context lost):

typescript-go|>internal|>ls.handleCrossProject[...]()
	typescript-go|>internal|>ls|>crossproject.go:210
typescript-go|>internal|>ls.(*LanguageService).ProvideReferences()
	typescript-go|>internal|>ls|>findallreferences.go:659
...

After (original crash site preserved via PanicWithStack):

typescript-go|>internal|>ls.handleCrossProject[...].func1.1()
	typescript-go|>internal|>ls|>crossproject.go:88
(REDACTED FRAME)
	(REDACTED FRAME)
typescript-go|>internal|>checker.(*Checker).checkExpression()
	typescript-go|>internal|>checker|>checker.go:5000
typescript-go|>internal|>ls.(*LanguageService).provideSymbolsAndEntries()
	typescript-go|>internal|>ls|>findallreferences.go:100
typescript-go|>internal|>ls.handleCrossProject[...].func1()
	typescript-go|>internal|>ls|>crossproject.go:105
typescript-go|>internal|>core.(*WorkGroup).worker()
	typescript-go|>internal|>core|>workgroup.go:50
typescript-go|>internal|>core.(*WorkGroup).Queue
	typescript-go|>internal|>core|>workgroup.go:35

The PanicWithStack.Stack field (captured by debug.Stack() in the worker goroutine) is passed directly to sanitizeStackTrace() for telemetry, so the original stack is what gets reported. Added a baseline test demonstrating this.

@DanielRosenwasser

Copy link
Copy Markdown
Member

Copilot can we avoid re-reporting parts of the stack? Right now the stack looks like

[panic error message]
B
A
B

…-B duplication

NewPanicWithStack now trims debug.Stack()/deferred-recovery/panic() frames
from the captured stack so telemetry only shows the actual crash site frames.
The sanitizer is updated to handle stacks without the runtime/debug.Stack()
marker (since those frames are trimmed). Added unit tests for trimPanicRecoveryFrames.

Agent-Logs-Url: https://github.com/microsoft/typescript-go/sessions/13260bed-9e38-4ff0-8fe9-141e0b8a2228

Co-authored-by: DanielRosenwasser <972891+DanielRosenwasser@users.noreply.github.com>

Copilot AI commented May 6, 2026

Copy link
Copy Markdown
Contributor Author

Copilot can we avoid re-reporting parts of the stack? Right now the stack looks like

...

Fixed in 70f8be4. NewPanicWithStack now strips the recovery infrastructure frames (debug.Stack, deferred recovery func, panic runtime) from the captured stack before storing it. The telemetry output now shows only the crash site without duplication:

typescript-go|>internal|>checker.(*Checker).checkExpression()
	typescript-go|>internal|>checker|>checker.go:5000
typescript-go|>internal|>ls.(*LanguageService).provideSymbolsAndEntries()
	typescript-go|>internal|>ls|>findallreferences.go:100
typescript-go|>internal|>ls.handleCrossProject[...].func1()
	typescript-go|>internal|>ls|>crossproject.go:105
typescript-go|>internal|>core.(*WorkGroup).worker()
	typescript-go|>internal|>core|>workgroup.go:50
typescript-go|>internal|>core.(*WorkGroup).Queue
	typescript-go|>internal|>core|>workgroup.go:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves panic diagnostics in the language server’s cross-project request handling by preserving the original crash-site stack trace across goroutine recovery/re-panics, ensuring logs and telemetry reflect where the panic actually occurred (rather than the re-panic site).

Changes:

  • Introduces core.PanicWithStack and stack-frame trimming (NewPanicWithStack / trimPanicRecoveryFrames) to retain the original panic stack.
  • Updates cross-project goroutine recovery to capture and re-panic with the preserved stack, and updates Server.recover to unwrap and report the original stack in logs/telemetry.
  • Extends stack sanitization to handle already-trimmed stacks and adds unit/baseline tests for trimming and sanitization.
Show a summary per file
File Description
internal/core/panic.go Adds PanicWithStack wrapper and recovery-frame trimming for preserved panic stacks.
internal/core/panic_test.go Adds unit tests for trimPanicRecoveryFrames.
internal/ls/crossproject.go Captures panics in worker goroutines as PanicWithStack and re-panics after RunAndWait.
internal/lsp/server.go Unwraps PanicWithStack in request recovery to log/report the original stack trace.
internal/lsp/stack_sanitizer.go Allows sanitizing stacks that don’t start at runtime/debug.Stack() (trimmed stacks).
internal/lsp/stack_sanitizer_test.go Adds a baseline test covering sanitized cross-project preserved stacks.
testdata/baselines/reference/lsp/stackSanitizer/crossProjectPanicStackTrace.md New reference baseline for cross-project preserved-stack sanitization output.

Copilot's findings

  • Files reviewed: 7/7 changed files
  • Comments generated: 1

Comment on lines 202 to 207
// Process existing known projects first
wg.RunAndWait()
// No need to use mu here since we are not in parallel at this point
if panicsOccured != nil {
panic(fmt.Sprintf("Panics occurred during cross-project handling: %v", panicsOccured))
if panicsOccurred != nil {
panic(panicsOccurred[0])
}
@RyanCavanaugh

Copy link
Copy Markdown
Member

Copilot address the merge conflicts

…ext-in-stack-traces

# Conflicts:
#	internal/lsp/server.go

Co-authored-by: RyanCavanaugh <6685088+RyanCavanaugh@users.noreply.github.com>
@RyanCavanaugh

Copy link
Copy Markdown
Member

Just doing some Copilot testing here, please ignore reviews

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (2)

internal/ls/crossproject.go:206

  • This discards every panic except the first append winner. Because project workers run concurrently, that winner is nondeterministic, and the previous implementation included all recovered panic values and stacks in the final report. Preserve an aggregate of the wrapped panics (and teach the server recovery path to log/sanitize it) so simultaneous project failures do not lose diagnostics.
		if panicsOccurred != nil {
			panic(panicsOccurred[0])

internal/lsp/stack_sanitizer.go:41

  • Trimmed stacks can legitimately begin in the runtime or a dependency before returning to this module. Starting at the first typescript-go/internal occurrence drops those actual crash-site frames entirely, whereas the existing sanitizer safely represents non-module frames as (REDACTED FRAME). Keep the full trimmed stack (while still rejecting stacks with no module frame) so telemetry preserves the location and ordering of external frames.
		moduleIndex := strings.Index(stack, "typescript-go/internal")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preserve more context in cross-project stack traces

5 participants