Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions .github/workflows/gpui.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,28 @@ jobs:
- name: Mac app-library runtime
run: make _runtime_macos

- name: Cache GPUI host
uses: actions/cache@v6
- name: Restore GPUI host
id: gpui-cache
uses: actions/cache/restore@v6
with:
path: src/toolchain/hosts/gpui/target
key: ${{ runner.os }}-gpui-host-${{ hashFiles('src/toolchain/hosts/gpui/Cargo.lock', 'src/toolchain/hosts/gpui/rust-toolchain*') }}
restore-keys: ${{ runner.os }}-gpui-host-

- name: Build the host and every app archive
run: |
make _gpui-apps
OSPREY_GPUI_PREBUILT=1 bash src/toolchain/hosts/gpui/run.sh --build

# Saved once the host is built, not in a post step, as on Linux: PR #413's cold build used the
# job's budget and its post-step save was lost to the timeout, so every rerun started cold again.
- name: Save GPUI host
if: steps.gpui-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v6
with:
path: src/toolchain/hosts/gpui/target
key: ${{ steps.gpui-cache.outputs.cache-primary-key }}

- name: Every desktop app's self-test in a real window
run: make _gpui-self-test GPUI_WINDOW=

Expand Down
9 changes: 5 additions & 4 deletions docs/plans/lsp-memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,10 @@ Checked against the published v0.4.0 source on 2026-11-16, and confirmed on TMC
## Upgrade steps (after lspkit publishes)

- [x] Agreed on TMC (2026-11-16). lspkit's repairs and heap proofs are green locally, still `0.0.0-dev`, and have no release ETA. After its `make test` passes and the release is published, it sends `osprey-lsp-memory` the version and any API breaks in the surfaces above.
- [ ] lspkit publishes the release with all five repairs.
- [ ] Bump `lspkit`, `lspkit-server` and `lsp-profiler` together in `src/toolchain/Cargo.toml` and refresh `Cargo.lock`. Keep `sysinfo` matched to the version `lsp-profiler` pins.
- [ ] Bump `@nimblesite/lsp-profiler-ui` in `src/toolchain/vscode-extension/package.json` if lspkit publishes a matching UI release.
- [ ] Fix any API breaks in `osprey-lsp`; no shims.
- [x] lspkit publishes the release with all five repairs (v0.6.0, 2026-11-18; it also drops humantime and chrono).
- [x] Bump `lspkit`, `lspkit-server` and `lsp-profiler` to 0.6.0 together in `src/toolchain/Cargo.toml` and refresh `Cargo.lock`. Keep `sysinfo` matched to the version `lsp-profiler` pins.
- [x] Bump `@nimblesite/lsp-profiler-ui` (to 0.6.0) in `src/toolchain/vscode-extension/package.json` if lspkit publishes a matching UI release.
- [x] Fix any API breaks in `osprey-lsp`; no shims. None: it builds unchanged and the 259 LSP and 21 profiler tests pass.
- [ ] Keep Osprey's own guard: 32 unknown-method requests through the real server leave `in_flight: 0` (`Dispatcher` `Debug`).
- [ ] `cargo test -p osprey-lsp`, then `make ci`, then one coordinated push.
- [ ] Re-measure with `heap -s` after the idle sweep: live heap at or below 128 MB.
Expand All @@ -42,3 +42,4 @@ Checked against the published v0.4.0 source on 2026-11-16, and confirmed on TMC
## Open, not yet fixed

- [x] The server quit with `analysis backlog limit reached` when 256 messages were queued, and VS Code restarted it until it gave up. Now requests that don't fit get `ServerCancelled` (-32802) so the client retries, notifications wait for room, and the session never ends. Test: `a_request_flood_is_answered_and_never_ends_the_session`. In PR #411.
- [x] Idle CPU at 100% (#384). Every watched save and every tab close re-queued the whole repository sweep, and the cache keeps only 4 closed projects, so each save re-checked all 191 projects. A watched change or a close now re-checks only the projects it affects, plus newly discovered files, and an in-progress sweep is kept, not restarted. Test: `watched_changes_recheck_only_the_projects_they_affect` (8 projects; one save went from 8 analyses to 1).
41 changes: 6 additions & 35 deletions src/toolchain/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions src/toolchain/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -59,16 +59,16 @@ pulldown-cmark = { version = "0.13.6", default-features = false, features = ["ht
# Language-server building blocks from the published lspkit crates.
# [LSP-REUSE-LSPKIT] The LSP is wired in-process, so only the transport, VFS and
# engine layers are pulled in (no tree-sitter feature).
lspkit = "0.5.0"
lspkit-server = "0.5.0"
lspkit = "0.6.0"
lspkit-server = "0.6.0"
tokio = { version = "1", features = ["rt-multi-thread", "macros", "io-std", "io-util", "sync"] }
async-trait = "0.1"
futures-util = "0.3"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
# lspkit's shared CPU and memory profiler. The published crate ships its
# generated wire model, so a build needs neither Node.js nor typeDiagram.
lsp-profiler = "0.5.0"
lsp-profiler = "0.6.0"
# The same version lsp-profiler pins, so the graph carries one sysinfo.
sysinfo = { version = "0.39.3", default-features = false, features = ["system", "user"] }
tracing = "0.1"
Expand Down
15 changes: 13 additions & 2 deletions src/toolchain/crates/osprey-lsp/src/server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -152,12 +152,12 @@ async fn notify(
scope::lock(engine).opened(doc.as_str(), false);
engine.project_cache.opened(doc.as_str(), false);
scope::publish_siblings(engine, bus, &doc).await;
scope::refresh(engine, bus).await;
scope::rescope(engine, bus, &[doc]).await;
}
}
"workspace/didChangeWatchedFiles" => {
forget_deleted(engine, &params);
scope::refresh(engine, bus).await;
scope::rescope(engine, bus, &watched(&params)).await;
}
"$/cancelRequest" => {
if let Some(id) = cancel_id(&params) {
Expand All @@ -174,6 +174,17 @@ const DELETED: u64 = 3;
/// A file deleted on disk leaves the project even while its tab stays open, so
/// the refresh clears its diagnostics; an edit to that buffer reopens it.
/// Implements [LSP-DIAGNOSTICS-SCOPE].
/// The documents a `didChangeWatchedFiles` notification names.
fn watched(params: &Value) -> Vec<DocumentUri> {
let changes = params.get("changes").and_then(Value::as_array);
changes
.into_iter()
.flatten()
.filter_map(|change| change.get("uri").and_then(Value::as_str))
.map(DocumentUri::new)
.collect()
}

fn forget_deleted(engine: &OspreyEngine, params: &Value) {
let changes = params.get("changes").and_then(Value::as_array);
let deleted = changes.into_iter().flatten().filter_map(|change| {
Expand Down
158 changes: 38 additions & 120 deletions src/toolchain/crates/osprey-lsp/src/server/scope.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,19 @@
//! less the files `osprey.diagnostics.exclude` names [LSP-DIAGNOSTICS-EXCLUDE].

use super::{publish, OspreyEngine};
use crate::workspace::{file_path, uri_of};
use crate::workspace::file_path;
use files::{discover, repository, resolve_root, unignored};
use lspkit::vfs::DocumentUri;
use lspkit::EngineApi;
use lspkit_server::diagnostics::DiagnosticsBatch;
use lspkit_server::DiagnosticsBus;
use serde_json::Value;
use std::collections::{BTreeSet, HashSet, VecDeque};
use std::collections::{BTreeSet, VecDeque};
use std::path::{Path, PathBuf};
use std::sync::{MutexGuard, PoisonError};

mod files;

#[derive(Debug, Default)]
pub(crate) struct Scope {
folders: Vec<PathBuf>,
Expand Down Expand Up @@ -245,127 +248,28 @@ fn wildcard(pattern: &[u8], name: &[u8]) -> bool {
}
}

fn repository(folder: &Path) -> PathBuf {
folder
.ancestors()
.find(|path| path.join(".git").exists())
.unwrap_or(folder)
.to_path_buf()
}

fn resolve_root(root: &str, folders: &[PathBuf]) -> Vec<PathBuf> {
if let Some(path) = file_path(root) {
return vec![path];
}
let path = Path::new(root);
if path.is_absolute() {
return vec![path.to_path_buf()];
}
folders.iter().map(|folder| folder.join(path)).collect()
}

/// The files `git` lists under `root`: tracked ones plus untracked files no
/// ignore rule matches — the same set the CLI's program gate reads with
/// `git ls-files` ([CI-PROGRAMS-COMPILE]). `None` when `root` is outside any
/// git work tree or git cannot answer, so a plain folder keeps every file.
fn unignored(root: &Path) -> Option<HashSet<PathBuf>> {
let listed = std::process::Command::new("git")
.arg("-C")
.arg(root)
.args([
"ls-files",
"--cached",
"--others",
"--exclude-standard",
"-z",
])
.output()
.ok()?;
listed.status.success().then(|| {
listed
.stdout
.split(|&byte| byte == 0)
.filter(|path| !path.is_empty())
.map(|path| root.join(String::from_utf8_lossy(path).into_owned()))
.collect()
})
}

fn discover(root: &Path, unignored: Option<&HashSet<PathBuf>>, documents: &mut BTreeSet<String>) {
let entries = match std::fs::read_dir(root) {
Ok(entries) => entries,
Err(error) => {
eprintln!("osprey lsp: cannot scan {}: {error}", root.display());
return;
}
};
for entry in entries {
match entry {
Ok(entry) => visit(&entry, unignored, documents),
Err(error) => eprintln!("osprey lsp: cannot read {}: {error}", root.display()),
}
}
}

fn visit(
entry: &std::fs::DirEntry,
unignored: Option<&HashSet<PathBuf>>,
documents: &mut BTreeSet<String>,
) {
let path = entry.path();
let kind = match entry.file_type() {
Ok(kind) => kind,
Err(error) => {
eprintln!("osprey lsp: cannot inspect {}: {error}", path.display());
return;
}
};
if kind.is_dir() && !excluded(&path) {
discover(&path, unignored, documents);
}
if kind.is_file() && source_file(&path) && unignored.is_none_or(|files| files.contains(&path)) {
if let Some(uri) = uri_of(&path) {
let _ = documents.insert(uri);
}
}
}

fn excluded(path: &Path) -> bool {
path.file_name()
.and_then(|name| name.to_str())
.is_some_and(|name| {
name.starts_with('.')
|| matches!(
name,
"target" | "node_modules" | "build" | "dist" | "__pycache__"
)
})
}

fn source_file(path: &Path) -> bool {
match path.extension().and_then(|extension| extension.to_str()) {
Some("osp" | "ospml") => true,
Some("py") => python_file(path),
_ => false,
}
/// Clear documents that left the scope, re-check the open ones now, and queue
/// every closed one for the sweep the work queue runs between requests.
pub(super) async fn refresh(engine: &OspreyEngine, bus: &DiagnosticsBus) {
rescan(engine, bus, None).await;
}

fn python_file(path: &Path) -> bool {
// CPython tools are not Osprey. A marker or manifest must opt in.
if std::fs::read_to_string(path)
.is_ok_and(|text| osprey_syntax::flavor_marker(&text) == Some("python"))
{
return true;
}
crate::workspace::project_root(path).is_some_and(|root| {
osprey_project::discover(&root)
.is_ok_and(|(_, files)| files.iter().any(|file| file == path))
})
/// After a watched change or a tab close, re-check only new documents and those
/// `changed` can affect: restarting the whole sweep on every save kept an idle
/// server at 100% CPU (#384).
pub(super) async fn rescope(engine: &OspreyEngine, bus: &DiagnosticsBus, changed: &[DocumentUri]) {
let affected = changed
.iter()
.flat_map(|doc| siblings(engine, doc).into_iter().chain([doc.clone()]))
.map(|doc| doc.as_str().to_string())
.collect();
rescan(engine, bus, Some(affected)).await;
}

/// Clear documents that left the scope, re-check the open ones now, and queue
/// every closed one for the sweep the work queue runs between requests.
pub(super) async fn refresh(engine: &OspreyEngine, bus: &DiagnosticsBus) {
/// Clear what left the scope, publish the open documents to re-check now, and
/// queue the closed ones: every document, or with `affected` only those and
/// the newly discovered, keeping the sweep already queued.
async fn rescan(engine: &OspreyEngine, bus: &DiagnosticsBus, affected: Option<BTreeSet<String>>) {
let (open, removed) = {
let mut scope = lock(engine);
let documents = scope.documents();
Expand All @@ -374,11 +278,25 @@ pub(super) async fn refresh(engine: &OspreyEngine, bus: &DiagnosticsBus) {
.difference(&documents)
.cloned()
.collect::<Vec<_>>();
let recheck = |uri: &&String| {
affected
.as_ref()
.is_none_or(|set| set.contains(*uri) || !scope.published.contains(*uri))
};
let (open, closed) = documents
.iter()
.filter(recheck)
.cloned()
.partition::<Vec<_>, _>(|uri| scope.open.contains(uri));
scope.pending = closed.into();
if affected.is_none() {
scope.pending.clear();
}
scope.pending.retain(|uri| documents.contains(uri));
for uri in closed {
if !scope.pending.contains(&uri) {
scope.pending.push_back(uri);
}
}
scope.published = documents;
(open, removed)
};
Expand Down
Loading
Loading