diff --git a/docs/specs/SIDECAR-LIFECYCLE-SPEC.md b/docs/specs/SIDECAR-LIFECYCLE-SPEC.md index 1d1354cd..b9eec698 100644 --- a/docs/specs/SIDECAR-LIFECYCLE-SPEC.md +++ b/docs/specs/SIDECAR-LIFECYCLE-SPEC.md @@ -207,16 +207,16 @@ A generation-scoped watcher reports every exit. Zero exit after acknowledged shu ### Memory Budget `[SIDECAR-PROCESS-MEMORY]` -The F# sidecar keeps its resident set under a budget, because FSharp.Compiler.Service keeps every project builder, typed tree and check result it has made (GitHub #566). +The F# sidecar keeps the memory it holds under a budget, because FSharp.Compiler.Service keeps every project builder, typed tree and check result it has made (GitHub #566). - **Soft limit:** 2 GB by default. A fully checked twelve-project F# repository settles near 1.5 GB, so a lower default would trim a healthy workspace into a rebuild loop. Set it in megabytes with `SHARPLSP_FSHARP_MEMORY_LIMIT_MB`; a missing, malformed or non-positive value keeps the default. - **Hard limit:** always twice the soft limit. -- **Check:** every 30 s, the process entry point measures the working set. - - Above the soft limit, the sidecar invalidates every checker cache, collects, and logs the before and after sizes. Later requests rebuild what they need. +- **Check:** every 30 s, the process entry point measures the larger of the working set and the heap the collector has committed. The working set alone reads low once the OS compresses or swaps an idle sidecar's pages, so it can never be the only measure. + - Above the soft limit, the sidecar invalidates every checker cache and its referenced-entity cache, collects, and logs the before and after sizes. Later requests rebuild what they need. - A trim raises the soft limit to the size it trimmed at plus an eighth of the hard limit, for the life of the process. A workspace whose live set settles above the default soft limit is therefore trimmed once, not every 30 s; only growth past the raised limit trims again. - Still above the hard limit after a trim, a sidecar older than ten minutes logs an error and exits with code `3`. The host's backoff restart ([SIDECAR-PROCESS-EXIT]) then brings up a fresh sidecar. - A sidecar that is over the hard limit within its first ten minutes holds a workspace that large: a restart would reload straight back over the limit. It keeps serving and logs an error naming `SHARPLSP_FSHARP_MEMORY_LIMIT_MB`. The size that trim reached is the workspace's proven floor, so the hard limit rises to it plus an eighth of the hard limit for the life of the process. A later trim that gets back under that raised limit keeps serving; only one that cannot exits. Judging it by the old limit restarted it every time its caches regrew, and each restart reloaded straight back to the same size. -- **Reloads:** a workspace reload releases the builds of the options it replaced. +- **Reloads:** a workspace reload releases the builds of the options it replaced, and the referenced-entity cache with them: each cached entity is a compiler symbol that holds the import tables of the build that made it. - **GC:** the sidecar runs with `System.GC.ConserveMemory`, so the collector compacts rather than grow the heap between collections. - **Tests:** in-process test hosts never start the budget. diff --git a/src/sidecars/SharpLsp.Sidecar.FSharp.Tests/FSharpMemoryRetentionTests.fs b/src/sidecars/SharpLsp.Sidecar.FSharp.Tests/FSharpMemoryRetentionTests.fs index 9b8d7af3..7f5967ba 100644 --- a/src/sidecars/SharpLsp.Sidecar.FSharp.Tests/FSharpMemoryRetentionTests.fs +++ b/src/sidecars/SharpLsp.Sidecar.FSharp.Tests/FSharpMemoryRetentionTests.fs @@ -5,6 +5,7 @@ module SharpLsp.Sidecar.FSharp.Tests.FSharpMemoryRetentionTests open System open System.IO +open System.Runtime.CompilerServices open System.Xml.Linq open FSharp.Compiler.Text open Xunit @@ -86,6 +87,69 @@ let ``reloading a project releases the builds of the options it replaced`` () = /// Long past the grace a fresh sidecar gets to load its workspace. let private settled = TimeSpan.FromHours 1.0 +/// A weak handle on a referenced assembly's cached entity: alive for as long as the +/// workspace's entity cache holds the build that made it. Not inlined, so no local of +/// the calling test keeps the entity alive. +[] +let private referencedEntity state (results: FSharp.Compiler.CodeAnalysis.FSharpCheckFileResults) = + FSharpEntityIndex.entities state results () + |> List.find (fun entity -> entity.FullName = "System.Object") + |> WeakReference + +/// Whether a full collection frees what `handle` points at. +let private collected (handle: WeakReference) = + GC.Collect() + GC.WaitForPendingFinalizers() + GC.Collect() + not handle.IsAlive + +/// Check `file`, build its project's referenced entities, and return the project +/// context with a weak handle on one of them, asserting the cache holds them. +let private warmedEntities (state: FSharpWorkspace.FSharpWorkspaceState) file = + task { + let! checkedFile = FSharpWorkspace.checkFile state file + let results = fst checkedFile.Value + FSharpEntityIndex.warm state results.ProjectContext Threading.CancellationToken.None + Assert.True(entitiesBuilt state results.ProjectContext, "warming builds the referenced entities") + let handle = referencedEntity state results + Assert.False(collected handle, "the cache holds the entities of the current build") + return results.ProjectContext, handle + } + +/// Each cached entity is an FCS symbol that holds the import tables of the build that +/// made it, so entities kept across a reload pin every build the workspace ever had: +/// thirteen hours of reloads held 17 import tables for 4 projects, 1.5 GB live. +/// A reload and a budget trim both release them. [SIDECAR-PROCESS-MEMORY] (GitHub #566) +[] +let ``a reload and a trim release the referenced entities of the build they replace`` () = + withProject (fun state root file -> + task { + let project = NativePaths.Resolve(root, "Leak", "Leak.fsproj") + + // Interaction 1 — a checked, warmed project holds its referenced entities. + let! context, first = warmedEntities state file + Assert.NotEmpty(FSharpEntityIndex.referencedFiles context) + + // Interaction 2 — a reload releases them with the build it replaces. + addFile project 1 + let! reloaded = FSharpWorkspaceLoading.loadProject state root + Assert.True(Result.isOk reloaded, $"the reload succeeds: {reloaded}") + Assert.False(entitiesBuilt state context, "the reload emptied the entity cache") + Assert.True(collected first, "nothing pins the replaced build's entities") + + // Interaction 3 — rebuilt on demand, then a trim over the soft limit releases them too. + let! rebuilt, second = warmedEntities state file + let stops = Collections.Generic.List() + watch state budget settled (readings [ 1500L; 1200L ]) stops.Add |> ignore + Assert.Empty(stops) + Assert.False(entitiesBuilt state rebuilt, "the trim emptied the entity cache") + Assert.True(collected second, "nothing pins the trimmed build's entities") + + // Interaction 4 — the workspace still answers, rebuilding what it needs. + let! _, third = warmedEntities state file + Assert.False(collected third, "the entities are rebuilt for the current build") + }) + /// One check of a settled sidecar: the budget it keeps next, and the exit codes it asked for. let private checkOnce state values = let stops = Collections.Generic.List() @@ -124,7 +188,11 @@ let ``still over the hard limit after trimming asks for a restart`` () = Assert.Equal([ OverLimitExitCode ], snd (checkOnce state [ 3000L; 2001L ])) Assert.False(cached state file options, "the sidecar trimmed before giving up") Assert.Equal(3, OverLimitExitCode) - Assert.True(residentBytes () > 0L, "the live measurement reads this process's resident set") + use self = Diagnostics.Process.GetCurrentProcess() + let held = heldBytes () + Assert.True(held > 0L, "the live measurement reads this process's memory") + Assert.True(held >= GC.GetGCMemoryInfo().TotalCommittedBytes / 2L, "swapped-out heap still counts: the committed heap is a floor") + Assert.True(held >= self.WorkingSet64 / 2L, "and so is the resident set") }) [] diff --git a/src/sidecars/SharpLsp.Sidecar.FSharp.Tests/FSharpMultiProjectTests.fs b/src/sidecars/SharpLsp.Sidecar.FSharp.Tests/FSharpMultiProjectTests.fs index 780e4672..6d0eb4cc 100644 --- a/src/sidecars/SharpLsp.Sidecar.FSharp.Tests/FSharpMultiProjectTests.fs +++ b/src/sidecars/SharpLsp.Sidecar.FSharp.Tests/FSharpMultiProjectTests.fs @@ -20,7 +20,7 @@ open SharpLsp.Sidecar.FSharp.Tests.FSharpCoverageTests /// Whether the entities of every assembly file `context` references are built. FCS's /// cache entry type is internal, so its option's members cannot be called from here: an /// entry is told apart by presence alone, since a missing one is the null option. -let private entitiesBuilt (state: FSharpWorkspace.FSharpWorkspaceState) (context: FSharp.Compiler.CodeAnalysis.FSharpProjectContext) = +let internal entitiesBuilt (state: FSharpWorkspace.FSharpWorkspaceState) (context: FSharp.Compiler.CodeAnalysis.FSharpProjectContext) = let cache = state.Entities :> FSharp.Compiler.EditorServices.IAssemblyContentCache FSharpEntityIndex.referencedFiles context diff --git a/src/sidecars/SharpLsp.Sidecar.FSharp/FSharpMemoryBudget.fs b/src/sidecars/SharpLsp.Sidecar.FSharp/FSharpMemoryBudget.fs index d41620c1..d9947a78 100644 --- a/src/sidecars/SharpLsp.Sidecar.FSharp/FSharpMemoryBudget.fs +++ b/src/sidecars/SharpLsp.Sidecar.FSharp/FSharpMemoryBudget.fs @@ -13,7 +13,7 @@ open System.Diagnostics open System.Threading open Serilog -/// The resident-set limits, in bytes, the sidecar keeps under. +/// The limits, in bytes, on the memory the sidecar holds. type Budget = { SoftBytes: int64; HardBytes: int64 } /// The environment variable that sets the soft limit in megabytes; the hard limit is twice it. @@ -40,13 +40,17 @@ let budgetFrom (setting: string option) = let configuredBudget () = Environment.GetEnvironmentVariable LimitVariable |> Option.ofObj |> budgetFrom -/// The sidecar's resident set, in bytes. -let residentBytes () = +/// The memory the sidecar holds, in bytes: the larger of its resident set and the heap +/// the collector has committed. The resident set alone reads low once the OS compresses +/// or swaps an idle sidecar's pages (141 MB resident for 2.2 GB committed), which is +/// exactly when the budget has to act. +let heldBytes () = use self = Process.GetCurrentProcess() - self.WorkingSet64 + max self.WorkingSet64 (GC.GetGCMemoryInfo().TotalCommittedBytes) -/// Drop every cached builder and check result, then hand the memory back. +/// Drop every cached builder, check result and referenced entity, then hand the memory back. let private trim (state: FSharpWorkspace.FSharpWorkspaceState) = + state.Entities.Clear() state.Checker.InvalidateAll() state.Checker.ClearLanguageServiceRootCachesAndCollectAndFinalizeAllTransients() @@ -107,5 +111,5 @@ let private exitProcess (code: int) = let start (state: FSharpWorkspace.FSharpWorkspaceState) : IDisposable = let mutable budget = configuredBudget () let age = Stopwatch.StartNew() - let tick _ = budget <- watch state budget age.Elapsed residentBytes exitProcess + let tick _ = budget <- watch state budget age.Elapsed heldBytes exitProcess new Timer(tick, null, interval, interval) diff --git a/src/sidecars/SharpLsp.Sidecar.FSharp/FSharpWorkspaceLoading.fs b/src/sidecars/SharpLsp.Sidecar.FSharp/FSharpWorkspaceLoading.fs index 44493ef0..b87e66f7 100644 --- a/src/sidecars/SharpLsp.Sidecar.FSharp/FSharpWorkspaceLoading.fs +++ b/src/sidecars/SharpLsp.Sidecar.FSharp/FSharpWorkspaceLoading.fs @@ -34,6 +34,7 @@ let private clearWorkspace (state: FSharpWorkspaceState) = state.References.Clear() state.CSharpBuilds.Clear() state.CompileSources.Clear() + state.Entities.Clear() state.Checker.InvalidateAll() /// Load EVERY project; the first one is the workspace's for project-wide queries. @@ -52,6 +53,8 @@ let private loadProjects (state: FSharpWorkspaceState) (fsprojFiles: string arra let! entries = fsprojFiles |> Array.map (fun path -> FSharpDesignTime.loadEntryWithRestore state.Checker fallback path autoRestore ct) |> Task.WhenAll state.Projects.Clear() state.References.Clear() + // Cached entities pin the import tables of the build that made them. [SIDECAR-PROCESS-MEMORY] + state.Entities.Clear() for entry in entries do let key = NativePaths.NormalizeFullPath entry.Path