Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/specs/SIDECAR-LIFECYCLE-SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,16 +207,16 @@ A generation-scoped watcher reports every exit. Zero exit after acknowledged shu

### Memory Budget `[SIDECAR-PROCESS-MEMORY]`

The F# sidecar keeps its resident set under a budget, because FSharp.Compiler.Service keeps every project builder, typed tree and check result it has made (GitHub #566).
The F# sidecar keeps the memory it holds under a budget, because FSharp.Compiler.Service keeps every project builder, typed tree and check result it has made (GitHub #566).

- **Soft limit:** 2 GB by default. A fully checked twelve-project F# repository settles near 1.5 GB, so a lower default would trim a healthy workspace into a rebuild loop. Set it in megabytes with `SHARPLSP_FSHARP_MEMORY_LIMIT_MB`; a missing, malformed or non-positive value keeps the default.
- **Hard limit:** always twice the soft limit.
- **Check:** every 30 s, the process entry point measures the working set.
- Above the soft limit, the sidecar invalidates every checker cache, collects, and logs the before and after sizes. Later requests rebuild what they need.
- **Check:** every 30 s, the process entry point measures the larger of the working set and the heap the collector has committed. The working set alone reads low once the OS compresses or swaps an idle sidecar's pages, so it can never be the only measure.
- Above the soft limit, the sidecar invalidates every checker cache and its referenced-entity cache, collects, and logs the before and after sizes. Later requests rebuild what they need.
- A trim raises the soft limit to the size it trimmed at plus an eighth of the hard limit, for the life of the process. A workspace whose live set settles above the default soft limit is therefore trimmed once, not every 30 s; only growth past the raised limit trims again.
- Still above the hard limit after a trim, a sidecar older than ten minutes logs an error and exits with code `3`. The host's backoff restart ([SIDECAR-PROCESS-EXIT]) then brings up a fresh sidecar.
- A sidecar that is over the hard limit within its first ten minutes holds a workspace that large: a restart would reload straight back over the limit. It keeps serving and logs an error naming `SHARPLSP_FSHARP_MEMORY_LIMIT_MB`. The size that trim reached is the workspace's proven floor, so the hard limit rises to it plus an eighth of the hard limit for the life of the process. A later trim that gets back under that raised limit keeps serving; only one that cannot exits. Judging it by the old limit restarted it every time its caches regrew, and each restart reloaded straight back to the same size.
- **Reloads:** a workspace reload releases the builds of the options it replaced.
- **Reloads:** a workspace reload releases the builds of the options it replaced, and the referenced-entity cache with them: each cached entity is a compiler symbol that holds the import tables of the build that made it.
- **GC:** the sidecar runs with `System.GC.ConserveMemory`, so the collector compacts rather than grow the heap between collections.
- **Tests:** in-process test hosts never start the budget.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ module SharpLsp.Sidecar.FSharp.Tests.FSharpMemoryRetentionTests

open System
open System.IO
open System.Runtime.CompilerServices
open System.Xml.Linq
open FSharp.Compiler.Text
open Xunit
Expand Down Expand Up @@ -86,6 +87,69 @@ let ``reloading a project releases the builds of the options it replaced`` () =
/// Long past the grace a fresh sidecar gets to load its workspace.
let private settled = TimeSpan.FromHours 1.0

/// A weak handle on a referenced assembly's cached entity: alive for as long as the
/// workspace's entity cache holds the build that made it. Not inlined, so no local of
/// the calling test keeps the entity alive.
[<MethodImpl(MethodImplOptions.NoInlining)>]
let private referencedEntity state (results: FSharp.Compiler.CodeAnalysis.FSharpCheckFileResults) =
FSharpEntityIndex.entities state results ()
|> List.find (fun entity -> entity.FullName = "System.Object")
|> WeakReference

/// Whether a full collection frees what `handle` points at.
let private collected (handle: WeakReference) =
GC.Collect()
GC.WaitForPendingFinalizers()
GC.Collect()
not handle.IsAlive

/// Check `file`, build its project's referenced entities, and return the project
/// context with a weak handle on one of them, asserting the cache holds them.
let private warmedEntities (state: FSharpWorkspace.FSharpWorkspaceState) file =
task {
let! checkedFile = FSharpWorkspace.checkFile state file
let results = fst checkedFile.Value
FSharpEntityIndex.warm state results.ProjectContext Threading.CancellationToken.None
Assert.True(entitiesBuilt state results.ProjectContext, "warming builds the referenced entities")
let handle = referencedEntity state results
Assert.False(collected handle, "the cache holds the entities of the current build")
return results.ProjectContext, handle
}

/// Each cached entity is an FCS symbol that holds the import tables of the build that
/// made it, so entities kept across a reload pin every build the workspace ever had:
/// thirteen hours of reloads held 17 import tables for 4 projects, 1.5 GB live.
/// A reload and a budget trim both release them. [SIDECAR-PROCESS-MEMORY] (GitHub #566)
[<Fact>]
let ``a reload and a trim release the referenced entities of the build they replace`` () =
withProject (fun state root file ->
task {
let project = NativePaths.Resolve(root, "Leak", "Leak.fsproj")

// Interaction 1 — a checked, warmed project holds its referenced entities.
let! context, first = warmedEntities state file
Assert.NotEmpty(FSharpEntityIndex.referencedFiles context)

// Interaction 2 — a reload releases them with the build it replaces.
addFile project 1
let! reloaded = FSharpWorkspaceLoading.loadProject state root
Assert.True(Result.isOk reloaded, $"the reload succeeds: {reloaded}")
Assert.False(entitiesBuilt state context, "the reload emptied the entity cache")
Assert.True(collected first, "nothing pins the replaced build's entities")

// Interaction 3 — rebuilt on demand, then a trim over the soft limit releases them too.
let! rebuilt, second = warmedEntities state file
let stops = Collections.Generic.List<int>()
watch state budget settled (readings [ 1500L; 1200L ]) stops.Add |> ignore
Assert.Empty(stops)
Assert.False(entitiesBuilt state rebuilt, "the trim emptied the entity cache")
Assert.True(collected second, "nothing pins the trimmed build's entities")

// Interaction 4 — the workspace still answers, rebuilding what it needs.
let! _, third = warmedEntities state file
Assert.False(collected third, "the entities are rebuilt for the current build")
})

/// One check of a settled sidecar: the budget it keeps next, and the exit codes it asked for.
let private checkOnce state values =
let stops = Collections.Generic.List<int>()
Expand Down Expand Up @@ -124,7 +188,11 @@ let ``still over the hard limit after trimming asks for a restart`` () =
Assert.Equal<int list>([ OverLimitExitCode ], snd (checkOnce state [ 3000L; 2001L ]))
Assert.False(cached state file options, "the sidecar trimmed before giving up")
Assert.Equal(3, OverLimitExitCode)
Assert.True(residentBytes () > 0L, "the live measurement reads this process's resident set")
use self = Diagnostics.Process.GetCurrentProcess()
let held = heldBytes ()
Assert.True(held > 0L, "the live measurement reads this process's memory")
Assert.True(held >= GC.GetGCMemoryInfo().TotalCommittedBytes / 2L, "swapped-out heap still counts: the committed heap is a floor")
Assert.True(held >= self.WorkingSet64 / 2L, "and so is the resident set")
})

[<Fact>]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ open SharpLsp.Sidecar.FSharp.Tests.FSharpCoverageTests
/// Whether the entities of every assembly file `context` references are built. FCS's
/// cache entry type is internal, so its option's members cannot be called from here: an
/// entry is told apart by presence alone, since a missing one is the null option.
let private entitiesBuilt (state: FSharpWorkspace.FSharpWorkspaceState) (context: FSharp.Compiler.CodeAnalysis.FSharpProjectContext) =
let internal entitiesBuilt (state: FSharpWorkspace.FSharpWorkspaceState) (context: FSharp.Compiler.CodeAnalysis.FSharpProjectContext) =
let cache = state.Entities :> FSharp.Compiler.EditorServices.IAssemblyContentCache

FSharpEntityIndex.referencedFiles context
Expand Down
16 changes: 10 additions & 6 deletions src/sidecars/SharpLsp.Sidecar.FSharp/FSharpMemoryBudget.fs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ open System.Diagnostics
open System.Threading
open Serilog

/// The resident-set limits, in bytes, the sidecar keeps under.
/// The limits, in bytes, on the memory the sidecar holds.
type Budget = { SoftBytes: int64; HardBytes: int64 }

/// The environment variable that sets the soft limit in megabytes; the hard limit is twice it.
Expand All @@ -40,13 +40,17 @@ let budgetFrom (setting: string option) =
let configuredBudget () =
Environment.GetEnvironmentVariable LimitVariable |> Option.ofObj |> budgetFrom

/// The sidecar's resident set, in bytes.
let residentBytes () =
/// The memory the sidecar holds, in bytes: the larger of its resident set and the heap
/// the collector has committed. The resident set alone reads low once the OS compresses
/// or swaps an idle sidecar's pages (141 MB resident for 2.2 GB committed), which is
/// exactly when the budget has to act.
let heldBytes () =
use self = Process.GetCurrentProcess()
self.WorkingSet64
max self.WorkingSet64 (GC.GetGCMemoryInfo().TotalCommittedBytes)

/// Drop every cached builder and check result, then hand the memory back.
/// Drop every cached builder, check result and referenced entity, then hand the memory back.
let private trim (state: FSharpWorkspace.FSharpWorkspaceState) =
state.Entities.Clear()
state.Checker.InvalidateAll()
state.Checker.ClearLanguageServiceRootCachesAndCollectAndFinalizeAllTransients()

Expand Down Expand Up @@ -107,5 +111,5 @@ let private exitProcess (code: int) =
let start (state: FSharpWorkspace.FSharpWorkspaceState) : IDisposable =
let mutable budget = configuredBudget ()
let age = Stopwatch.StartNew()
let tick _ = budget <- watch state budget age.Elapsed residentBytes exitProcess
let tick _ = budget <- watch state budget age.Elapsed heldBytes exitProcess
new Timer(tick, null, interval, interval)
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ let private clearWorkspace (state: FSharpWorkspaceState) =
state.References.Clear()
state.CSharpBuilds.Clear()
state.CompileSources.Clear()
state.Entities.Clear()
state.Checker.InvalidateAll()

/// Load EVERY project; the first one is the workspace's for project-wide queries.
Expand All @@ -52,6 +53,8 @@ let private loadProjects (state: FSharpWorkspaceState) (fsprojFiles: string arra
let! entries = fsprojFiles |> Array.map (fun path -> FSharpDesignTime.loadEntryWithRestore state.Checker fallback path autoRestore ct) |> Task.WhenAll
state.Projects.Clear()
state.References.Clear()
// Cached entities pin the import tables of the build that made them. [SIDECAR-PROCESS-MEMORY]
state.Entities.Clear()

for entry in entries do
let key = NativePaths.NormalizeFullPath entry.Path
Expand Down
Loading