Skip to content

Harden config persistence and orphan cleanup against disk exhaustion / config-load failure #4197

Description

@ThomasK33

Summary

During a September 9, 2026 incident in a Coder dogfood workspace, disk-full errors preceded detection of an incomplete Mux config and fallback to an empty registry. The UI appeared to have lost all projects/workspaces; the user restored the config.

Disk exhaustion is the leading trigger, not a proven explanation of the exact write failure. It is not yet established whether exhaustion occurred at the workspace volume, quota, or underlying shared host layer.

Evidence reported from the workspace investigation

  • At 16:39 and 16:43 UTC, the live Mux server log reported ENOSPC: no space left on device during metadata saves, status persistence, workspace-recency updates, and workflow lock-directory creation.
  • At 16:43:41 UTC, Mux rejected the config and preserved it as ~/.mux/config.json.corrupt-1788972221254.
  • That backup was exactly 684,032 bytes; JSON parsing failed at end of file. Separate extension metadata also became invalid around the same time.
  • The running Mux Node process had been alive since 08:04:37 UTC. The observed disappearance therefore did not require a server-process restart; a UI reload/reconnect may have looked like one.
  • After recovery, the config parsed successfully and contained 31 projects. This does not establish that every historical session was recovered.
  • Relevant server log: ~/.coder-modules/coder/mux/logs/mux.log. In this instance, ~/.xum resolves to ~/.mux.

These findings were reported in the incident investigation; this issue does not include raw logs or config contents because they may contain sensitive data.

Secondary risk: orphan cleanup after empty fallback

The code audit identified that cleanupOrphanSessionDirs removes session directories absent from the config when idle for more than 24 hours. If config loading fails and Mux substitutes an empty registry, otherwise valid sessions could consequently be treated as orphans on startup.

No actual orphan-session deletion was found in the retained server logs for this incident. This is a potential amplification path, not a confirmed loss mechanism here.

Expected behavior / investigation

  • Reproduce config persistence under ENOSPC and other write/flush/rename failures; identify how incomplete content reached the live config despite the apparent atomic-replacement write path.
  • Preserve the last known-good config on failed saves, and surface a clear actionable storage/persistence error rather than silently presenting an empty installation.
  • Fail closed for destructive orphan cleanup when config loading failed or the registry is an untrusted fallback. A failed load must not establish that sessions are orphaned.
  • Add regression tests covering failed persistence, invalid/incomplete config recovery, and cleanup suppression after config-load failure.

Unknowns

The exact writer/failing operation, affected build/commit, and exhausted storage layer still need confirmation. Another user reported a similar empty-config incident, but a shared cause has not been established. No PR is identified as the cause.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions