`netlify dev` errors and hangs for large number of files

[ianlewis]

Describe the bug

I have a large site with lots of files.

 ➜  find _site/ -type f | wc -l
302005

When running netlify dev I get the following error message printed many times over continuously in the terminal.

 ›   Error: Netlify CLI has terminated unexpectedly
This is a problem with the Netlify CLI, not with your application.
If you recently updated the CLI, consider reverting to an older version by running:

npm install -g netlify-cli@VERSION

You can use any version from https://ntl.fyi/cli-versions.

Please report this problem at https://ntl.fyi/cli-error including the error details below.

While it says "including the error details below", there are no other error details printed.

The CPU is pegged at 100% and the server does not respond to HTTP requests. The Netlify CLI seems to be continuously starting subshells which are not waited on, creating many zombie processes. There are thousands of these started within a few seconds of running netlify dev.

 ➜  ps -ef
...
ian      2970231 2952121  0 07:23 pts/2    00:00:00 [sh] <defunct>
ian      2970233 2952121  0 07:23 pts/2    00:00:00 [sh] <defunct>
ian      2970244 2952121  0 07:23 pts/2    00:00:00 [sh] <defunct>
ian      2970246 2952121  0 07:23 pts/2    00:00:00 [sh] <defunct>
...

I notice subshells running npm in the output of ps from the same parent process.

ian      3006104 2952121  0 07:25 pts/2    00:00:00 /bin/sh -c npm -v

ian      3006017 2952121  0 07:25 pts/2    00:00:00 /bin/sh -c npm get prefix --global

The cli does not respond to Ctrl-C and I need to kill it with SIGKILL.

If I remove most of the files from the _site directory, the errors go away and the remainder of the files are served normally.

Steps to reproduce

1. Run netlify dev

Configuration

[build]
base = "."
publish = "_site/"  # relative to `base` directory
command = "make fx-build"
# NOTE: This should match the Python version in `.python-version`.
environment = { PYTHON_VERSION = "3.13.5" }

[context.deploy-preview]
  # Skip deploy previews for Renovate PRs (branches starting with "renovate/")
  ignore = "sh -c '[ \"${HEAD#renovate/}\" != \"$HEAD\" ]'"

[[headers]]
  for = "/v1/*"
  [headers.values]
    # https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin
    access-control-allow-origin = "*"

Environment

 ➜  npx envinfo --system --binaries --npmPackages netlify-cli --npmGlobalPackages netlify-cli

  System:
    OS: Linux 5.15 Ubuntu 22.04 LTS 22.04 (Jammy Jellyfish)
    CPU: (4) x64 Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
    Memory: 14.01 GB / 15.51 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 22.21.1 - /home/ian/.local/share/nodenv/versions/22.21.1/bin/node
    npm: 10.9.4 - /home/ian/.local/share/nodenv/versions/22.21.1/bin/npm
  npmPackages:
    netlify-cli: 23.13.0 => 23.13.0