Skip to content

Supervise the atom runtime so it never outlives its caller - #267

Merged
prabhu merged 11 commits into
mainfrom
fix/cdxgen-4419-process-supervision
Oct 1, 2026
Merged

prabhu merged 11 commits into
mainfrom
fix/cdxgen-4419-process-supervision

Conversation

@prabhu

@prabhu prabhu commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

atom no longer outlives the process that started it. Part of cdxgen/cdxgen#4419.

The npm dispatcher ran the runtime (native binary or java) with spawnSync. When a caller's timeout killed the dispatcher, the runtime was reparented to init and kept its heap.

Dispatcher (wrapper/nodejs):

  • Supervises the runtime asynchronously and forwards SIGINT/TERM/HUP/QUIT; a second signal kills it.
  • Enforces ATOM_TIMEOUT: SIGTERM, then SIGKILL after ATOM_KILL_GRACE_MS, then exits 124.
  • Stops the runtime when the process named by ATOM_PARENT_PID (cdxgen sets its own pid) is gone.
  • Hands leading -XX:/-Xm* options to java on the jar provider.
  • Keeps a caller-set PHP_PARSER_BIN.

atom:

  • Exits when its dispatcher is killed outright, and stops helper processes (astgen, php-parse, atom-tools) on exit.
  • Writes slices files atomically.
  • The container images install @appthreat/atom-parsetools 1.8.0 (Stop a helper and its children when its supervisor is gone atom-parsetools#14).
  • Removes stale reachables chunks (and leftover temp files) from the output directory, not its subdirectories, before writing new ones.

CI: the musl build container now runs sbt with user.home set to the mounted home, so it finds the chen that chen-from-main publishes (this job failed on earlier PRs too).

Testing

  • New dispatcher tests use real processes and signals.
  • New tests for ProcessSupervisionTests, SliceFilesTests and ReachableChunkFilesTests; sbt test passes.
  • The cdxgen reproducers leave no process behind after a cdxgen timeout, a killed cdxgen, or a killed dispatcher. Checked with the jar and with a locally built native image (GraalVM CE 25).

Related: cdxgen/cdxgen#4433 (cdxgen side), AppThreat/chen#189, AppThreat/atom-parsetools#14.

prabhu added 7 commits October 1, 2026 10:49
The dispatcher ran the runtime (the native binary or java) with spawnSync,
so a caller's timeout killed the dispatcher and left the runtime running,
adopted by init with its heap (cdxgen/cdxgen#4419). The dispatcher now:

- forwards SIGINT/SIGTERM/SIGHUP/SIGQUIT to the runtime and waits for it;
- enforces ATOM_TIMEOUT itself, SIGTERM then SIGKILL after
  ATOM_KILL_GRACE_MS (default 10 s), and exits with status 124;
- stops the runtime when the process named by ATOM_PARENT_PID goes away,
  and passes its own pid to the runtime the same way;
- reports a runtime killed by a signal as 128 + signal number;
- hands leading -XX:/-Xmx runtime options to java on the jar provider, so
  callers can bound the heap the same way for both providers;
- keeps a caller-set PHP_PARSER_BIN instead of overwriting it.
When ATOM_PARENT_PID names a supervisor, a daemon thread checks once a
second and atom exits once that process has exited or atom has been
re-parented. This covers a dispatcher killed with SIGKILL, which cannot be
forwarded. A shutdown hook stops the helper processes atom started (astgen,
php-parse, atom-tools), so SIGTERM, a timeout or the watchdog does not leave
those behind.
A rerun into the same path with fewer flows left the earlier run's higher
numbered <base>_N.json chunks in place, and consumers that read chunks until
the first gap, or glob them, picked up stale flows.
A run stopped while writing (a timeout, a kill) left a truncated JSON file
at the final path, which consumers took for a finished slice and failed to
parse. Reachables chunks, the usages and data-flow slices and the empty
reachables result are now written to a temporary sibling and renamed into
place once complete.
The first SIGINT/SIGTERM is relayed so atom can stop cleanly. A runtime that
ignored it kept the dispatcher waiting with no way out short of killing the
dispatcher itself; a second signal now kills the runtime.
A run killed between writing a reachables file and renaming it into place
left its hidden temporary file behind; the next run's chunk cleanup now
removes it too.
prabhu added 3 commits October 1, 2026 15:18
better-files' list(filter) walks the whole tree. With the slices file in /tmp it reached
unreadable systemd-private directories and failed the reachables run, and in a project
directory it would delete same-named chunks in subdirectories. Clean-up is best effort, so
a directory that cannot be listed no longer fails the run.
Java takes user.home from the passwd entry (/root), not from HOME, so sbt looked in
/root/.ivy2/local and missed the chen that the chen-from-main step publishes into the
Ivy directory mounted at /tmp/home/.ivy2.
1.8.0 stops a helper and the processes it started once its supervisor is gone.
The npm-publish job was the only sbt job without the temporary chen-from-main step, so it could not resolve chen 4.0.0.
@prabhu
prabhu merged commit 9b97524 into main Oct 1, 2026
30 checks passed
@prabhu
prabhu deleted the fix/cdxgen-4419-process-supervision branch October 1, 2026 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-auto bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant