WIP: atexit handling in Ansible modules

[moreati]

Pin down if (and when) callbacks added by atexit.register() are executed. Fix or document shortcomings.

refs #1360

[moreati]

I expect the test to fail, based on

➜  ansible git:(issue1360) ✗ ANSIBLE_STRATEGY=linear ansible localhost -m atexit_cleanup 
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details
localhost | CHANGED => 
    changed: true
➜  ansible git:(issue1360) ✗ ls -l /tmp/mitogen_test*
zsh: no matches found: /tmp/mitogen_test*

➜  ansible git:(issue1360) ✗ ANSIBLE_STRATEGY=mitogen_linear ansible localhost -m atexit_cleanup
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details
localhost | CHANGED => 
    changed: true
➜  ansible git:(issue1360) ✗ ls -l /tmp/mitogen_test*                                           
-rw-r--r--@ 1 alex  wheel  0 Dec  4 09:42 /tmp/mitogen_test_atexit_cleanup_canary.txt

[moreati]

In old CPython (probably 2.x) atexit was a pure Python module. The list of registered callbacks was stored in atexit._exithandlers.

In current CPython (since python/cpython@670e692) atexit is a builtin module and the list of handlers is stored internally. I don't see a way to introspect it. The number of handlers can be introspected, and maybe > 0 depending how CPython was started

$ python3.14 -S
Python 3.14.0 (main, Oct 14 2025, 21:10:22) [Clang 20.1.4 ] on darwin
>>> import atexit; atexit._ncallbacks()
1

$ python3.14   
Python 3.14.0 (main, Oct 14 2025, 21:10:22) [Clang 20.1.4 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import atexit; atexit._ncallbacks()
2

$ python3.14 -c "import atexit; print(atexit._ncallbacks())"
0

[moreati]

From -vvv output it looks like the process(es) for the target are being terminated by SIGTERM (request graceful process shutdown)

[mux 53761] 11:42:47.504339 D mitogen.parent: Router(Broker(2510)): deleting route to 3
[mux 53761] 11:42:47.504575 D mitogen.[local.53763]: MitogenProtocol(fork.53765): disconnecting
[mux 53761] 11:42:47.540454 D mitogen.parent.[local.53763]: Process fork.53765 pid 53765: exited due to signal 15 (SIGTERM)
[mux 53761] 11:42:47.540868 D mitogen.[local.53763]: Broker(01a0): force disconnecting <Side of parent fd 5>
[mux 53761] 11:42:47.541144 D mitogen.[local.53763]: parent stream is gone, dying.
[mux 53761] 11:42:47.541417 D mitogen.[local.53763]: Broker(01a0): shutting down
[mux 53761] 11:42:47.541711 D mitogen: <Side of local.53763 fd 105>: empty read, disconnecting
[mux 53761] 11:42:47.542019 D mitogen.parent: PopenProcess local.53763 pid 53763: exited due to signal 15 (SIGTERM)
[mux 53761] 11:42:47.542289 I ansible_mitogen.services: ContextService(): Forgetting Context(2, 'local.53763') due to stream disconnect
[mux 53761] 11:42:47.542549 D mitogen.route_monitor: stream local.53763 is gone; propagating DEL_ROUTE for {2}

atexit doesn't run handlers in that case (emphasis mine)

functions registered via this module are not called when the program is killed by a signal not handled by Python, when a Python fatal internal error is detected, or when os._exit() is called.

[moreati]

Two third party packages worth perusing

1. https://pypi.org/project/multiexit/

   A better, saner and more useful atexit replacement for Python 3 that supports multiprocessing.

2. https://pypi.org/project/safe-exit/

   Safe Exit is a Python package that provides functionality to handle graceful process termination. The package allows users to register functions that will be called when the program exits.

The limitation is that third-party packages (e.g. kubernetes client) are already using these, so they may not be applicable as is.

[moreati]

Reproduced with just Mitogen

import atexit
import sys
import os

import mitogen.master


def cleanup(file):
    try:
        os.unlink(file)
    except FileNotFoundError:
        pass


def foo(file):
    atexit.register(cleanup, file)
    with open(file, 'wb') as f:
        f.truncate()
    return 42


if __name__ == '__main__':
    broker = mitogen.master.Broker()
    router = mitogen.master.Router(broker)
    context = router.local(python_path=sys.executable)

    canary = '/tmp/issue1360_canary.txt'
    cleanup(canary)

    result = context.call(foo, canary)
    print(result)

➜  mitogen git:(issue1360) ✗ python3 issue1360.py
42
➜  mitogen git:(issue1360) ✗ ls /tmp/issue1360_canary.txt                  
/tmp/issue1360_canary.txt