From ec3892c8a7a3a719c91b238af0b25307b7df109b Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Sun, 6 Sep 2026 18:35:53 +0200 Subject: [PATCH 01/20] feat: declared background workers + frankenphp_get_worker_handle() Background workers run a script in a loop outside the HTTP request cycle, sharing the PHP runtime with the request threads. Rebuilt on Server from #2499: a background worker attaches to a php_server through WithWorkerServerScope() like any other worker. Declared with "background" in a worker block (php_server or global) or WithWorkerBackground() in Go. name is required, match is rejected, num >= 1. The lifecycle mirrors HTTP workers: re-run on a cooperative exit, restart with a capped quadratic backoff on a crash, max_consecutive_failures fails Init() during startup only. drain() runs on shutdown, reboot and handler transitions so a parked script wakes up instead of waiting out the force-kill grace period. Their threads live outside the num_threads / max_threads budget, which describes HTTP capacity: those settings size the pool background workers never draw from, so calculateMaxThreads() resolves them against the HTTP workers alone and returns the background threads separately, for Init() to add to the totals. Nothing is subtracted back out. Every worker sees its declared name in $_SERVER['FRANKENPHP_WORKER'], HTTP workers included: the documented contract is to test its presence, not its value. Background workers also get $_SERVER['FRANKENPHP_WORKER_BACKGROUND'], so a script serving both roles can tell them apart with isset(). Both names are reserved: an env of the worker or of its server never leaks either into a worker of the other kind. The script gets one handle, frankenphp_get_worker_handle(), a stream that reaches EOF when the worker is drained, meant to carry control messages later. It is backed by a socket pair, not a pipe: on Windows PHP's php_select() only waits properly on sockets before 8.5. Streams do not own the socket (php_sockop_close() would shutdown() it on Windows), so a stream can be closed and fetched again without losing the drain signal; the read timeout is infinite so a blocking read parks as well as stream_select() does. Both ends are non-inheritable. A worker counts as ready on its first wait on the handle (select cast or read), the background analog of frankenphp_handle_request(): Init() waits for it, ready_workers counts from it, and an exit before it is a boot failure. The handle's stream ops, copied from the socket ops at MINIT, report it once per run. A run gets one stream: every call returns the same resource until the script closes it, so fetching the handle in a loop does not grow the resource list of a request that never ends. Worker names are scoped like paths: unique within a php_server or among global workers. The script sees the declared name; metrics and logs report a scoped worker as ":", with a numeric suffix on server names when two blocks resolve to the same one, never a name another block configured. The collision-driven renaming in the Caddy module is gone, and WithWorkerName() resolves within the request's server first. FRANKENPHP_WORKER held "1" in HTTP workers before, and workers of a php_server block were reported under their bare name unless it collided: both changes are called out in the docs. Two places absorb the new worker kind rather than growing a copy of what exists. The states a worker thread walks through between two runs live in workerLifecycle, embedded by both handlers, which supply only what differs: how a run starts, and what a reboot resets. And a worker without a scope now belongs to the fallback server, the one already serving the requests that have no server either, so a lookup is always a lookup in a server and the parallel registry of global workers is gone. Supersedes #2543 and #2398. --- bgworker_test.go | 497 ++++++++++++++++++++++++ caddy/app.go | 62 +-- caddy/caddy_test.go | 53 ++- caddy/config_test.go | 139 +++++-- caddy/module.go | 4 + caddy/workerconfig.go | 24 +- cgi.go | 2 +- context.go | 19 +- docs/config.md | 8 +- docs/library.md | 4 + docs/metrics.md | 4 +- docs/worker.md | 38 ++ frankenphp.c | 274 +++++++++++++ frankenphp.go | 62 ++- frankenphp.h | 4 + frankenphp.stub.php | 13 + frankenphp_arginfo.h | 9 +- metrics.go | 6 +- metrics_test.go | 2 +- options.go | 15 + phpmainthread.go | 4 + phpmainthread_test.go | 45 +-- phpthread.go | 15 +- requestoptions.go | 15 +- scaling.go | 6 +- scaling_test.go | 2 +- server.go | 76 +++- server_test.go | 49 ++- testdata/_executor.php | 2 +- testdata/bgworker/basic.php | 20 + testdata/bgworker/count.php | 13 + testdata/bgworker/crash-after-ready.php | 13 + testdata/bgworker/crash.php | 29 ++ testdata/bgworker/early-return.php | 5 + testdata/bgworker/fail-then-succeed.php | 16 + testdata/bgworker/fetch-no-wait.php | 7 + testdata/bgworker/flag.php | 14 + testdata/bgworker/named.php | 19 + testdata/bgworker/pool.php | 11 + testdata/bgworker/read.php | 11 + testdata/bgworker/stuck.php | 18 + testdata/handle-outside.php | 8 + testdata/symlinks/test/index.php | 2 +- testdata/symlinks/test/nested/index.php | 2 +- testdata/worker-name.php | 11 + threadbackgroundworker.go | 277 +++++++++++++ threadworker.go | 100 ++--- worker.go | 125 ++++-- workerextension.go | 26 +- workerextension_test.go | 30 ++ workerlifecycle.go | 69 ++++ 51 files changed, 1989 insertions(+), 290 deletions(-) create mode 100644 bgworker_test.go create mode 100644 testdata/bgworker/basic.php create mode 100644 testdata/bgworker/count.php create mode 100644 testdata/bgworker/crash-after-ready.php create mode 100644 testdata/bgworker/crash.php create mode 100644 testdata/bgworker/early-return.php create mode 100644 testdata/bgworker/fail-then-succeed.php create mode 100644 testdata/bgworker/fetch-no-wait.php create mode 100644 testdata/bgworker/flag.php create mode 100644 testdata/bgworker/named.php create mode 100644 testdata/bgworker/pool.php create mode 100644 testdata/bgworker/read.php create mode 100644 testdata/bgworker/stuck.php create mode 100644 testdata/handle-outside.php create mode 100644 testdata/worker-name.php create mode 100644 threadbackgroundworker.go create mode 100644 workerlifecycle.go diff --git a/bgworker_test.go b/bgworker_test.go new file mode 100644 index 0000000000..8699f40f35 --- /dev/null +++ b/bgworker_test.go @@ -0,0 +1,497 @@ +package frankenphp_test + +import ( + "bytes" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "runtime" + "testing" + "time" + + "github.com/dunglas/frankenphp" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// requireFileEventually asserts that `path` appears on disk before the +// deadline. Wraps require.Eventually so call sites stay short. +func requireFileEventually(t testing.TB, path string, msgAndArgs ...any) { + t.Helper() + require.Eventually(t, func() bool { + _, err := os.Stat(path) + return err == nil + }, 5*time.Second, 25*time.Millisecond, msgAndArgs...) +} + +// requireFileContentEventually waits for `path` to appear with content and +// returns it +func requireFileContentEventually(t *testing.T, path string) string { + t.Helper() + require.Eventually(t, func() bool { + b, err := os.ReadFile(path) + return err == nil && len(b) > 0 + }, 5*time.Second, 25*time.Millisecond, "file %q did not appear", path) + b, err := os.ReadFile(path) + require.NoError(t, err) + + return string(b) +} + +// TestBackgroundWorkerLifecycle boots a background worker that touches a +// sentinel file then parks on its handle. It proves the bg worker runs +// (sentinel appears) and that Shutdown returns within a reasonable time. +// The test asserts on Shutdown timing, so it manages Shutdown itself +// instead of using initServers' t.Cleanup hook. +func TestBackgroundWorkerLifecycle(t *testing.T) { + tmp := t.TempDir() + sentinel := filepath.Join(tmp, "bg-lifecycle.sentinel") + + require.NoError(t, frankenphp.Init( + frankenphp.WithWorkers("bg-lifecycle", "testdata/bgworker/basic.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(2), + )) + + requireFileEventually(t, sentinel, "background worker did not touch sentinel") + + done := make(chan struct{}) + go func() { + frankenphp.Shutdown() + close(done) + }() + + select { + case <-done: + case <-time.After(10 * time.Second): + t.Fatalf("Shutdown did not return within 10s") + } +} + +// TestBackgroundWorkerCrashRestarts boots a worker that exit(1)s on its +// first run and touches a "restarted" sentinel on its second run. The +// sentinel proves the crash-restart loop fired. +func TestBackgroundWorkerCrashRestarts(t *testing.T) { + tmp := t.TempDir() + crashMarker := filepath.Join(tmp, "bg-crash.marker") + restarted := filepath.Join(tmp, "bg-crash.restarted") + + initServers(t, + frankenphp.WithWorkers("bg-crash", "testdata/bgworker/crash.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{ + "BG_CRASH_MARKER": crashMarker, + "BG_RESTARTED_SENTINEL": restarted, + }), + ), + frankenphp.WithNumThreads(2), + ) + + requireFileEventually(t, restarted, "background worker did not restart after crash") +} + +// TestBackgroundWorkerOnServer scopes a background worker to a Server. It +// proves that the worker inherits the server env (the sentinel directory is +// declared on the server, not on the worker), that FRANKENPHP_WORKER holds +// the worker name, and that the worker does not intercept HTTP requests +// served by the same server. +func TestBackgroundWorkerOnServer(t *testing.T) { + tmp := t.TempDir() + + server, err := frankenphp.NewServer( + testDataDir, + frankenphp.WithServerName("sidekick-server"), + frankenphp.WithServerEnv(map[string]string{"BG_SENTINEL_DIR": tmp}), + ) + require.NoError(t, err) + + globalSentinel := filepath.Join(tmp, "global.sentinel") + initServers(t, + frankenphp.WithServer(server), + frankenphp.WithWorkers("jobs", "testdata/bgworker/named.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerServerScope(server), + ), + // a global worker may reuse the name: names are scoped to their server + frankenphp.WithWorkers("jobs", "testdata/bgworker/basic.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": globalSentinel}), + ), + frankenphp.WithNumThreads(3), + ) + + // named.php touches "/": the script sees + // the declared name, not the server-qualified one used by metrics and logs + requireFileEventually(t, filepath.Join(tmp, "jobs"), "background worker did not touch its per-name sentinel") + requireFileEventually(t, globalSentinel, "the global worker sharing the name did not start") + + body := serverGet(t, server, "http://example.com/index.php") + assert.Contains(t, body, "I am by birth a Genevese", "the server must still serve regular requests") +} + +// TestBackgroundWorkerValidation covers the declaration-time errors. +func TestBackgroundWorkerValidation(t *testing.T) { + t.Cleanup(frankenphp.Shutdown) + + t.Run("name is required", func(t *testing.T) { + err := frankenphp.Init( + frankenphp.WithWorkers("", "testdata/bgworker/basic.php", 1, frankenphp.WithWorkerBackground()), + frankenphp.WithNumThreads(2), + ) + require.ErrorContains(t, err, "must have an explicit name") + }) + + t.Run("num must be >= 1", func(t *testing.T) { + err := frankenphp.Init( + frankenphp.WithWorkers("bg-zero", "testdata/bgworker/basic.php", 0, frankenphp.WithWorkerBackground()), + frankenphp.WithNumThreads(2), + ) + require.ErrorContains(t, err, "must declare num >= 1") + }) + + t.Run("names are unique within a server", func(t *testing.T) { + // a global and a server-scoped worker may share a name (see + // TestBackgroundWorkerOnServer), two workers of one server may not + server, err := frankenphp.NewServer(testDataDir) + require.NoError(t, err) + err = frankenphp.Init( + frankenphp.WithServer(server), + frankenphp.WithWorkers("bg-shared", "testdata/bgworker/basic.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerServerScope(server), + ), + frankenphp.WithWorkers("bg-shared", "testdata/bgworker/named.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerServerScope(server), + ), + frankenphp.WithNumThreads(3), + ) + require.ErrorContains(t, err, "two workers in a server cannot have the same name") + }) + + t.Run("early return without the handle fails startup", func(t *testing.T) { + err := frankenphp.Init( + frankenphp.WithWorkers("bg-early", "testdata/bgworker/early-return.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerMaxFailures(2), + ), + frankenphp.WithNumThreads(2), + ) + require.ErrorContains(t, err, "frankenphp_get_worker_handle") + }) + + t.Run("fetching the handle without waiting on it fails startup", func(t *testing.T) { + err := frankenphp.Init( + frankenphp.WithWorkers("bg-no-wait", "testdata/bgworker/fetch-no-wait.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerMaxFailures(2), + ), + frankenphp.WithNumThreads(2), + ) + require.ErrorContains(t, err, "waiting on its handle") + }) + + t.Run("max_threads is rejected", func(t *testing.T) { + err := frankenphp.Init( + frankenphp.WithWorkers("bg-scaled", "testdata/bgworker/basic.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerMaxThreads(2), + ), + frankenphp.WithNumThreads(2), + ) + require.ErrorContains(t, err, "cannot set max_threads") + }) + + t.Run("two workers cannot report under the same name", func(t *testing.T) { + // scoping keeps names apart, except for a global name shaped like + // the ":" of a scoped one + server, err := frankenphp.NewServer(testDataDir, frankenphp.WithServerName("api")) + require.NoError(t, err) + err = frankenphp.Init( + frankenphp.WithServer(server), + frankenphp.WithWorkers("jobs", "testdata/bgworker/basic.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerServerScope(server), + ), + frankenphp.WithWorkers("api:jobs", "testdata/bgworker/named.php", 1, + frankenphp.WithWorkerBackground(), + ), + frankenphp.WithNumThreads(3), + ) + require.ErrorContains(t, err, `two workers cannot report under the same name: "api:jobs"`) + }) + + t.Run("an unregistered server scope is rejected", func(t *testing.T) { + unregistered, err := frankenphp.NewServer(testDataDir) + require.NoError(t, err) + err = frankenphp.Init( + frankenphp.WithWorkers("bg-orphan", "testdata/bgworker/basic.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerServerScope(unregistered), + ), + frankenphp.WithNumThreads(2), + ) + require.ErrorContains(t, err, "not passed to WithServer()") + }) + + t.Run("request matchers are rejected", func(t *testing.T) { + err := frankenphp.Init( + frankenphp.WithWorkers("bg-matched", "testdata/bgworker/basic.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerMatcher(func(*http.Request) bool { return true }), + ), + frankenphp.WithNumThreads(2), + ) + require.ErrorContains(t, err, "cannot match requests") + }) +} + +// TestBackgroundWorkerCannotHandleRequests checks that a request targeting a +// background worker by name is refused rather than dispatched to it. +func TestBackgroundWorkerCannotHandleRequests(t *testing.T) { + server, err := frankenphp.NewServer(testDataDir) + require.NoError(t, err) + initServers(t, + frankenphp.WithServer(server), + frankenphp.WithWorkers("jobs", "testdata/bgworker/basic.php", 1, frankenphp.WithWorkerBackground(), frankenphp.WithWorkerServerScope(server)), + frankenphp.WithNumThreads(2), + ) + + err = server.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest(http.MethodGet, "http://example.com/index.php", nil), frankenphp.WithWorkerName("jobs")) + require.ErrorContains(t, err, `background worker "jobs" cannot handle requests`) +} + +// TestBackgroundWorkerParksOnRead checks that a blocking read on the handle +// is a wait too: Init() returns only once the worker is ready, and the EOF +// of the drain unblocks the read so Shutdown() returns promptly. +func TestBackgroundWorkerParksOnRead(t *testing.T) { + tmp := t.TempDir() + sentinel := filepath.Join(tmp, "bg-read.sentinel") + + require.NoError(t, frankenphp.Init( + frankenphp.WithWorkers("bg-read", "testdata/bgworker/read.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(2), + )) + requireFileEventually(t, sentinel, "background worker parked on a read did not start") + + done := make(chan struct{}) + go func() { + frankenphp.Shutdown() + close(done) + }() + select { + case <-done: + case <-time.After(10 * time.Second): + t.Fatal("Shutdown did not return within 10s: the read did not observe EOF") + } +} + +// TestBackgroundWorkerRestartDrainsParkedScript checks that RestartWorkers() +// wakes a parked background script through the drain and re-runs it. +func TestBackgroundWorkerRestartDrainsParkedScript(t *testing.T) { + tmp := t.TempDir() + countFile := filepath.Join(tmp, "bg-count.log") + + initServers(t, + frankenphp.WithWorkers("bg-count", "testdata/bgworker/count.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_COUNT_FILE": countFile}), + ), + frankenphp.WithNumThreads(2), + ) + runs := func() int { + b, _ := os.ReadFile(countFile) + return bytes.Count(b, []byte("\n")) + } + require.Eventually(t, func() bool { return runs() == 1 }, 5*time.Second, 25*time.Millisecond, "background worker did not start") + + frankenphp.RestartWorkers() + + require.Eventually(t, func() bool { return runs() == 2 }, 5*time.Second, 25*time.Millisecond, "background worker was not re-run after the restart") +} + +// TestGetWorkerHandleOutsideBackgroundWorker checks the function throws on a +// regular request thread instead of handing out a stream. +func TestGetWorkerHandleOutsideBackgroundWorker(t *testing.T) { + server, err := frankenphp.NewServer(testDataDir) + require.NoError(t, err) + initServers(t, frankenphp.WithServer(server), frankenphp.WithNumThreads(1)) + + body := serverGet(t, server, "http://example.com/handle-outside.php") + + assert.Contains(t, body, "can only be called from a background worker") +} + +// TestWorkerNameInServerVars checks that every worker sees its declared name +// in FRANKENPHP_WORKER and that only background workers get the +// FRANKENPHP_WORKER_BACKGROUND flag. +func TestWorkerNameInServerVars(t *testing.T) { + sentinel := filepath.Join(t.TempDir(), "flag.txt") + server, err := frankenphp.NewServer(testDataDir) + require.NoError(t, err) + initServers(t, + frankenphp.WithServer(server), + // the flag is reserved: an env setting it must not make an HTTP + // worker look like a background one + frankenphp.WithWorkers("web", testDataDir+"worker-name.php", 1, + frankenphp.WithWorkerServerScope(server), + frankenphp.WithWorkerEnv(map[string]string{"FRANKENPHP_WORKER_BACKGROUND": "1"}), + ), + frankenphp.WithWorkers("jobs", "testdata/bgworker/flag.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerServerScope(server), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(3), + ) + + assert.Equal(t, "web http", serverGet(t, server, "http://example.com/worker-name.php")) + + flag := requireFileContentEventually(t, sentinel) + assert.Contains(t, flag, "'worker' => 'jobs'") + assert.Contains(t, flag, "'background' => 'set'") +} + +// TestBackgroundWorkerPool checks that num > 1 threads share the name, each +// parks on its own handle, and one drain wakes them all. +func TestBackgroundWorkerPool(t *testing.T) { + dir := t.TempDir() + initServers(t, + frankenphp.WithWorkers("pool", "testdata/bgworker/pool.php", 3, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL_DIR": dir}), + ), + frankenphp.WithNumThreads(4), + ) + + require.Eventually(t, func() bool { + entries, _ := os.ReadDir(dir) + return len(entries) == 3 + }, 5*time.Second, 25*time.Millisecond, "the three pool threads did not all start") + + done := make(chan struct{}) + go func() { + frankenphp.Shutdown() + close(done) + }() + select { + case <-done: + case <-time.After(10 * time.Second): + t.Fatal("Shutdown did not drain the whole pool within 10s") + } +} + +// TestBackgroundWorkerMultiEntrypoint checks that two named background +// workers of one server may share a script, since they are not matched by path. +func TestBackgroundWorkerMultiEntrypoint(t *testing.T) { + tmp := t.TempDir() + first, second := filepath.Join(tmp, "first"), filepath.Join(tmp, "second") + server, err := frankenphp.NewServer(testDataDir) + require.NoError(t, err) + initServers(t, + frankenphp.WithServer(server), + frankenphp.WithWorkers("first", "testdata/bgworker/basic.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerServerScope(server), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": first}), + ), + frankenphp.WithWorkers("second", "testdata/bgworker/basic.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerServerScope(server), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": second}), + ), + frankenphp.WithNumThreads(3), + ) + + requireFileEventually(t, first, "the first worker on the shared script did not start") + requireFileEventually(t, second, "the second worker on the shared script did not start") +} + +// TestBackgroundWorkerThreadsComeOnTop checks that background threads are +// reserved on top of num_threads: one HTTP thread plus one background worker +// starts with num_threads 1. +func TestBackgroundWorkerThreadsComeOnTop(t *testing.T) { + sentinel := filepath.Join(t.TempDir(), "bg-only.sentinel") + initServers(t, + frankenphp.WithWorkers("bg-only", "testdata/bgworker/basic.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(1), + ) + + requireFileEventually(t, sentinel, "background worker did not start with num_threads 1") +} + +// TestBackgroundWorkerBootFailuresThenSucceeds checks that boot failures below +// max_consecutive_failures are retried with the backoff and Init() still +// succeeds once a run reaches its ready point. +func TestBackgroundWorkerBootFailuresThenSucceeds(t *testing.T) { + tmp := t.TempDir() + countFile, sentinel := filepath.Join(tmp, "boots"), filepath.Join(tmp, "ready") + initServers(t, + frankenphp.WithWorkers("bg-flaky", "testdata/bgworker/fail-then-succeed.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_COUNT_FILE": countFile, "BG_SENTINEL": sentinel, "BG_FAIL_UNTIL": "2"}), + ), + frankenphp.WithNumThreads(2), + ) + + requireFileEventually(t, sentinel, "background worker did not recover from its boot failures") + boots, err := os.ReadFile(countFile) + require.NoError(t, err) + assert.Equal(t, "3", string(boots), "two boot failures then a success") +} + +// TestBackgroundWorkerCrashAfterReadyRestarts checks that a crash after the +// ready point restarts right away without counting toward +// max_consecutive_failures, and that a zero-timeout stream_select() counts +// as the wait. +func TestBackgroundWorkerCrashAfterReadyRestarts(t *testing.T) { + countFile := filepath.Join(t.TempDir(), "runs") + initServers(t, + frankenphp.WithWorkers("bg-crashy", "testdata/bgworker/crash-after-ready.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerMaxFailures(2), + frankenphp.WithWorkerEnv(map[string]string{"BG_COUNT_FILE": countFile}), + ), + frankenphp.WithNumThreads(2), + ) + + require.Eventually(t, func() bool { + b, _ := os.ReadFile(countFile) + return bytes.Count(b, []byte("\n")) >= 4 + }, 5*time.Second, 25*time.Millisecond, "the worker was not restarted after crashing past its ready point") +} + +// TestBackgroundWorkerRebootForceKillsStuckScript checks that a script +// ignoring its handle does not stall RestartWorkers() past the reboot grace +// period: the force-kill ends it and the next run parks normally. +func TestBackgroundWorkerRebootForceKillsStuckScript(t *testing.T) { + if runtime.GOOS != "linux" && runtime.GOOS != "freebsd" { + t.Skipf("force-kill cannot interrupt a blocking syscall on %s", runtime.GOOS) + } + + tmp := t.TempDir() + once, sentinel := filepath.Join(tmp, "once"), filepath.Join(tmp, "parked") + initServers(t, + frankenphp.WithWorkers("bg-stuck", "testdata/bgworker/stuck.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_ONCE": once, "BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(2), + ) + requireFileEventually(t, once, "background worker never entered its sleep") + + start := time.Now() + frankenphp.RestartWorkers() + assert.WithinDuration(t, start, time.Now(), 10*time.Second, "the reboot must force-kill the stuck script within its grace period") + + requireFileEventually(t, sentinel, "the re-run script did not park") +} diff --git a/caddy/app.go b/caddy/app.go index fcee129180..88636401c4 100644 --- a/caddy/app.go +++ b/caddy/app.go @@ -17,7 +17,6 @@ import ( "github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile" "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/dunglas/frankenphp" - "github.com/dunglas/frankenphp/internal/fastabs" ) var ( @@ -60,15 +59,14 @@ type FrankenPHPApp struct { // EXPERIMENTAL: MaxRequests sets the maximum number of requests a PHP thread handles before restarting (0 = unlimited) MaxRequests int `json:"max_requests,omitempty"` - opts []frankenphp.Option - metrics frankenphp.Metrics - ctx context.Context - logger *slog.Logger - modules []*FrankenPHPModule - usedWorkerNames map[string]bool - httpApp *caddyhttp.App - hasStarted atomic.Bool - started chan any + opts []frankenphp.Option + metrics frankenphp.Metrics + ctx context.Context + logger *slog.Logger + modules []*FrankenPHPModule + httpApp *caddyhttp.App + hasStarted atomic.Bool + started chan any } var errIni = errors.New(`"php_ini" must be in the format: php_ini "" ""`) @@ -133,7 +131,6 @@ func (f *FrankenPHPApp) Start() error { // register global workers for _, w := range f.Workers { w.FileName = repl.ReplaceKnown(w.FileName, "") - w.Name = f.createUniqueWorkerName(w, "") opts, err := w.toWorkerOptions() if err != nil { return err @@ -224,7 +221,6 @@ func (f *FrankenPHPApp) registerModule(repl *caddy.Replacer, module *FrankenPHPM for _, w := range module.Workers { w.FileName = repl.ReplaceKnown(w.FileName, "") - w.Name = f.createUniqueWorkerName(w, serverName) workerOptions, err := w.toWorkerOptions() if err != nil { return err @@ -236,37 +232,6 @@ func (f *FrankenPHPApp) registerModule(repl *caddy.Replacer, module *FrankenPHPM return nil } -// avoid name collisions for workers -// on collision, a name is first qualified with the server name -// (":") before falling back to a numeric postfix -func (f *FrankenPHPApp) createUniqueWorkerName(wc workerConfig, serverName string) string { - if f.usedWorkerNames == nil { - f.usedWorkerNames = make(map[string]bool) - } - - if wc.Name == "" { - wc.Name, _ = fastabs.FastAbs(wc.FileName) - } - - name := wc.Name - suffix := 0 - for { - if _, ok := f.usedWorkerNames[name]; !ok { - f.usedWorkerNames[name] = true - break - } - if serverName != "" { - name = serverName + ":" + wc.Name - serverName = "" - continue - } - suffix++ - name = fmt.Sprintf("%s_%d", wc.Name, suffix) - } - - return name -} - // UnmarshalCaddyfile implements caddyfile.Unmarshaler. func (f *FrankenPHPApp) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { for d.Next() { @@ -381,10 +346,13 @@ func (f *FrankenPHPApp) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { if len(wc.MatchPath) != 0 { return d.Errf(`"match" can only be used in a php_server worker block, not in a global one: %q`, wc.FileName) } - // check for duplicate workers - for _, existingWorker := range f.Workers { - if existingWorker.FileName == wc.FileName { - return d.Errf("global workers must not have duplicate filenames: %q", wc.FileName) + // check for duplicate workers; background workers are keyed + // by name, several may share a script + if !wc.Background { + for _, existingWorker := range f.Workers { + if !existingWorker.Background && existingWorker.FileName == wc.FileName { + return d.Errf("global workers must not have duplicate filenames: %q", wc.FileName) + } } } diff --git a/caddy/caddy_test.go b/caddy/caddy_test.go index b7d6c231eb..29004d10b7 100644 --- a/caddy/caddy_test.go +++ b/caddy/caddy_test.go @@ -757,6 +757,45 @@ func TestMetrics(t *testing.T) { require.NoError(t, testutil.GatherAndCompare(ctx.GetMetricsRegistry(), strings.NewReader(expectedMetrics), "frankenphp_total_threads", "frankenphp_busy_threads")) } +// TestBackgroundWorkerFromCaddyfile starts a background worker from a +// Caddyfile and checks it runs: the sentinel its script touches appears +func TestBackgroundWorkerFromCaddyfile(t *testing.T) { + sentinel := filepath.ToSlash(filepath.Join(t.TempDir(), "bg.sentinel")) + tester := caddytest.NewTester(t) + initServer(t, tester, ` + { + skip_install_trust + admin localhost:2999 + http_port `+testPort+` + https_port 9443 + + frankenphp { + worker { + file ../testdata/bgworker/basic.php + num 1 + name bg-caddy + background + env BG_SENTINEL `+sentinel+` + } + } + } + + localhost:`+testPort+` { + route { + php { + root ../testdata + } + } + } + `, "caddyfile") + + require.Eventually(t, func() bool { + _, err := os.Stat(sentinel) + + return err == nil + }, 5*time.Second, 25*time.Millisecond, "the background worker declared in the Caddyfile did not run") +} + func TestWorkerMetrics(t *testing.T) { var wg sync.WaitGroup tester := caddytest.NewTester(t) @@ -839,7 +878,7 @@ func TestWorkerMetrics(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="` + workerName + `"} 10 - # HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="` + workerName + `"} 2 ` @@ -996,7 +1035,7 @@ func TestNamedWorkerMetrics(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="my_app"} 10 - # HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="my_app"} 2 ` @@ -1092,7 +1131,7 @@ func TestAutoWorkerConfig(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="` + workerName + `"} 10 - # HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="` + workerName + `"} ` + workers + ` ` @@ -1460,7 +1499,7 @@ func TestMultiWorkersMetrics(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="service1"} 10 - # HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="service1"} 2 frankenphp_ready_workers{worker="service2"} 3 @@ -1614,7 +1653,7 @@ func TestWorkerRestart(t *testing.T) { // Check metrics expectedMetrics := ` - # HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="service"} 1 # HELP frankenphp_total_workers Total number of PHP workers for this worker @@ -1642,7 +1681,7 @@ func TestWorkerRestart(t *testing.T) { // frankenphp_ready_workers should be back to 1 even after worker restarts expectedMetrics = ` - # HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="service"} 1 # HELP frankenphp_total_workers Total number of PHP workers for this worker @@ -2113,7 +2152,7 @@ func TestSymlinkWorkerBehavior(t *testing.T) { // Accessing the worker script without worker configuration MUST fail // The script checks $_SERVER['FRANKENPHP_WORKER'] and dies if not set - tester.AssertGetResponse("http://localhost:"+testPort+"/index.php", http.StatusOK, "Error: This script must be run in worker mode (FRANKENPHP_WORKER not set to '1')\n") + tester.AssertGetResponse("http://localhost:"+testPort+"/index.php", http.StatusOK, "Error: This script must be run in worker mode (FRANKENPHP_WORKER not set)\n") }) t.Run("MultipleRequests", func(t *testing.T) { diff --git a/caddy/config_test.go b/caddy/config_test.go index 607051cbd8..d71b307bb4 100644 --- a/caddy/config_test.go +++ b/caddy/config_test.go @@ -1,7 +1,6 @@ package caddy import ( - "path/filepath" "testing" "time" @@ -81,6 +80,45 @@ func TestModuleWorkerDuplicateFilenamesFail(t *testing.T) { require.Contains(t, err.Error(), "must not have duplicate filenames", "Error message should mention duplicate filenames") } +// two global background workers may share a script, like their php_server +// counterparts and the Go API: they are keyed by name +func TestGlobalBackgroundWorkersShareAFilename(t *testing.T) { + d := caddyfile.NewTestDispenser(` + { + frankenphp { + worker { + name first + file ../testdata/worker-with-env.php + num 1 + background + } + worker { + name second + file ../testdata/worker-with-env.php + num 1 + background + } + } + }`) + app := &FrankenPHPApp{} + + require.NoError(t, app.UnmarshalCaddyfile(d)) + require.Len(t, app.Workers, 2) +} + +func TestGlobalWorkerDuplicateFilenamesFail(t *testing.T) { + d := caddyfile.NewTestDispenser(` + { + frankenphp { + worker ../testdata/worker-with-env.php + worker ../testdata/worker-with-env.php + } + }`) + app := &FrankenPHPApp{} + + require.ErrorContains(t, app.UnmarshalCaddyfile(d), "must not have duplicate filenames") +} + func TestModuleWorkersWithDifferentFilenames(t *testing.T) { // Create a test configuration with different worker filenames configWithDifferentFilenames := ` @@ -249,38 +287,73 @@ func TestModuleWorkerWithCustomName(t *testing.T) { require.Equal(t, "../testdata/worker-with-env.php", module.Workers[0].FileName, "Worker should have the correct filename") } -func TestCreateUniqueWorkerNames(t *testing.T) { - app := &FrankenPHPApp{} - filename := "../testdata/worker-with-env.php" - absFileName, _ := filepath.Abs(filename) - names := make([]string, 6) - for i := range 3 { - names[i] = app.createUniqueWorkerName(workerConfig{ - FileName: filename, - Name: "custom-worker-name", - }, "") - names[i+3] = app.createUniqueWorkerName(workerConfig{ - FileName: filename, - }, "") - } - - require.Equal(t, "custom-worker-name", names[0]) - require.Equal(t, "custom-worker-name_1", names[1]) - require.Equal(t, "custom-worker-name_2", names[2]) - require.Equal(t, absFileName, names[3]) - require.Equal(t, absFileName+"_1", names[4]) - require.Equal(t, absFileName+"_2", names[5]) +func TestWorkerBackgroundConfig(t *testing.T) { + d := caddyfile.NewTestDispenser(` + { + php_server { + worker { + name jobs + file ../testdata/worker-with-env.php + num 2 + background + } + } + }`) + module := &FrankenPHPModule{} + + require.NoError(t, module.UnmarshalCaddyfile(d)) + require.Len(t, module.Workers, 1) + require.True(t, module.Workers[0].Background) + require.Equal(t, "jobs", module.Workers[0].Name) } -func TestCreateUniqueWorkerNamesQualifiedByServer(t *testing.T) { - app := &FrankenPHPApp{} - wc := workerConfig{FileName: "../testdata/worker-with-env.php", Name: "queue"} - - require.Equal(t, "queue", app.createUniqueWorkerName(wc, "one.example.com")) - // on collision, the name is qualified with the server name - require.Equal(t, "two.example.com:queue", app.createUniqueWorkerName(wc, "two.example.com")) - // when the qualified name is also taken, fall back to the numeric postfix - require.Equal(t, "queue_1", app.createUniqueWorkerName(wc, "two.example.com")) - // workers without a server keep the numeric postfix behavior - require.Equal(t, "queue_2", app.createUniqueWorkerName(wc, "")) +func TestWorkerBackgroundRequiresName(t *testing.T) { + d := caddyfile.NewTestDispenser(` + { + php_server { + worker { + file ../testdata/worker-with-env.php + background + } + } + }`) + module := &FrankenPHPModule{} + + err := module.UnmarshalCaddyfile(d) + require.ErrorContains(t, err, `background workers must have an explicit "name"`) +} + +func TestWorkerBackgroundRequiresNum(t *testing.T) { + d := caddyfile.NewTestDispenser(` + { + php_server { + worker { + name jobs + file ../testdata/worker-with-env.php + background + } + } + }`) + module := &FrankenPHPModule{} + + err := module.UnmarshalCaddyfile(d) + require.ErrorContains(t, err, `background workers must declare "num" >= 1`) +} + +func TestWorkerBackgroundRejectsMatch(t *testing.T) { + d := caddyfile.NewTestDispenser(` + { + php_server { + worker { + name jobs + file ../testdata/worker-with-env.php + match /jobs/* + background + } + } + }`) + module := &FrankenPHPModule{} + + err := module.UnmarshalCaddyfile(d) + require.ErrorContains(t, err, `"match" is not supported for background workers`) } diff --git a/caddy/module.go b/caddy/module.go index 20dcec9ee2..ff6e2db5ff 100644 --- a/caddy/module.go +++ b/caddy/module.go @@ -315,6 +315,10 @@ func (f *FrankenPHPModule) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { // Check if a worker with this filename already exists in this module fileNames := make(map[string]struct{}, len(f.Workers)) for _, w := range f.Workers { + // background workers are keyed by name, several may share a script + if w.Background { + continue + } if _, ok := fileNames[w.FileName]; ok { return fmt.Errorf(`workers in a single "php" or "php_server" block must not have duplicate filenames: %q`, w.FileName) } diff --git a/caddy/workerconfig.go b/caddy/workerconfig.go index b39eb731e0..7f5c15b253 100644 --- a/caddy/workerconfig.go +++ b/caddy/workerconfig.go @@ -22,7 +22,7 @@ import ( type workerConfig struct { mercureContext - // Name for the worker. Default: the absolute path of the worker file, postfixed with a number if the name is already used. + // Name for the worker, unique within its php_server (or among global workers). Default: the absolute path of the worker file. Name string `json:"name,omitempty"` // FileName sets the path to the worker script. FileName string `json:"file_name,omitempty"` @@ -38,6 +38,8 @@ type workerConfig struct { MatchPath []string `json:"match_path,omitempty"` // MaxConsecutiveFailures sets the maximum number of consecutive failures before panicking (defaults to 6, set to -1 to never panick) MaxConsecutiveFailures int `json:"max_consecutive_failures,omitempty"` + // Background marks this worker as a background (non-HTTP) worker. + Background bool `json:"background,omitempty"` options []frankenphp.WorkerOption } @@ -139,8 +141,10 @@ func unmarshalWorker(d *caddyfile.Dispenser) (workerConfig, error) { } wc.MaxConsecutiveFailures = v + case "background": + wc.Background = true default: - return wc, wrongSubDirectiveError("worker", "name, file, num, env, watch, match, max_consecutive_failures, max_threads", v) + return wc, wrongSubDirectiveError("worker", "name, file, num, env, watch, match, max_consecutive_failures, max_threads, background", v) } } @@ -148,6 +152,18 @@ func unmarshalWorker(d *caddyfile.Dispenser) (workerConfig, error) { return wc, d.Err(`the "file" argument must be specified`) } + if wc.Background { + if wc.Name == "" { + return wc, d.Err(`background workers must have an explicit "name"`) + } + if len(wc.MatchPath) != 0 { + return wc, d.Err(`"match" is not supported for background workers`) + } + if wc.Num < 1 { + return wc, d.Err(`background workers must declare "num" >= 1`) + } + } + if frankenphp.EmbeddedAppPath != "" && filepath.IsLocal(wc.FileName) { wc.FileName = filepath.Join(frankenphp.EmbeddedAppPath, wc.FileName) } @@ -166,6 +182,10 @@ func (wc *workerConfig) toWorkerOptions() ([]frankenphp.WorkerOption, error) { // options collected while provisioning the module, e.g. the Mercure hub opts = append(opts, wc.options...) + if wc.Background { + opts = append(opts, frankenphp.WithWorkerBackground()) + } + // copy the caddy match logic and create a unique matcher function for this worker // inject the matcher into frankenphp if len(wc.MatchPath) > 0 { diff --git a/cgi.go b/cgi.go index 85b6a4469a..25a7b87e44 100644 --- a/cgi.go +++ b/cgi.go @@ -225,7 +225,7 @@ func splitCgiPath(fc *frankenPHPContext) { // see if a php_server worker or global worker matches the request path // aka: root + request path == worker.filename if fc.worker = fc.server.workersByPath[fc.scriptFilename]; fc.worker == nil { - fc.worker = globalWorkersByPath[fc.scriptFilename] + fc.worker = fallbackServer.workersByPath[fc.scriptFilename] } } diff --git a/context.go b/context.go index d184965fea..b0fbcd52a4 100644 --- a/context.go +++ b/context.go @@ -130,20 +130,14 @@ func newWorkerDummyContext(w *worker) (*frankenPHPContext, error) { return nil, err } - server := w.server - if server == nil { - // global worker, not associated with a server - server = fallbackServer - } - fc := &frankenPHPContext{ done: make(chan any), ctx: r.Context(), - server: server, + server: w.server, request: r, startedAt: time.Now(), // startup output of a scoped worker belongs to its server's logger - logger: server.logger, + logger: w.server.logger, worker: w, } @@ -160,11 +154,6 @@ func newWorkerDummyContext(w *worker) (*frankenPHPContext, error) { // newContextFromMessage creates a context from a message (external workers) func newContextFromMessage(message any, rw http.ResponseWriter, ctx context.Context, w *worker) *frankenPHPContext { - server := w.server - if server == nil { - server = fallbackServer - } - if ctx == nil { ctx = globalCtx } @@ -172,9 +161,9 @@ func newContextFromMessage(message any, rw http.ResponseWriter, ctx context.Cont return &frankenPHPContext{ done: make(chan any), startedAt: time.Now(), - server: server, + server: w.server, worker: w, - logger: server.logger, + logger: w.server.logger, responseWriter: rw, handlerParameters: message, ctx: ctx, diff --git a/docs/config.md b/docs/config.md index 281f05dc75..1a009c49b7 100644 --- a/docs/config.md +++ b/docs/config.md @@ -109,8 +109,9 @@ You can also explicitly configure FrankenPHP using the [global option](https://c num # Sets the number of PHP threads to start, defaults to 2x the number of available CPUs. env # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables. watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. - name # Sets the name of the worker, used in logs and metrics. Default: absolute path of worker file + name # Sets the name of the worker, used in logs and metrics and exposed as $_SERVER['FRANKENPHP_WORKER']. Must be unique among global workers. Default: absolute path of the worker file. max_consecutive_failures # Sets the maximum number of consecutive failures before the worker is considered unhealthy, -1 means the worker will always restart. Default: 6. + background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it waits on the stream returned by frankenphp_get_worker_handle() (stream_select() or a blocking read); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] is set in it, test its presence rather than its value. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. } } } @@ -187,17 +188,18 @@ php_server [] { root # Sets the root folder to the site. Default: `root` directive. split_path # Sets the substrings for splitting the URI into two parts. The first matching substring will be used to split the "path info" from the path. The first piece is suffixed with the matching substring and will be assumed as the actual resource (CGI script) name. The second piece will be set to PATH_INFO for the script to use. Default: `.php` resolve_root_symlink false # Disables resolving the `root` directory to its actual value by evaluating a symbolic link, if one exists (enabled by default). - name # Sets the name for this server, used to attribute workers, metrics and logs. Default: the first host matcher of the enclosing route, or the first listener address. + name # Sets the name for this server, used to attribute workers, metrics and logs. Default: the first host matcher of the enclosing route, or the first listener address. Suffixed with a number if another php_server resolves to the same name. env # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables. file_server off # Disables the built-in file_server directive. request_body_timeout # Sets an idle timeout on request body reads: a stalled (slow POST) client is cut off while a steady upload of any size succeeds. Default: 60s. Set to 0 to disable. worker { # Creates a worker specific to this server. Can be specified more than once for multiple workers. file # Sets the path to the worker script, can be relative to the php_server root num # Sets the number of PHP threads to start, defaults to 2x the number of available - name # Sets the name for the worker, used in logs and metrics. Default: absolute path of worker file. Postfixed with a number if name is already in use. + name # Sets the name for the worker, used in logs and metrics and exposed as $_SERVER['FRANKENPHP_WORKER']. Must be unique within this php_server. In logs and metrics, the worker is reported as ":". Default: absolute path of the worker file. watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. env # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables. Environment variables for this worker are also inherited from the php_server parent, but can be overwritten here. match # match the worker to a path pattern. Overrides try_files and can only be used in the php_server directive. + background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it waits on the stream returned by frankenphp_get_worker_handle() (stream_select() or a blocking read); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] is set in it, test its presence rather than its value. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. } worker # Can also use the short form like in the global frankenphp block. } diff --git a/docs/library.md b/docs/library.md index 7782f03182..03f0ef5898 100644 --- a/docs/library.md +++ b/docs/library.md @@ -60,6 +60,10 @@ err := frankenphp.Init( Workers declared without a server scope are global: they match by file path on any server. Since a global worker has no set of requests to match against, combining `WithWorkerMatcher()` with a global worker is a configuration error and `Init()` rejects it. +Worker names are unique within their server (or among global workers), so two servers may each declare a worker named `queue`. The script sees the declared name, while metrics and logs report a server-scoped worker as `:`; server names are made unique with a numeric suffix when needed. `WithWorkerName()` resolves a name within the request's server first, then among global workers. + +`WithWorkerBackground()` declares a [background worker](worker.md#background-workers), which runs outside the request cycle. + ## Per-request options `Server.ServeHTTP()` accepts `RequestOption`s to override the server configuration for a single request, e.g. `WithRequestDocumentRoot()`, `WithRequestSplitPath()`, `WithRequestEnv()` or `WithRequestLogger()`. diff --git a/docs/metrics.md b/docs/metrics.md index 932707265a..6239ae8829 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -19,12 +19,12 @@ When [Caddy metrics](https://caddyserver.com/docs/metrics) are enabled, FrankenP - `frankenphp_busy_workers{worker="[worker_name]"}`: The number of workers currently processing a request. - `frankenphp_worker_request_time{worker="[worker_name]"}`: The time spent processing requests by all workers. - `frankenphp_worker_request_count{worker="[worker_name]"}`: The number of requests processed by all workers. -- `frankenphp_ready_workers{worker="[worker_name]"}`: The number of workers that have called `frankenphp_handle_request` at least once. +- `frankenphp_ready_workers{worker="[worker_name]"}`: The number of workers that have reached their ready point at least once: `frankenphp_handle_request()` for HTTP workers, `frankenphp_get_worker_handle()` for background workers. - `frankenphp_worker_crashes{worker="[worker_name]"}`: The number of times a worker has unexpectedly terminated. - `frankenphp_worker_restarts{worker="[worker_name]"}`: The number of times a worker has been deliberately restarted. - `frankenphp_worker_queue_depth{worker="[worker_name]"}`: The number of queued requests. -For worker metrics, the `[worker_name]` placeholder is replaced by the worker name in the Caddyfile, otherwise the absolute path of the worker file will be used. +`[worker_name]` is the worker name from the Caddyfile, or the absolute path of the worker file when it has none. Workers of a `php_server` block are prefixed with the name of that block: `:`. They used to be reported under their bare name unless two blocks declared the same one, so dashboards and alerts built on those series need the prefix. ## Threads State Endpoint diff --git a/docs/worker.md b/docs/worker.md index 466c7cf684..b9fe70674a 100644 --- a/docs/worker.md +++ b/docs/worker.md @@ -199,6 +199,44 @@ frankenphp { } ``` +## Background workers + +This feature is experimental. + +A background worker runs its script in a loop outside the HTTP request cycle, on its own PHP thread. It is declared like any worker, with the `background` option; `name` is required and `num` must be at least 1: + +```caddyfile +php_server { + worker { + file jobs.php + num 1 + name jobs + background + } +} +``` + +The script must wait on the stream returned by `frankenphp_get_worker_handle()`, which reaches EOF when FrankenPHP drains the worker on shutdown, reboot or restart. The first wait on it marks the worker ready: the server start waits for that point, and an exit before it counts as a failure. Polling `feof()` is not a wait, block in `stream_select()` or in a read: + +```php + 0) { + // drained: return, FrankenPHP re-runs or stops the script + break; + } + + doSomeWork(); +} +``` + +`$_SERVER['FRANKENPHP_WORKER']` holds the declared name, as it does in HTTP workers, and `$_SERVER['FRANKENPHP_WORKER_BACKGROUND']` is set, so a script serving both roles can tell them apart with `isset()`. Its value is unspecified, only its presence is part of the contract. `FRANKENPHP_WORKER` used to hold `1` in HTTP workers for the same reason: a script comparing it to that value must test its presence instead. Background threads come on top of `num_threads` and `max_threads`; they don't autoscale, so `max_threads` is not allowed on them. From Go, declare one with `WithWorkerBackground()`. + ## Superglobals behavior [PHP superglobals](https://www.php.net/manual/language.variables.superglobals.php) (`$_SERVER`, `$_ENV`, `$_GET`...) diff --git a/frankenphp.c b/frankenphp.c index 2378ac8ff6..18bdeffdfe 100644 --- a/frankenphp.c +++ b/frankenphp.c @@ -13,11 +13,14 @@ #include #ifdef PHP_WIN32 #include +#include #else #include #endif +#include
#include #include +#include #include #include #include @@ -28,6 +31,7 @@ #include #include #ifndef PHP_WIN32 +#include #include #endif #if defined(__linux__) @@ -126,6 +130,18 @@ HashTable *main_thread_env = NULL; static THREAD_LOCAL uintptr_t thread_index; static THREAD_LOCAL bool is_worker_thread = false; +static THREAD_LOCAL bool is_background_worker = false; +/* Stop socket pair of a background worker thread: [0] is the script's end, + * exposed via frankenphp_get_worker_handle(); [1] is transferred to the Go + * side, which closes it to signal a drain. */ +static THREAD_LOCAL php_socket_t worker_stop_socks[2] = {SOCK_ERR, SOCK_ERR}; +/* set on the first wait on the handle of the current run, see + * frankenphp_worker_handle_ops */ +static THREAD_LOCAL bool worker_handle_waited = false; +/* the stream of the current run, see frankenphp_get_worker_handle(); the + * cache holds a ref, and the resource list of the run frees it at request + * shutdown, so the pointer is only reset, never released, between runs */ +static THREAD_LOCAL zend_resource *worker_handle_res = NULL; static THREAD_LOCAL HashTable *sandboxed_env = NULL; /* prepared_env holds entries from php(_server)'s `env KEY VAL`, exposed to * getenv() and merged into $_ENV when 'E' is in variables_order. Separate from @@ -342,7 +358,137 @@ void frankenphp_release_thread_for_kill(force_kill_slot slot) { #endif } +/* Stop channel of background workers: a socket pair. One end is exposed to + * the PHP script via frankenphp_get_worker_handle(), the other is handed to + * the Go side, which closes it on drain so the script's end reaches EOF and + * a stream_select() or a blocking read on it returns. A socket pair rather + * than a pipe because on Windows PHP's php_select() only really waits on + * sockets: before 8.5 it reports any other handle as always ready. */ +static void frankenphp_worker_close_sock(php_socket_t s) { + if (s == SOCK_ERR) { + return; + } +#ifdef PHP_WIN32 + closesocket(s); +#else + close(s); +#endif +} + +/* keep the pair out of processes the script may spawn: a child holding the + * Go side's end would keep the script's end from ever reaching EOF */ +static void frankenphp_worker_sock_no_inherit(php_socket_t s) { +#ifdef PHP_WIN32 + SetHandleInformation((HANDLE)s, HANDLE_FLAG_INHERIT, 0); +#else + fcntl(s, F_SETFD, FD_CLOEXEC); +#endif +} + +static void frankenphp_worker_close_stop_socks(void) { + for (int i = 0; i < 2; i++) { + frankenphp_worker_close_sock(worker_stop_socks[i]); + worker_stop_socks[i] = SOCK_ERR; + } +} + +static int frankenphp_worker_open_stop_pair(void) { +#ifdef PHP_WIN32 + /* PHP's emulation, a loopback TCP pair; it only accepts AF_INET, listens + * on INADDR_ANY and accepts the first peer, so check the pair is ours */ + if (socketpair(AF_INET, SOCK_STREAM, 0, worker_stop_socks) != 0) { + worker_stop_socks[0] = SOCK_ERR; + worker_stop_socks[1] = SOCK_ERR; + + return -1; + } + + struct sockaddr_in peer = {0}, local = {0}; + int peer_len = sizeof(peer), local_len = sizeof(local); + if (getpeername(worker_stop_socks[0], (struct sockaddr *)&peer, &peer_len) != + 0 || + getsockname(worker_stop_socks[1], (struct sockaddr *)&local, + &local_len) != 0 || + peer.sin_port != local.sin_port || + peer.sin_addr.s_addr != local.sin_addr.s_addr) { + frankenphp_worker_close_stop_socks(); + + return -1; + } +#else +#ifdef SOCK_CLOEXEC + int type = SOCK_STREAM | SOCK_CLOEXEC; +#else + int type = SOCK_STREAM; +#endif + if (socketpair(AF_UNIX, type, 0, worker_stop_socks) != 0) { + worker_stop_socks[0] = SOCK_ERR; + worker_stop_socks[1] = SOCK_ERR; + + return -1; + } +#endif + /* redundant where SOCK_CLOEXEC applied; a fork()ed child (pcntl) still + * inherits both ends and delays the EOF until it exits */ + frankenphp_worker_sock_no_inherit(worker_stop_socks[0]); + frankenphp_worker_sock_no_inherit(worker_stop_socks[1]); + + return 0; +} + +/* Marks the calling thread as a background worker, opens its stop socket + * pair and transfers the Go side's end to the caller (clearing the TLS slot + * so a later recycle won't double-close it). Returns -1 if the pair could + * not be created. max_execution_time is disarmed after php_request_startup() + * re-arms it, see php_thread(). */ +intptr_t frankenphp_set_background_worker_and_get_stop_sock(void) { + is_background_worker = true; + worker_handle_waited = false; + worker_handle_res = NULL; + + frankenphp_worker_close_stop_socks(); + if (frankenphp_worker_open_stop_pair() != 0) { + return -1; + } + + intptr_t s = (intptr_t)worker_stop_socks[1]; + worker_stop_socks[1] = SOCK_ERR; + + return s; +} + +/* Closes the Go side's end of a stop socket pair, which lands as EOF on the + * script's end so its stream_select() or blocking read returns promptly. */ +void frankenphp_worker_close_stop_sock(intptr_t s) { + if (s < 0) { + return; + } + frankenphp_worker_close_sock((php_socket_t)s); +} + +/* Sets max_execution_time to 0 for the current request, which also disarms + * the timer through OnUpdateTimeout(). */ +static void frankenphp_disable_execution_timeout(void) { + zend_string *key = zend_string_init("max_execution_time", + sizeof("max_execution_time") - 1, 0); + zend_string *value = zend_string_init("0", 1, 0); + zend_alter_ini_entry(key, value, PHP_INI_USER, PHP_INI_STAGE_RUNTIME); + zend_string_release(key); + zend_string_release(value); +} + void frankenphp_update_local_thread_context(bool is_worker) { + /* A thread that ran a background worker can be recycled into an HTTP + * worker or a regular request thread: reset the bg TLS so + * frankenphp_get_worker_handle() rejects callers again, and release the + * stop socket. The streams handed out by frankenphp_get_worker_handle() + * do not own it and were destroyed by request shutdown. */ + if (is_background_worker) { + is_background_worker = false; + worker_handle_res = NULL; + frankenphp_worker_close_stop_socks(); + } + is_worker_thread = is_worker; /* workers should keep running if the user aborts the connection */ @@ -857,6 +1003,15 @@ PHP_FUNCTION(frankenphp_handle_request) { RETURN_THROWS(); } + if (is_background_worker) { + /* background workers never receive HTTP requests */ + zend_throw_exception( + spl_ce_RuntimeException, + "frankenphp_handle_request() cannot be called from a background worker", + 0); + RETURN_THROWS(); + } + #ifdef ZEND_MAX_EXECUTION_TIMERS /* Disable timeouts while waiting for a request to handle */ zend_unset_timeout(); @@ -1017,6 +1172,100 @@ PHP_FUNCTION(frankenphp_log) { } } +/* Ops of the streams returned by frankenphp_get_worker_handle(): the socket + * ops, except that the first wait on the handle, a select cast or a read, + * reports the worker ready, and that closing a stream leaves the socket + * alone: it belongs to the thread, every handle of a run shares it, and it + * is closed at the next run setup or on thread exit. Waiting is the + * background analog of an HTTP worker reaching frankenphp_handle_request(): + * it comes after the script's bootstrap by construction, where merely + * fetching the handle does not. Initialized in MINIT. */ +static php_stream_ops frankenphp_worker_handle_ops; + +static void frankenphp_worker_handle_waited(void) { + if (!worker_handle_waited) { + worker_handle_waited = true; + go_frankenphp_background_worker_ready(frankenphp_thread_index()); + } +} + +static ssize_t frankenphp_worker_handle_read(php_stream *stream, char *buf, + size_t count) { + frankenphp_worker_handle_waited(); + + return php_stream_socket_ops.read(stream, buf, count); +} + +static int frankenphp_worker_handle_cast(php_stream *stream, int castas, + void **ret) { + if (castas == PHP_STREAM_AS_FD_FOR_SELECT) { + frankenphp_worker_handle_waited(); + } + + return php_stream_socket_ops.cast(stream, castas, ret); +} + +static int frankenphp_worker_handle_close(php_stream *stream, + int close_handle) { + (void)close_handle; + + /* free the stream data only, never the shared socket */ + return php_stream_socket_ops.close(stream, 0); +} + +PHP_FUNCTION(frankenphp_get_worker_handle) { + ZEND_PARSE_PARAMETERS_NONE(); + + if (!is_background_worker) { + zend_throw_exception(spl_ce_RuntimeException, + "frankenphp_get_worker_handle() can only be called " + "from a background worker", + 0); + RETURN_THROWS(); + } + + if (worker_stop_socks[0] == SOCK_ERR) { + zend_throw_exception(spl_ce_RuntimeException, + "the background worker stop socket is not available", + 0); + RETURN_THROWS(); + } + + /* One stream per run: the same resource is returned until the script + * closes it, so fetching the handle in a loop does not grow the resource + * list of a run that never ends. The stream does not own the socket (see + * frankenphp_worker_handle_ops), so closing it never affects a later one + * and the EOF of a drain reaches all. */ + if (worker_handle_res != NULL) { + if (worker_handle_res->type == php_file_le_stream()) { + GC_ADDREF(worker_handle_res); + RETURN_RES(worker_handle_res); + } + /* closed by the script: drop the cache's ref */ + zend_list_delete(worker_handle_res); + worker_handle_res = NULL; + } + + php_stream *stream = + php_stream_sock_open_from_socket(worker_stop_socks[0], NULL); + if (stream == NULL) { + zend_throw_exception(spl_ce_RuntimeException, + "failed to create a stream over the stop socket", 0); + RETURN_THROWS(); + } + + /* a blocking read is a valid way to park: wait without the + * default_socket_timeout wake-ups */ + ((php_netstream_data_t *)stream->abstract)->timeout.tv_sec = -1; + + /* report the worker ready on its first wait on the stream */ + stream->ops = &frankenphp_worker_handle_ops; + + php_stream_to_zval(stream, return_value); + worker_handle_res = Z_RES_P(return_value); + GC_ADDREF(worker_handle_res); +} + /* {{{ thread-safe opcache reset */ PHP_FUNCTION(frankenphp_opcache_reset) { go_schedule_opcache_reset(frankenphp_thread_index()); @@ -1075,6 +1324,12 @@ static const zend_function_entry frankenphp_test_hook_functions[] = { #endif PHP_MINIT_FUNCTION(frankenphp) { + frankenphp_worker_handle_ops = php_stream_socket_ops; + frankenphp_worker_handle_ops.label = "FrankenPHP worker handle"; + frankenphp_worker_handle_ops.read = frankenphp_worker_handle_read; + frankenphp_worker_handle_ops.cast = frankenphp_worker_handle_cast; + frankenphp_worker_handle_ops.close = frankenphp_worker_handle_close; + register_frankenphp_symbols(module_number); #ifndef PHP_WIN32 /* MINIT runs once per ZTS thread — guard the atfork registration */ @@ -1527,6 +1782,16 @@ static void *php_thread(void *arg) { frankenphp_override_opcache_reset(); #endif + /* A background worker runs for the lifetime of its thread, so it has + * no execution limit, like the CLI SAPI. Disarming the timer is not + * enough: php_execute_script() re-arms it from the ini right before + * running the script whenever max_input_time is set, so change the + * setting itself. Request shutdown restores it, and the next run + * applies it again. */ + if (is_background_worker) { + frankenphp_disable_execution_timeout(); + } + zend_file_handle file_handle; zend_stream_init_filename(&file_handle, scriptName); @@ -1598,6 +1863,15 @@ static void *php_thread(void *arg) { } zend_end_try(); + /* The stop socket of a background worker is plain thread-local state that + * frankenphp_update_local_thread_context() only releases on recycle: close + * it here too so it does not outlive the thread on shutdown, reboot or an + * unhealthy exit. The Go side's end is closed by the Go side. */ + if (is_background_worker) { + is_background_worker = false; + frankenphp_worker_close_stop_socks(); + } + /* Must precede ts_free_thread: that frees the TSRM storage backing * the slot's &EG() pointers. Clearing first means any concurrent * force-kill either ran before us or sees a zero slot. */ diff --git a/frankenphp.go b/frankenphp.go index 3f7bbdf582..63c1242799 100644 --- a/frankenphp.go +++ b/frankenphp.go @@ -172,26 +172,42 @@ func checkPHPConfig(config PHPConfig) error { return nil } -func calculateMaxThreads(opt *opt) (numWorkers int, _ error) { +// calculateMaxThreads resolves num_threads and max_threads against the HTTP +// workers and returns their thread count, plus the threads of the background +// workers, which take no part in that budget: they come on top of it +func calculateMaxThreads(opt *opt) (numWorkers, backgroundThreads int, _ error) { maxProcs := runtime.GOMAXPROCS(0) * 2 maxThreadsFromWorkers := 0 for i, w := range opt.workers { + if w.isBackgroundWorker { + if w.num < 1 { + name := w.name + if name == "" { + name = w.fileName + } + + return 0, 0, fmt.Errorf("background worker %q must declare num >= 1", name) + } + backgroundThreads += w.num + + continue + } + if w.num <= 0 { // https://github.com/php/frankenphp/issues/126 opt.workers[i].num = maxProcs } - metrics.TotalWorkers(w.name, w.num) numWorkers += opt.workers[i].num if w.maxThreads > 0 { if w.maxThreads < w.num { - return 0, fmt.Errorf("worker max_threads (%d) must be greater or equal to worker num (%d) (%q)", w.maxThreads, w.num, w.fileName) + return 0, 0, fmt.Errorf("worker max_threads (%d) must be greater or equal to worker num (%d) (%q)", w.maxThreads, w.num, w.fileName) } if w.maxThreads > opt.maxThreads && opt.maxThreads > 0 { - return 0, fmt.Errorf("worker max_threads (%d) cannot be greater than total max_threads (%d) (%q)", w.maxThreads, opt.maxThreads, w.fileName) + return 0, 0, fmt.Errorf("worker max_threads (%d) cannot be greater than total max_threads (%d) (%q)", w.maxThreads, opt.maxThreads, w.fileName) } maxThreadsFromWorkers += w.maxThreads - w.num @@ -215,19 +231,19 @@ func calculateMaxThreads(opt *opt) (numWorkers int, _ error) { if numThreadsIsSet && !maxThreadsIsSet { opt.maxThreads = opt.numThreads if opt.numThreads <= numWorkers { - return 0, fmt.Errorf("num_threads (%d) must be greater than the number of worker threads (%d)", opt.numThreads, numWorkers) + return 0, 0, fmt.Errorf("num_threads (%d) must be greater than the number of worker threads (%d)", opt.numThreads, numWorkers) } - return numWorkers, nil + return numWorkers, backgroundThreads, nil } if maxThreadsIsSet && !numThreadsIsSet { opt.numThreads = numWorkers + 1 if !maxThreadsIsAuto && opt.numThreads > opt.maxThreads { - return 0, fmt.Errorf("max_threads (%d) must be greater than the number of worker threads (%d)", opt.maxThreads, numWorkers) + return 0, 0, fmt.Errorf("max_threads (%d) must be greater than the number of worker threads (%d)", opt.maxThreads, numWorkers) } - return numWorkers, nil + return numWorkers, backgroundThreads, nil } if !numThreadsIsSet { @@ -239,19 +255,19 @@ func calculateMaxThreads(opt *opt) (numWorkers int, _ error) { } opt.maxThreads = opt.numThreads - return numWorkers, nil + return numWorkers, backgroundThreads, nil } // both num_threads and max_threads are set if opt.numThreads <= numWorkers { - return 0, fmt.Errorf("num_threads (%d) must be greater than the number of worker threads (%d)", opt.numThreads, numWorkers) + return 0, 0, fmt.Errorf("num_threads (%d) must be greater than the number of worker threads (%d)", opt.numThreads, numWorkers) } if !maxThreadsIsAuto && opt.maxThreads < opt.numThreads { - return 0, fmt.Errorf("max_threads (%d) must be greater than or equal to num_threads (%d)", opt.maxThreads, opt.numThreads) + return 0, 0, fmt.Errorf("max_threads (%d) must be greater than or equal to num_threads (%d)", opt.maxThreads, opt.numThreads) } - return numWorkers, nil + return numWorkers, backgroundThreads, nil } // Init starts the PHP runtime and the configured workers. @@ -300,13 +316,15 @@ func Init(options ...Option) error { registerServers(opt.servers) - workerThreadCount, err := calculateMaxThreads(opt) + workerThreadCount, backgroundThreads, err := calculateMaxThreads(opt) if err != nil { shutdown() return err } - metrics.TotalThreads(opt.numThreads) + // background workers run on threads of their own, on top of the budget + // num_threads and max_threads describe for HTTP traffic + metrics.TotalThreads(opt.numThreads + backgroundThreads) config := Config() @@ -324,13 +342,23 @@ func Init(options ...Option) error { } } else { opt.numThreads = 1 + if workerThreadCount > 1 || backgroundThreads > 0 { + shutdown() + return fmt.Errorf("%d worker threads are declared, but this PHP build is not ZTS and runs a single thread", workerThreadCount+backgroundThreads) + } if globalLogger.Enabled(globalCtx, slog.LevelWarn) { globalLogger.LogAttrs(globalCtx, slog.LevelWarn, `ZTS is not enabled, only 1 thread will be available, recompile PHP using the "--enable-zts" configuration option or performance will be degraded`) } } - mainThread, err := initPHPThreads(opt.numThreads, opt.maxThreads, opt.phpIni) + maxThreads := opt.maxThreads + if maxThreads > 0 { + // in auto mode (maxThreads < 0), the resolved value is floored to the + // thread count, background threads included + maxThreads += backgroundThreads + } + mainThread, err := initPHPThreads(opt.numThreads+backgroundThreads, maxThreads, opt.phpIni) if err != nil { shutdown() return err @@ -484,7 +512,7 @@ func go_apache_request_headers(threadIndex C.uintptr_t) (*C.go_string, C.size_t) // worker mode, not handling a request if fc.logger.Enabled(fc.ctx, slog.LevelDebug) { - fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "apache_request_headers() called in non-HTTP context", slog.String("worker", fc.worker.name)) + fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "apache_request_headers() called in non-HTTP context", slog.String("worker", fc.worker.qualifiedName)) } return nil, 0 @@ -824,8 +852,6 @@ func resetGlobals() { globalCtx = context.Background() globalLogger = slog.Default() workers = nil - workersByName = nil - globalWorkersByPath = nil servers = nil watcherIsEnabled = false maxIdleTime = defaultMaxIdleTime diff --git a/frankenphp.h b/frankenphp.h index 99ac0ab7ec..e5612ee714 100644 --- a/frankenphp.h +++ b/frankenphp.h @@ -201,6 +201,10 @@ size_t frankenphp_get_thread_memory_usage(uintptr_t thread_index); void frankenphp_force_kill_thread(force_kill_slot slot); void frankenphp_release_thread_for_kill(force_kill_slot slot); +/* Background worker primitives. */ +intptr_t frankenphp_set_background_worker_and_get_stop_sock(void); +void frankenphp_worker_close_stop_sock(intptr_t s); + void register_extensions(zend_module_entry **m, int len); #endif diff --git a/frankenphp.stub.php b/frankenphp.stub.php index d6c85aa05f..bf1587cc64 100644 --- a/frankenphp.stub.php +++ b/frankenphp.stub.php @@ -54,3 +54,16 @@ function mercure_publish(string|array $topics, string $data = '', bool $private * array $context Values of the array will be converted to the corresponding Go type (if supported by FrankenPHP) and added to the context of the structured logs using https://pkg.go.dev/log/slog#Attr */ function frankenphp_log(string $message, int $level = 0, array $context = []): void {} + +/** + * Returns a stop-signal stream for the current background worker. The + * stream reaches EOF when FrankenPHP drains the worker, so the script can + * park on stream_select() and exit its loop gracefully. Every call of a run + * returns the same stream, a fresh one over the same socket once the script + * closed it. The worker counts as ready, and its startup as successful, once + * it waits on the stream (stream_select() or a blocking read). Only callable + * from inside a background worker. + * + * @return resource + */ +function frankenphp_get_worker_handle() {} diff --git a/frankenphp_arginfo.h b/frankenphp_arginfo.h index 4f2707cbca..8223be08ba 100644 --- a/frankenphp_arginfo.h +++ b/frankenphp_arginfo.h @@ -1,5 +1,5 @@ -/* This is a generated file, edit the .stub.php file instead. - * Stub hash: 60f0d27c04f94d7b24c052e91ef294595a2bc421 */ +/* This is a generated file, edit frankenphp.stub.php instead. + * Stub hash: 2f36fc81e0981975adabf170febaaa863653817d */ ZEND_BEGIN_ARG_WITH_RETURN_TYPE_INFO_EX(arginfo_frankenphp_handle_request, 0, 1, _IS_BOOL, 0) ZEND_ARG_TYPE_INFO(0, callback, IS_CALLABLE, 0) @@ -41,6 +41,8 @@ ZEND_BEGIN_ARG_WITH_RETURN_TYPE_INFO_EX(arginfo_frankenphp_log, 0, 1, IS_VOID, 0 ZEND_ARG_TYPE_INFO_WITH_DEFAULT_VALUE(0, context, IS_ARRAY, 0, "[]") ZEND_END_ARG_INFO() +ZEND_BEGIN_ARG_INFO_EX(arginfo_frankenphp_get_worker_handle, 0, 0, 0) +ZEND_END_ARG_INFO() ZEND_FUNCTION(frankenphp_handle_request); ZEND_FUNCTION(headers_send); @@ -49,7 +51,7 @@ ZEND_FUNCTION(frankenphp_request_headers); ZEND_FUNCTION(frankenphp_response_headers); ZEND_FUNCTION(mercure_publish); ZEND_FUNCTION(frankenphp_log); - +ZEND_FUNCTION(frankenphp_get_worker_handle); static const zend_function_entry ext_functions[] = { ZEND_FE(frankenphp_handle_request, arginfo_frankenphp_handle_request) @@ -63,6 +65,7 @@ static const zend_function_entry ext_functions[] = { ZEND_FALIAS(apache_response_headers, frankenphp_response_headers, arginfo_apache_response_headers) ZEND_FE(mercure_publish, arginfo_mercure_publish) ZEND_FE(frankenphp_log, arginfo_frankenphp_log) + ZEND_FE(frankenphp_get_worker_handle, arginfo_frankenphp_get_worker_handle) ZEND_FE_END }; diff --git a/metrics.go b/metrics.go index fc25816506..d51a3726a8 100644 --- a/metrics.go +++ b/metrics.go @@ -11,7 +11,7 @@ import ( const ( StopReasonCrash = iota StopReasonRestart - StopReasonBootFailure // worker crashed before reaching frankenphp_handle_request + StopReasonBootFailure // worker exited before reaching its ready point: frankenphp_handle_request, or frankenphp_get_worker_handle for background workers ) type StopReason int @@ -144,7 +144,7 @@ func (m *PrometheusMetrics) StopWorker(name string, reason StopReason) { m.totalWorkers.WithLabelValues(name).Dec() - // only decrement readyWorkers if the worker actually reached frankenphp_handle_request + // only decrement readyWorkers if the worker actually reached its ready point if reason != StopReasonBootFailure { m.readyWorkers.WithLabelValues(name).Dec() } @@ -177,7 +177,7 @@ func (m *PrometheusMetrics) TotalWorkers(string, int) { m.readyWorkers = prometheus.NewGaugeVec(prometheus.GaugeOpts{ Namespace: ns, Name: "ready_workers", - Help: "Running workers that have successfully called frankenphp_handle_request at least once", + Help: "Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers", }, basicLabels) m.mustRegister(m.readyWorkers) } diff --git a/metrics_test.go b/metrics_test.go index 846a926569..e5ca5e3c71 100644 --- a/metrics_test.go +++ b/metrics_test.go @@ -185,7 +185,7 @@ func TestPrometheusMetrics_TestStopReasonCrash(t *testing.T) { name: "Testing ReadyWorkers", c: m.readyWorkers, metadata: ` - # HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge `, expect: ` diff --git a/options.go b/options.go index e1eaeb7b55..930eb5d182 100644 --- a/options.go +++ b/options.go @@ -57,6 +57,7 @@ type workerOpt struct { onServerStartup func() onServerShutdown func() server *Server + isBackgroundWorker bool } // WithContext sets the main context to use. @@ -239,6 +240,20 @@ func WithWorkerServerScope(s *Server) WorkerOption { } } +// EXPERIMENTAL: WithWorkerBackground marks this worker as a background +// (non-HTTP) worker. Background workers run outside the request cycle: +// they share the PHP runtime with HTTP threads but never receive HTTP +// requests. The script can park on the stream returned by +// frankenphp_get_worker_handle(), which reaches EOF when FrankenPHP +// drains the worker, to exit gracefully on shutdown or restart. +func WithWorkerBackground() WorkerOption { + return func(w *workerOpt) error { + w.isBackgroundWorker = true + + return nil + } +} + // WithWorkerMaxFailures sets the maximum number of consecutive failures before panicking func WithWorkerMaxFailures(maxFailures int) WorkerOption { return func(w *workerOpt) error { diff --git a/phpmainthread.go b/phpmainthread.go index 5e19c0fc75..0878762341 100644 --- a/phpmainthread.go +++ b/phpmainthread.go @@ -161,6 +161,10 @@ func (mainThread *phpMainThread) rebootAllThreads() bool { for _, thread := range rebootingThreads { rebootWg.Go(func() { + // wake up handlers parked in a blocking C call (background + // workers' stream_select on the stop socket) so they can yield + // for the reboot without waiting for the force-kill below + thread.handler.drain() close(thread.drainChan) if thread.state.WaitForStateWithTimeout(rebootGracePeriod, state.YieldingForReboot) { return diff --git a/phpmainthread_test.go b/phpmainthread_test.go index 3ae65e68b9..08212466d9 100644 --- a/phpmainthread_test.go +++ b/phpmainthread_test.go @@ -252,28 +252,24 @@ func TestFinishBootingAWorkerScript(t *testing.T) { func TestReturnAnErrorIf2WorkersHaveTheSameFileName(t *testing.T) { resetGlobals() - workers = []*worker{} - workersByName = map[string]*worker{} - globalWorkersByPath = map[string]*worker{} - w, err1 := newWorker(workerOpt{fileName: testDataPath + "/index.php"}) - assert.NoError(t, err1) - workers = append(workers, w) - workersByName[w.name] = w - globalWorkersByPath[w.fileName] = w - _, err2 := newWorker(workerOpt{fileName: testDataPath + "/index.php"}) - assert.Error(t, err2, "two workers cannot have the same filename") + fallbackServer.resetWorkers() + w1, err := newWorker(workerOpt{fileName: testDataPath + "/index.php"}) + assert.NoError(t, err) + assert.NoError(t, fallbackServer.addWorker(w1)) + w2, err := newWorker(workerOpt{fileName: testDataPath + "/index.php", name: "other"}) + assert.NoError(t, err) + assert.ErrorContains(t, fallbackServer.addWorker(w2), "two global workers cannot have the same filename") } func TestReturnAnErrorIf2ModuleWorkersHaveTheSameName(t *testing.T) { resetGlobals() - workers = []*worker{} - workersByName = map[string]*worker{} - w, err1 := newWorker(workerOpt{fileName: testDataPath + "/index.php", name: "workername"}) - assert.NoError(t, err1) - workers = append(workers, w) - workersByName[w.name] = w - _, err2 := newWorker(workerOpt{fileName: testDataPath + "/hello.php", name: "workername"}) - assert.Error(t, err2, "two workers cannot have the same name") + fallbackServer.resetWorkers() + w1, err := newWorker(workerOpt{fileName: testDataPath + "/index.php", name: "workername"}) + assert.NoError(t, err) + assert.NoError(t, fallbackServer.addWorker(w1)) + w2, err := newWorker(workerOpt{fileName: testDataPath + "/hello.php", name: "workername"}) + assert.NoError(t, err) + assert.ErrorContains(t, fallbackServer.addWorker(w2), "two global workers cannot have the same name") } func getDummyWorker(t *testing.T, fileName string) *worker { @@ -315,9 +311,9 @@ func allPossibleTransitions(worker1Path string, worker2Path string) []func(*phpT thread.boot() } }, - func(thread *phpThread) { convertToWorkerThread(thread, globalWorkersByPath[worker1Path]) }, + func(thread *phpThread) { convertToWorkerThread(thread, fallbackServer.workersByPath[worker1Path]) }, convertToInactiveThread, - func(thread *phpThread) { convertToWorkerThread(thread, globalWorkersByPath[worker2Path]) }, + func(thread *phpThread) { convertToWorkerThread(thread, fallbackServer.workersByPath[worker2Path]) }, convertToInactiveThread, } } @@ -371,7 +367,7 @@ func TestCorrectThreadCalculation(t *testing.T) { func testThreadCalculation(t *testing.T, expectedNumThreads int, expectedMaxThreads int, o *opt) { t.Helper() - _, err := calculateMaxThreads(o) + _, _, err := calculateMaxThreads(o) assert.NoError(t, err, "no error should be returned") assert.Equal(t, expectedNumThreads, o.numThreads, "num_threads must be correct") assert.Equal(t, expectedMaxThreads, o.maxThreads, "max_threads must be correct") @@ -380,7 +376,7 @@ func testThreadCalculation(t *testing.T, expectedNumThreads int, expectedMaxThre func testThreadCalculationError(t *testing.T, o *opt) { t.Helper() - _, err := calculateMaxThreads(o) + _, _, err := calculateMaxThreads(o) assert.Error(t, err, "configuration must error") } @@ -389,7 +385,8 @@ func TestContextAndLoggerMustNotBeNil(t *testing.T) { assert.NotNil(t, log, "logger is defined if all threads are inactive") assert.NotNil(t, ctx, "context is defined if all threads are inactive") - fc := newContextFromMessage(nil, nil, nil, &worker{}) + // a worker always belongs to a server, see newWorker() + fc := newContextFromMessage(nil, nil, nil, &worker{server: fallbackServer}) assert.NotNil(t, fc.logger, "logger is defined for message context") assert.NotNil(t, fc.ctx, "context is defined for message context") @@ -398,7 +395,7 @@ func TestContextAndLoggerMustNotBeNil(t *testing.T) { assert.NotNil(t, fc.logger, "logger is defined for request context") assert.NotNil(t, fc.ctx, "context is defined for request context") - fc, _ = newWorkerDummyContext(&worker{}) + fc, _ = newWorkerDummyContext(&worker{server: fallbackServer}) assert.NotNil(t, fc.logger, "logger is defined for worker dummy context") assert.NotNil(t, fc.ctx, "context is defined for worker dummy context") } diff --git a/phpthread.go b/phpthread.go index 39ee24b5d4..4165da482a 100644 --- a/phpthread.go +++ b/phpthread.go @@ -39,11 +39,10 @@ type threadHandler interface { beforeScriptExecution() string afterScriptExecution(exitStatus int) frankenPHPContext() *frankenPHPContext - // drain is a hook called by drainWorkerThreads right before drainChan is - // closed. Handlers that need to wake up a thread parked in a blocking C - // call (e.g. by closing a stop pipe) plug their signal in here. All - // current handlers are no-ops; this is the seam later handler types use - // without having to modify drainWorkerThreads. + // drain is a hook called right before drainChan is closed on shutdown + // and reboot. Handlers that need to wake up a thread parked in a + // blocking C call (background workers' stream_select on the stop socket) + // plug their signal in here; the other handlers are no-ops. drain() } @@ -119,6 +118,9 @@ func (thread *phpThread) shutdown() { return } + // wake up handlers parked in a blocking C call (background workers' + // stream_select on the stop socket); no-op for the other handlers + thread.handler.drain() close(thread.drainChan) // Arm force-kill after the grace period to wake any thread stuck in @@ -157,6 +159,9 @@ func (thread *phpThread) setHandler(handler threadHandler) { return } + // wake up a handler parked in a blocking C call (background workers' + // stream_select on the stop socket) so it can yield for the transition + thread.handler.drain() close(thread.drainChan) thread.state.WaitFor(state.TransitionInProgress) diff --git a/requestoptions.go b/requestoptions.go index 962727562f..00c5892f05 100644 --- a/requestoptions.go +++ b/requestoptions.go @@ -2,6 +2,7 @@ package frankenphp import ( "errors" + "fmt" "log/slog" "net/http" "path/filepath" @@ -206,12 +207,22 @@ func WithRequestBodyTimeout(timeout time.Duration) RequestOption { } // WithWorkerName sets the worker that should handle the request +// the name is resolved among the workers of the request's server first, then among global workers func WithWorkerName(name string) RequestOption { return func(o *frankenPHPContext) error { - if name != "" { - o.worker = workersByName[name] + if name == "" { + return nil } + w := o.server.workersByName[name] + if w == nil { + w = fallbackServer.workersByName[name] + } + if w != nil && w.isBackgroundWorker { + return fmt.Errorf("background worker %q cannot handle requests", name) + } + o.worker = w + return nil } } diff --git a/scaling.go b/scaling.go index dd21a7e37c..c26465efbd 100644 --- a/scaling.go +++ b/scaling.go @@ -96,7 +96,7 @@ func scaleWorkerThread(worker *worker, done chan struct{}, mstate *state.ThreadS thread, err := addWorkerThread(worker) if err != nil { if globalLogger.Enabled(globalCtx, slog.LevelWarn) { - globalLogger.LogAttrs(globalCtx, slog.LevelWarn, "could not increase max_threads, consider raising this limit", slog.String("worker", worker.name), slog.Any("error", err)) + globalLogger.LogAttrs(globalCtx, slog.LevelWarn, "could not increase max_threads, consider raising this limit", slog.String("worker", worker.qualifiedName), slog.Any("error", err)) } return @@ -105,7 +105,7 @@ func scaleWorkerThread(worker *worker, done chan struct{}, mstate *state.ThreadS autoScaledThreads = append(autoScaledThreads, thread) if globalLogger.Enabled(globalCtx, slog.LevelInfo) { - globalLogger.LogAttrs(globalCtx, slog.LevelInfo, "upscaling worker thread", slog.String("worker", worker.name), slog.Int("thread", thread.threadIndex), slog.Int("num_threads", len(autoScaledThreads))) + globalLogger.LogAttrs(globalCtx, slog.LevelInfo, "upscaling worker thread", slog.String("worker", worker.qualifiedName), slog.Int("thread", thread.threadIndex), slog.Int("num_threads", len(autoScaledThreads))) } } @@ -177,7 +177,7 @@ func startUpscalingThreads(maxScaledThreads int, scale chan *frankenPHPContext, // check for max worker threads here again in case requests overflowed while waiting if fc.worker.isAtThreadLimit() { if globalLogger.Enabled(globalCtx, slog.LevelInfo) { - globalLogger.LogAttrs(globalCtx, slog.LevelInfo, "cannot scale worker thread, max threads reached for worker", slog.String("worker", fc.worker.name)) + globalLogger.LogAttrs(globalCtx, slog.LevelInfo, "cannot scale worker thread, max threads reached for worker", slog.String("worker", fc.worker.qualifiedName)) } continue diff --git a/scaling_test.go b/scaling_test.go index d2784a8eeb..a702ea8241 100644 --- a/scaling_test.go +++ b/scaling_test.go @@ -48,7 +48,7 @@ func TestScaleAWorkerThreadUpAndDown(t *testing.T) { autoScaledThread := phpThreads[2] // scale up - scaleWorkerThread(globalWorkersByPath[workerPath], mainThread.done, mainThread.state) + scaleWorkerThread(fallbackServer.workersByPath[workerPath], mainThread.done, mainThread.state) assert.Equal(t, state.Ready, autoScaledThread.state.Get()) // on down-scale, the thread will be marked as inactive diff --git a/server.go b/server.go index 8274f23c54..332eeafa5f 100644 --- a/server.go +++ b/server.go @@ -20,7 +20,7 @@ type Server struct { root string splitPath []string env PreparedEnv - workers []*worker + workersByName map[string]*worker workersByPath map[string]*worker workersWithRequestMatcher []*worker @@ -35,13 +35,15 @@ var ( fallbackServer = newFallbackServer() ) +// newFallbackServer creates the server of requests and workers that are not +// scoped to one, so a lookup is always a lookup in a server func newFallbackServer() *Server { s := &Server{ - idx: -1, - workersByPath: make(map[string]*worker), - env: make(map[string]string), - logger: globalLogger, + idx: -1, + env: make(map[string]string), + logger: globalLogger, } + s.resetWorkers() return s } @@ -54,12 +56,36 @@ func registerServers(newServers []*Server) { fallbackServer.logger = globalLogger fallbackServer.resetWorkers() + // several servers may resolve to the same name (e.g. the same host), but + // the name qualifies worker names in metrics and logs, so it must be + // unique: the first server keeps a name, the next ones get a numeric + // suffix that never takes a name another server configured + configured := make(map[string]struct{}, len(servers)) + for _, s := range servers { + if s.configuredName != "" { + configured[s.configuredName] = struct{}{} + } + } + + taken := make(map[string]struct{}, len(servers)) for i, s := range servers { s.idx = i - s.name = s.configuredName - if s.name == "" { - s.name = "server_" + strconv.Itoa(i) + name := s.configuredName + if name == "" { + name = "server_" + strconv.Itoa(i) } + + for base, n := name, 1; ; n++ { + _, isTaken := taken[name] + _, isConfigured := configured[name] + if !isTaken && (!isConfigured || name == s.configuredName) { + break + } + name = base + "_" + strconv.Itoa(n) + } + taken[name] = struct{}{} + + s.name = name s.resetWorkers() } } @@ -83,7 +109,7 @@ func unregisterServers() { // resetWorkers drops the workers of a previous run; initWorkers() adds them back func (s *Server) resetWorkers() { - s.workers = nil + s.workersByName = make(map[string]*worker) s.workersByPath = make(map[string]*worker) s.workersWithRequestMatcher = nil } @@ -99,9 +125,9 @@ func NewServer(root string, options ...ServerOption) (*Server, error) { } s := &Server{ - root: root, - workersByPath: make(map[string]*worker), + root: root, } + s.resetWorkers() for _, option := range options { if err := option(s); err != nil { @@ -125,20 +151,40 @@ func NewServer(root string, options ...ServerOption) (*Server, error) { } // Name returns the human-readable name of the server. -// It is empty until registration if none was passed to NewServer(). +// It is empty until registration if none was passed to NewServer(), and gets +// a numeric suffix if another registered server has the same name. func (s *Server) Name() string { return s.name } +// addWorker registers a worker scoped to this server +// scope names the worker set in errors: a server, or the global workers +func (s *Server) scope() string { + if s == fallbackServer { + return "two global workers" + } + + return "two workers in a server" +} + func (s *Server) addWorker(w *worker) error { - s.workers = append(s.workers, w) + if s.workersByName[w.name] != nil { + return fmt.Errorf("%s cannot have the same name: %q", s.scope(), w.name) + } + s.workersByName[w.name] = w + + // background workers never serve requests, so they are not matched at all + if w.isBackgroundWorker { + return nil + } + if w.matchRequest != nil { s.workersWithRequestMatcher = append(s.workersWithRequestMatcher, w) return nil } - if _, exists := s.workersByPath[w.fileName]; exists { - return fmt.Errorf("two workers in a server cannot have the same filename: %q", w.fileName) + if s.workersByPath[w.fileName] != nil { + return fmt.Errorf("%s cannot have the same filename: %q", s.scope(), w.fileName) } s.workersByPath[w.fileName] = w diff --git a/server_test.go b/server_test.go index f297db7c29..4bd078040c 100644 --- a/server_test.go +++ b/server_test.go @@ -59,12 +59,59 @@ func TestServer(t *testing.T) { t.Run("name", func(t *testing.T) { named, _ := frankenphp.NewServer(testDataDir, frankenphp.WithServerName("api")) unnamed, _ := frankenphp.NewServer(testDataDir) + alsoNamed, _ := frankenphp.NewServer(testDataDir, frankenphp.WithServerName("api")) + explicit, _ := frankenphp.NewServer(testDataDir, frankenphp.WithServerName("api_1")) - initServers(t, frankenphp.WithServer(named), frankenphp.WithServer(unnamed)) + initServers(t, frankenphp.WithServer(named), frankenphp.WithServer(unnamed), frankenphp.WithServer(alsoNamed), frankenphp.WithServer(explicit)) assert.Equal(t, "api", named.Name()) // an empty name defaults to the server index at registration assert.Equal(t, "server_1", unnamed.Name()) + // names qualify worker names in metrics, so they are made unique, and + // a generated suffix never takes a name another server configured + assert.Equal(t, "api_2", alsoNamed.Name()) + assert.Equal(t, "api_1", explicit.Name()) + }) + + t.Run("same_worker_name_in_two_servers", func(t *testing.T) { + server1, _ := frankenphp.NewServer(testDataDir, frankenphp.WithServerName("one")) + server2, _ := frankenphp.NewServer(testDataDir, frankenphp.WithServerName("two")) + initServers( + t, + frankenphp.WithServer(server1), + frankenphp.WithServer(server2), + frankenphp.WithWorkers("counter", testDataDir+"worker-with-counter.php", 1, frankenphp.WithWorkerServerScope(server1)), + frankenphp.WithWorkers("counter", testDataDir+"worker-with-counter.php", 1, frankenphp.WithWorkerServerScope(server2)), + ) + + // WithWorkerName resolves the name within the request's server + byName := func(server *frankenphp.Server) string { + t.Helper() + w := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodGet, "http://example.com/index.php", nil) + require.NoError(t, server.ServeHTTP(w, req, frankenphp.WithWorkerName("counter"))) + body, err := io.ReadAll(w.Result().Body) + require.NoError(t, err) + + return string(body) + } + + assert.Equal(t, "requests:1", byName(server1)) + assert.Equal(t, "requests:1", byName(server2), "server 2 must get its own worker, not server 1's") + assert.Equal(t, "requests:2", byName(server1)) + }) + + t.Run("error_on_duplicate_worker_names", func(t *testing.T) { + t.Cleanup(frankenphp.Shutdown) + + server, _ := frankenphp.NewServer(testDataDir) + err := frankenphp.Init( + frankenphp.WithServer(server), + frankenphp.WithWorkers("same", testDataDir+"worker-with-counter.php", 1, frankenphp.WithWorkerServerScope(server)), + frankenphp.WithWorkers("same", testDataDir+"index.php", 1, frankenphp.WithWorkerServerScope(server)), + ) + + assert.ErrorContains(t, err, "two workers in a server cannot have the same name") }) t.Run("root", func(t *testing.T) { diff --git a/testdata/_executor.php b/testdata/_executor.php index 61a5319f11..31b87c79cb 100644 --- a/testdata/_executor.php +++ b/testdata/_executor.php @@ -1,7 +1,7 @@ $_SERVER['FRANKENPHP_WORKER'] ?? null, + 'background' => isset($_SERVER['FRANKENPHP_WORKER_BACKGROUND']) ? 'set' : 'unset', +], true)); +$stream = frankenphp_get_worker_handle(); +$read = [$stream]; +$write = null; +$except = null; +stream_select($read, $write, $except, null); diff --git a/testdata/bgworker/named.php b/testdata/bgworker/named.php new file mode 100644 index 0000000000..4b12f88961 --- /dev/null +++ b/testdata/bgworker/named.php @@ -0,0 +1,19 @@ + 1 threads share the name; each touches a file of +// its own under BG_SENTINEL_DIR, then parks on its own handle. +set_time_limit(0); +@touch($_SERVER['BG_SENTINEL_DIR'] . DIRECTORY_SEPARATOR . bin2hex(random_bytes(8))); +$stream = frankenphp_get_worker_handle(); +$read = [$stream]; +$write = null; +$except = null; +stream_select($read, $write, $except, null); diff --git a/testdata/bgworker/read.php b/testdata/bgworker/read.php new file mode 100644 index 0000000000..c57e38b341 --- /dev/null +++ b/testdata/bgworker/read.php @@ -0,0 +1,11 @@ +getMessage(); +} diff --git a/testdata/symlinks/test/index.php b/testdata/symlinks/test/index.php index 15aa1a9cf1..9037dbe762 100644 --- a/testdata/symlinks/test/index.php +++ b/testdata/symlinks/test/index.php @@ -1,7 +1,7 @@ = 0 { + C.frankenphp_worker_close_stop_sock(C.intptr_t(s)) + } +} + +func (handler *backgroundWorkerThread) beforeScriptExecution() string { + return handler.workerLifecycle.beforeScriptExecution(handler) +} + +// startScript keeps trying to start the script: unlike an HTTP worker, whose +// setup cannot fail, a background worker needs a socket pair per run +func (handler *backgroundWorkerThread) startScript() string { + for { + err := handler.setupScript() + if err == nil { + return handler.worker.fileName + } + + if globalLogger.Enabled(globalCtx, slog.LevelError) { + globalLogger.LogAttrs(globalCtx, slog.LevelError, "failed to start background worker", slog.String("worker", handler.worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex), slog.Any("error", err)) + } + + // fail fast during startup so Init() surfaces the error to the + // operator; past startup, back off and retry like a crash + if startupFailChan != nil { + startupFailChan <- err + handler.thread.state.Set(state.ShuttingDown) + + return handler.beforeScriptExecution() + } + + handler.backoff() + if !handler.state.Is(state.Ready) && !handler.state.Is(state.TransitionComplete) { + // drained during the backoff (shutdown, reboot, transition) + return handler.beforeScriptExecution() + } + } +} + +// a background worker counts nothing per run of the thread +func (handler *backgroundWorkerThread) resetForReboot() {} + +// setupScript marks the thread as a background worker on the C side and +// takes ownership of the Go side's end of its stop socket pair. +func (handler *backgroundWorkerThread) setupScript() error { + s := int64(C.frankenphp_set_background_worker_and_get_stop_sock()) + if s < 0 { + return fmt.Errorf("failed to create the stop socket pair of background worker %q", handler.worker.qualifiedName) + } + handler.stopSock.Store(s) + + switch handler.state.Get() { + case state.ShuttingDown, state.Rebooting, state.ForceRebooting, state.TransitionRequested: + // a concurrent drain may have run before the socket was published; + // close it now so the script observes EOF immediately + handler.drain() + } + + fc, err := newWorkerDummyContext(handler.worker) + if err != nil { + handler.drain() + return err + } + handler.dummyFrankenPHPContext = fc + + handler.isBootingScript = true + metrics.StartWorker(handler.worker.qualifiedName) + handler.bootTimer = time.AfterFunc(backgroundBootWarnDelay, func() { + if globalLogger.Enabled(globalCtx, slog.LevelWarn) { + globalLogger.LogAttrs(globalCtx, slog.LevelWarn, "background worker has not waited on its handle yet, Init() and Shutdown() wait for it, see frankenphp_get_worker_handle()", slog.String("worker", handler.worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex)) + } + }) + + if fc.logger.Enabled(fc.ctx, slog.LevelDebug) { + fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "starting background worker", slog.String("worker", handler.worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex)) + } + + // the thread stays in TransitionComplete until the script waits on its + // handle, see go_frankenphp_background_worker_ready + + return nil +} + +func (handler *backgroundWorkerThread) afterScriptExecution(exitStatus int) { + // the Go side's end of the stop socket pair belongs to this thread; + // release it on every exit path so the next run gets a fresh pair + // (drain() already took it when the exit was drain-triggered) + handler.drain() + worker := handler.worker + handler.dummyFrankenPHPContext = nil + + handler.stopBootTimer() + handler.state.MarkAsWaiting(false) + + // cooperative exit: the script waited on its handle and returned cleanly, + // re-run it, unless the thread is being drained (beforeScriptExecution + // checks the state) + if exitStatus == 0 && !handler.isBootingScript { + handler.crashCount = 0 + metrics.StopWorker(worker.qualifiedName, StopReasonRestart) + + if globalLogger.Enabled(globalCtx, slog.LevelDebug) { + globalLogger.LogAttrs(globalCtx, slog.LevelDebug, "restarting background worker", slog.String("worker", worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex)) + } + + return + } + + // crash after the ready point: restart without counting toward + // max_consecutive_failures, that cap is about a script that never boots. + // The wait still applies: unlike an HTTP worker, which can only crash + // after a request reached frankenphp_handle_request() and is therefore + // paced by traffic, a background worker reaches its ready point on its + // own and a script crashing right after it would spin + if !handler.isBootingScript { + metrics.StopWorker(worker.qualifiedName, StopReasonCrash) + + if globalLogger.Enabled(globalCtx, slog.LevelWarn) { + globalLogger.LogAttrs(globalCtx, slog.LevelWarn, "background worker crashed, restarting", slog.String("worker", worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex), slog.Int("exit_status", exitStatus), slog.Int("crashes", handler.crashCount)) + } + + time.Sleep(restartBackoff(handler.crashCount)) + handler.crashCount++ + + return + } + + // boot failure: the script exited before waiting on its handle, a clean + // exit included, which would otherwise respawn in a tight loop. + // StopReasonBootFailure skips the ready-gauge decrement, matching the + // ReadyWorker call that never happened + metrics.StopWorker(worker.qualifiedName, StopReasonBootFailure) + + // max_consecutive_failures only fails hard during startup, where it + // surfaces on startupFailChan so Init() returns the error to the + // operator. Past startup, a failing background worker keeps + // restarting with a louder log line: silently giving up would leave + // the server in a broken half-state with no clear way to recover. + pastCap := worker.maxConsecutiveFailures >= 0 && handler.failureCount >= worker.maxConsecutiveFailures + if pastCap && startupFailChan != nil && !watcherIsEnabled { + if exitStatus == 0 { + startupFailChan <- fmt.Errorf("background worker %s exits without waiting on its handle, see frankenphp_get_worker_handle()", worker.fileName) + } else { + startupFailChan <- fmt.Errorf("too many consecutive failures: background worker %s keeps crashing", worker.fileName) + } + handler.thread.state.Set(state.ShuttingDown) + return + } + + logLevel := slog.LevelWarn + logMsg := "background worker failed before waiting on its handle, restarting" + if exitStatus == 0 { + logMsg = "background worker exited without waiting on its handle, restarting" + } + if pastCap { + logLevel = slog.LevelError + logMsg = "background worker exceeded max_consecutive_failures, still restarting" + } + if globalLogger.Enabled(globalCtx, logLevel) { + globalLogger.LogAttrs(globalCtx, logLevel, logMsg, slog.String("worker", worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex), slog.Int("failures", handler.failureCount), slog.Int("exit_status", exitStatus)) + } + + handler.backoff() +} + +func (handler *backgroundWorkerThread) stopBootTimer() { + if handler.bootTimer != nil { + handler.bootTimer.Stop() + handler.bootTimer = nil + } +} + +//export go_frankenphp_background_worker_ready +func go_frankenphp_background_worker_ready(threadIndex C.uintptr_t) { + // called on the PHP thread on the first wait on the handle; the handler + // is a backgroundWorkerThread because frankenphp_get_worker_handle() + // throws on every other thread kind + if handler, ok := phpThreads[threadIndex].handler.(*backgroundWorkerThread); ok && handler.isBootingScript { + handler.isBootingScript = false + // the boot succeeded, only consecutive boot failures count + handler.failureCount = 0 + handler.stopBootTimer() + metrics.ReadyWorker(handler.worker.qualifiedName) + // parked from now on as far as the threads state endpoint is concerned + handler.state.MarkAsWaiting(true) + + // like an HTTP worker reaching frankenphp_handle_request(), the thread + // is ready only now: initWorkers() waits for this state, so a script + // that fails before waiting on its handle still fails Init() + if handler.state.Is(state.TransitionComplete) { + handler.state.Set(state.Ready) + } + } +} + +// backoff waits before the next run of a crashed script, see restartBackoff +func (handler *backgroundWorkerThread) backoff() { + time.Sleep(restartBackoff(handler.failureCount)) + handler.failureCount++ +} diff --git a/threadworker.go b/threadworker.go index 8a308b4d87..233a58a65a 100644 --- a/threadworker.go +++ b/threadworker.go @@ -15,9 +15,8 @@ import ( // executes the PHP worker script in a loop // implements the threadHandler interface type workerThread struct { - state *state.ThreadState - thread *phpThread - worker *worker + workerLifecycle + dummyFrankenPHPContext *frankenPHPContext workerFrankenPHPContext *frankenPHPContext isBootingScript bool // true if the worker has not reached frankenphp_handle_request yet @@ -26,49 +25,23 @@ type workerThread struct { } func convertToWorkerThread(thread *phpThread, worker *worker) { - thread.setHandler(&workerThread{ - state: thread.state, - thread: thread, - worker: worker, - }) + thread.setHandler(&workerThread{workerLifecycle: newWorkerLifecycle(thread, worker)}) worker.attachThread(thread) } -// beforeScriptExecution returns the name of the script or an empty string on shutdown func (handler *workerThread) beforeScriptExecution() string { - switch handler.state.Get() { - case state.TransitionRequested: - if handler.worker.onThreadShutdown != nil { - handler.worker.onThreadShutdown(handler.thread.threadIndex) - } - handler.worker.detachThread(handler.thread) - return handler.thread.transitionToNewHandler() - case state.Ready, state.TransitionComplete: - handler.thread.updateContext(true) - if handler.worker.onThreadReady != nil { - handler.worker.onThreadReady(handler.thread.threadIndex) - } + return handler.workerLifecycle.beforeScriptExecution(handler) +} - setupWorkerScript(handler, handler.worker) +// startScript runs the worker script; it always has one to run +func (handler *workerThread) startScript() string { + setupWorkerScript(handler, handler.worker) - return handler.worker.fileName - case state.Rebooting, state.ForceRebooting: - return "" - case state.RebootReady: - handler.requestCount = 0 - handler.state.Set(state.Ready) - return handler.beforeScriptExecution() - case state.ShuttingDown: - if handler.worker.onThreadShutdown != nil { - handler.worker.onThreadShutdown(handler.thread.threadIndex) - } - handler.worker.detachThread(handler.thread) + return handler.worker.fileName +} - // signal to stop - return "" - default: - panic("unexpected state: " + handler.state.Name()) - } +func (handler *workerThread) resetForReboot() { + handler.requestCount = 0 } func (handler *workerThread) afterScriptExecution(exitStatus int) { @@ -90,7 +63,7 @@ func (handler *workerThread) name() string { func (handler *workerThread) drain() {} func setupWorkerScript(handler *workerThread, worker *worker) { - metrics.StartWorker(worker.name) + metrics.StartWorker(worker.qualifiedName) // Create a dummy request to set up the worker fc, err := newWorkerDummyContext(worker) @@ -103,7 +76,7 @@ func setupWorkerScript(handler *workerThread, worker *worker) { handler.requestCount = 0 if fc.logger.Enabled(fc.ctx, slog.LevelDebug) { - fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "starting", slog.String("worker", worker.name), slog.Int("thread", handler.thread.threadIndex)) + fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "starting", slog.String("worker", worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex)) } } @@ -122,10 +95,10 @@ func tearDownWorkerScript(handler *workerThread, exitStatus int) { // on exit status 0 we just run the worker script again if exitStatus == 0 && !handler.isBootingScript { - metrics.StopWorker(worker.name, StopReasonRestart) + metrics.StopWorker(worker.qualifiedName, StopReasonRestart) if globalLogger.Enabled(globalCtx, slog.LevelDebug) { - globalLogger.LogAttrs(globalCtx, slog.LevelDebug, "restarting", slog.String("worker", worker.name), slog.Int("thread", handler.thread.threadIndex), slog.Int("exit_status", exitStatus)) + globalLogger.LogAttrs(globalCtx, slog.LevelDebug, "restarting", slog.String("worker", worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex), slog.Int("exit_status", exitStatus)) } return @@ -133,9 +106,9 @@ func tearDownWorkerScript(handler *workerThread, exitStatus int) { // worker has thrown a fatal error or has not reached frankenphp_handle_request if handler.isBootingScript { - metrics.StopWorker(worker.name, StopReasonBootFailure) + metrics.StopWorker(worker.qualifiedName, StopReasonBootFailure) } else { - metrics.StopWorker(worker.name, StopReasonCrash) + metrics.StopWorker(worker.qualifiedName, StopReasonCrash) } if !handler.isBootingScript { @@ -143,7 +116,7 @@ func tearDownWorkerScript(handler *workerThread, exitStatus int) { // unlike a clean restart, this took down any in-flight request, so // surface it above debug level, with the exit status needed to triage it if globalLogger.Enabled(globalCtx, slog.LevelWarn) { - globalLogger.LogAttrs(globalCtx, slog.LevelWarn, "unexpected termination, restarting", slog.String("worker", worker.name), slog.Int("thread", handler.thread.threadIndex), slog.Int("exit_status", exitStatus)) + globalLogger.LogAttrs(globalCtx, slog.LevelWarn, "unexpected termination, restarting", slog.String("worker", worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex), slog.Int("exit_status", exitStatus)) } return @@ -158,23 +131,26 @@ func tearDownWorkerScript(handler *workerThread, exitStatus int) { if watcherIsEnabled { // worker script has probably failed due to script changes while watcher is enabled if globalLogger.Enabled(globalCtx, slog.LevelError) { - globalLogger.LogAttrs(globalCtx, slog.LevelError, "(watcher enabled) worker script has not reached frankenphp_handle_request()", slog.String("worker", worker.name), slog.Int("thread", handler.thread.threadIndex)) + globalLogger.LogAttrs(globalCtx, slog.LevelError, "(watcher enabled) worker script has not reached frankenphp_handle_request()", slog.String("worker", worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex)) } } else { // rare case where worker script has failed on a restart during normal operation // this can happen if startup success depends on external resources if globalLogger.Enabled(globalCtx, slog.LevelWarn) { - globalLogger.LogAttrs(globalCtx, slog.LevelWarn, "worker script has failed on restart", slog.String("worker", worker.name), slog.Int("thread", handler.thread.threadIndex), slog.Int("failures", handler.failureCount)) + globalLogger.LogAttrs(globalCtx, slog.LevelWarn, "worker script has failed on restart", slog.String("worker", worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex), slog.Int("failures", handler.failureCount)) } } - // wait a bit and try again (exponential backoff) - backoffDuration := time.Duration(handler.failureCount*handler.failureCount*100) * time.Millisecond - if backoffDuration > time.Second { - backoffDuration = time.Second - } + // wait a bit and try again + time.Sleep(restartBackoff(handler.failureCount)) handler.failureCount++ - time.Sleep(backoffDuration) +} + +// restartBackoff is the wait before a worker script is re-run after a +// failure: quadratic in the number of consecutive failures, capped at one +// second; shared by HTTP and background workers +func restartBackoff(failures int) time.Duration { + return min(time.Duration(failures*failures*100)*time.Millisecond, time.Second) } // waitForWorkerRequest is called during frankenphp_handle_request in the php worker script. @@ -183,7 +159,7 @@ func (handler *workerThread) waitForWorkerRequest() (bool, any) { handler.thread.Unpin() if globalLogger.Enabled(globalCtx, slog.LevelDebug) { - globalLogger.LogAttrs(globalCtx, slog.LevelDebug, "waiting for request", slog.String("worker", handler.worker.name), slog.Int("thread", handler.thread.threadIndex)) + globalLogger.LogAttrs(globalCtx, slog.LevelDebug, "waiting for request", slog.String("worker", handler.worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex)) } // Clear the first dummy request created to initialize the worker @@ -195,14 +171,14 @@ func (handler *workerThread) waitForWorkerRequest() (bool, any) { } // worker is truly ready only after reaching frankenphp_handle_request() - metrics.ReadyWorker(handler.worker.name) + metrics.ReadyWorker(handler.worker.qualifiedName) } // max_requests reached: signal reboot for full ZTS cleanup if maxRequestsPerThread > 0 && handler.requestCount >= maxRequestsPerThread { if globalLogger.Enabled(globalCtx, slog.LevelDebug) { globalLogger.LogAttrs(globalCtx, slog.LevelDebug, "max requests reached, restarting", - slog.String("worker", handler.worker.name), + slog.String("worker", handler.worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex), slog.Int("max_requests", maxRequestsPerThread), ) @@ -223,7 +199,7 @@ func (handler *workerThread) waitForWorkerRequest() (bool, any) { select { case <-handler.thread.drainChan: if globalLogger.Enabled(globalCtx, slog.LevelDebug) { - globalLogger.LogAttrs(globalCtx, slog.LevelDebug, "shutting down", slog.String("worker", handler.worker.name), slog.Int("thread", handler.thread.threadIndex)) + globalLogger.LogAttrs(globalCtx, slog.LevelDebug, "shutting down", slog.String("worker", handler.worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex)) } return false, nil @@ -239,9 +215,9 @@ func (handler *workerThread) waitForWorkerRequest() (bool, any) { if fc.logger.Enabled(fc.ctx, slog.LevelDebug) { if handler.workerFrankenPHPContext.request == nil { - fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "request handling started", slog.String("worker", handler.worker.name), slog.Int("thread", handler.thread.threadIndex)) + fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "request handling started", slog.String("worker", handler.worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex)) } else { - fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "request handling started", slog.String("worker", handler.worker.name), slog.Int("thread", handler.thread.threadIndex), slog.String("url", handler.workerFrankenPHPContext.request.RequestURI)) + fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "request handling started", slog.String("worker", handler.worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex), slog.String("url", handler.workerFrankenPHPContext.request.RequestURI)) } } @@ -298,9 +274,9 @@ func go_frankenphp_finish_worker_request(threadIndex C.uintptr_t, retval *C.zval if fc.logger.Enabled(fc.ctx, slog.LevelDebug) { if fc.request == nil { - fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "request handling finished", slog.String("worker", fc.worker.name), slog.Int("thread", thread.threadIndex)) + fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "request handling finished", slog.String("worker", fc.worker.qualifiedName), slog.Int("thread", thread.threadIndex)) } else { - fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "request handling finished", slog.String("worker", fc.worker.name), slog.Int("thread", thread.threadIndex), slog.String("url", fc.request.RequestURI)) + fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "request handling finished", slog.String("worker", fc.worker.qualifiedName), slog.Int("thread", thread.threadIndex), slog.String("url", fc.request.RequestURI)) } } } diff --git a/worker.go b/worker.go index 388dfbd031..9824edc934 100644 --- a/worker.go +++ b/worker.go @@ -8,6 +8,7 @@ import ( "os" "path/filepath" "runtime" + "slices" "sync" "sync/atomic" "time" @@ -20,7 +21,11 @@ import ( type worker struct { mercureContext - name string + // name as declared, unique within its server (or among global workers) + name string + // qualifiedName is unique across the process: ":" for + // server-scoped workers, name otherwise; used for metrics and logs + qualifiedName string fileName string matchRequest func(*http.Request) bool num int @@ -34,14 +39,14 @@ type worker struct { onThreadShutdown func(int) queuedRequests atomic.Int32 server *Server + // isBackgroundWorker marks this as a background (non-HTTP) worker + isBackgroundWorker bool } var ( - workers []*worker - workersByName map[string]*worker - globalWorkersByPath map[string]*worker - watcherIsEnabled bool - startupFailChan chan error + workers []*worker + watcherIsEnabled bool + startupFailChan chan error ) func initWorkers(opts []workerOpt) error { @@ -55,8 +60,7 @@ func initWorkers(opts []workerOpt) error { ) workers = make([]*worker, 0, len(opts)) - workersByName = make(map[string]*worker, len(opts)) - globalWorkersByPath = make(map[string]*worker, len(opts)) + qualifiedNames := make(map[string]bool, len(opts)) for _, o := range opts { w, err := newWorker(o) @@ -64,14 +68,27 @@ func initWorkers(opts []workerOpt) error { return err } - totalThreadsToStart += w.num - workers = append(workers, w) - workersByName[w.name] = w - if w.server == nil { - globalWorkersByPath[w.fileName] = w - } else if err := w.server.addWorker(w); err != nil { + if w.server != fallbackServer && !slices.Contains(servers, w.server) { + return fmt.Errorf("worker %q is scoped to a server that was not passed to WithServer()", w.name) + } + + // names and paths are unique within a server + if err := w.server.addWorker(w); err != nil { return err } + + // scoping makes qualified names unique in all but pathological + // cases: a global worker may still be named like the ":" + // of a scoped one, and metrics would merge the two series + if qualifiedNames[w.qualifiedName] { + return fmt.Errorf("two workers cannot report under the same name: %q", w.qualifiedName) + } + qualifiedNames[w.qualifiedName] = true + + totalThreadsToStart += w.num + workers = append(workers, w) + // reported here rather than in calculateMaxThreads(), where the name is not resolved yet + metrics.TotalWorkers(w.qualifiedName, w.num) } startupFailChan = make(chan error, totalThreadsToStart) @@ -79,7 +96,11 @@ func initWorkers(opts []workerOpt) error { for _, w := range workers { for range w.num { thread := getInactivePHPThread() - convertToWorkerThread(thread, w) + if w.isBackgroundWorker { + convertToBackgroundWorkerThread(thread, w) + } else { + convertToWorkerThread(thread, w) + } workersReady.Go(func() { thread.state.WaitFor(state.Ready, state.ShuttingDown, state.Done) @@ -119,23 +140,38 @@ func newWorker(o workerOpt) (*worker, error) { return nil, fmt.Errorf("worker file not found %q: %w", absFileName, err) } + if o.isBackgroundWorker { + // the name is the script's identity (exposed via FRANKENPHP_WORKER); + // empty names are reserved for the catch-all workers of a future build + if o.name == "" { + return nil, fmt.Errorf("background worker %q must have an explicit name", o.fileName) + } + if o.matchRequest != nil { + return nil, fmt.Errorf("background worker %q cannot match requests", o.name) + } + if o.maxThreads > 0 { + return nil, fmt.Errorf("background worker %q cannot set max_threads, it does not autoscale", o.name) + } + // Workers.SendRequest() and SendMessage() dispatch on requestChan, + // which no background thread reads + if o.extensionWorkers != nil { + return nil, fmt.Errorf("background worker %q cannot be an extension worker, those handle requests", o.name) + } + } + if o.name == "" { o.name = absFileName } - if o.server == nil { - if globalWorkersByPath[absFileName] != nil { - return nil, fmt.Errorf("two global workers cannot have the same filename: %q", absFileName) - } - - // no server means no set of requests to match against, the matcher would never run - if o.matchRequest != nil { - return nil, fmt.Errorf("worker %q has a request matcher but no server scope, use WithWorkerServerScope()", o.name) - } + // no server means no set of requests to match against, the matcher would never run + if o.server == nil && o.matchRequest != nil { + return nil, fmt.Errorf("worker %q has a request matcher but no server scope, use WithWorkerServerScope()", o.name) } - if workersByName[o.name] != nil { - return nil, fmt.Errorf("two workers cannot have the same name: %q", o.name) + // the same name may be declared in several servers, metrics and logs need a unique one + qualifiedName := o.name + if o.server != nil { + qualifiedName = o.server.name + ":" + o.name } // env should always contain FRANKENPHP_WORKER and the parent php_server env @@ -152,10 +188,23 @@ func newWorker(o workerOpt) (*worker, error) { } } - o.env["FRANKENPHP_WORKER\x00"] = "1" + // $_SERVER['FRANKENPHP_WORKER'] carries the worker name; scripts are + // documented to test its presence, not its value, so HTTP workers moving + // from "1" to their name breaks nothing. FRANKENPHP_WORKER_BACKGROUND + // tells a script it runs as a background worker, and is a presence-only + // flag for the same reason: its value is not part of the contract + o.env["FRANKENPHP_WORKER\x00"] = o.name + if o.isBackgroundWorker { + o.env["FRANKENPHP_WORKER_BACKGROUND\x00"] = "1" + } else { + // both are reserved: an env of the worker or of its server must not + // make an HTTP worker look like a background one + delete(o.env, "FRANKENPHP_WORKER_BACKGROUND\x00") + } w := &worker{ name: o.name, + qualifiedName: qualifiedName, fileName: absFileName, matchRequest: o.matchRequest, requestOptions: o.requestOptions, @@ -167,6 +216,14 @@ func newWorker(o workerOpt) (*worker, error) { onThreadReady: o.onThreadReady, onThreadShutdown: o.onThreadShutdown, server: o.server, + isBackgroundWorker: o.isBackgroundWorker, + } + + // a worker declared without a scope belongs to the fallback server, the + // one serving the requests that have no server either, so a worker + // always has one + if w.server == nil { + w.server = fallbackServer } w.configureMercure(&o) @@ -235,7 +292,7 @@ func (worker *worker) isAtThreadLimit() bool { } func (worker *worker) handleRequest(fc *frankenPHPContext) error { - metrics.StartWorkerRequest(worker.name) + metrics.StartWorkerRequest(worker.qualifiedName) runtime.Gosched() @@ -247,7 +304,7 @@ func (worker *worker) handleRequest(fc *frankenPHPContext) error { case thread.requestChan <- fc: worker.threadMutex.RUnlock() <-fc.done - metrics.StopWorkerRequest(worker.name, time.Since(fc.startedAt)) + metrics.StopWorkerRequest(worker.qualifiedName, time.Since(fc.startedAt)) return nil default: @@ -259,7 +316,7 @@ func (worker *worker) handleRequest(fc *frankenPHPContext) error { // if no thread was available, mark the request as queued and apply the scaling strategy worker.queuedRequests.Add(1) - metrics.QueuedWorkerRequest(worker.name) + metrics.QueuedWorkerRequest(worker.qualifiedName) for { workerScaleChan := scaleChan @@ -270,9 +327,9 @@ func (worker *worker) handleRequest(fc *frankenPHPContext) error { select { case worker.requestChan <- fc: worker.queuedRequests.Add(-1) - metrics.DequeuedWorkerRequest(worker.name) + metrics.DequeuedWorkerRequest(worker.qualifiedName) <-fc.done - metrics.StopWorkerRequest(worker.name, time.Since(fc.startedAt)) + metrics.StopWorkerRequest(worker.qualifiedName, time.Since(fc.startedAt)) return nil case workerScaleChan <- fc: @@ -280,8 +337,8 @@ func (worker *worker) handleRequest(fc *frankenPHPContext) error { case <-timeoutChan(time.Duration(maxWaitTime.Load())): // the request has timed out stalling worker.queuedRequests.Add(-1) - metrics.DequeuedWorkerRequest(worker.name) - metrics.StopWorkerRequest(worker.name, time.Since(fc.startedAt)) + metrics.DequeuedWorkerRequest(worker.qualifiedName) + metrics.StopWorkerRequest(worker.qualifiedName, time.Since(fc.startedAt)) fc.reject(ErrMaxWaitTimeExceeded) diff --git a/workerextension.go b/workerextension.go index 814eea1565..0965afd0c8 100644 --- a/workerextension.go +++ b/workerextension.go @@ -26,12 +26,20 @@ type extensionWorkers struct { // EXPERIMENTAL: SendRequest sends an HTTP request to the worker and writes the response to the provided ResponseWriter. func (w *extensionWorkers) SendRequest(rw http.ResponseWriter, r *http.Request) error { - fr, err := NewRequestWithContext( - r, - WithOriginalRequest(r), - WithWorkerName(w.name), - ) + // the worker only exists between Init() and Shutdown() + if w.internalWorker == nil { + return ErrNotRunning + } + + opts := []RequestOption{WithOriginalRequest(r), WithWorkerName(w.name)} + + // worker names are resolved within a server, so a scoped worker is only + // reachable through its own server + if server := w.internalWorker.server; server != nil { + return server.ServeHTTP(rw, r, opts...) + } + fr, err := NewRequestWithContext(r, opts...) if err != nil { return err } @@ -40,11 +48,19 @@ func (w *extensionWorkers) SendRequest(rw http.ResponseWriter, r *http.Request) } func (w *extensionWorkers) NumThreads() int { + if w.internalWorker == nil { + return 0 + } + return w.internalWorker.countThreads() } // EXPERIMENTAL: SendMessage sends a message to the worker and waits for a response. func (w *extensionWorkers) SendMessage(ctx context.Context, message any, rw http.ResponseWriter) (any, error) { + if w.internalWorker == nil { + return nil, ErrNotRunning + } + fc := newContextFromMessage(message, rw, ctx, w.internalWorker) err := w.internalWorker.handleRequest(fc) diff --git a/workerextension_test.go b/workerextension_test.go index e861ec97a3..f8dfab0a23 100644 --- a/workerextension_test.go +++ b/workerextension_test.go @@ -77,6 +77,36 @@ func TestWorkerExtensionSendMessage(t *testing.T) { assert.Equal(t, "received message: Hello Workers", ret) } +// an extension worker scoped to a server stays reachable through +// SendRequest(), which resolves the name within that server +func TestWorkerExtensionOnServer(t *testing.T) { + t.Cleanup(Shutdown) + + server, err := NewServer("testdata/", WithServerName("api")) + require.NoError(t, err) + externalWorker, o := WithExtensionWorkers("scopedWorker", "testdata/worker.php", 1, WithWorkerServerScope(server)) + require.NoError(t, Init(o, WithServer(server))) + + // a URI that does not match the worker's own script, so only the name + // lookup can route this to it + w := httptest.NewRecorder() + require.NoError(t, externalWorker.SendRequest(w, httptest.NewRequest("GET", "http://example.com/index.php", nil))) + + body, err := io.ReadAll(w.Result().Body) + require.NoError(t, err) + assert.Contains(t, string(body), "Requests handled: 0") +} + +// background workers never read requestChan, so an extension cannot send +// them requests or messages +func TestErrorIfExtensionWorkerIsBackground(t *testing.T) { + t.Cleanup(Shutdown) + + _, o := WithExtensionWorkers("backgroundExtension", "testdata/bgworker/basic.php", 1, WithWorkerBackground()) + + require.ErrorContains(t, Init(o), "cannot be an extension worker") +} + func TestErrorIf2WorkersHaveSameName(t *testing.T) { _, o1 := WithExtensionWorkers("duplicateWorker", "testdata/worker.php", 1) _, o2 := WithExtensionWorkers("duplicateWorker", "testdata/worker2.php", 1) diff --git a/workerlifecycle.go b/workerlifecycle.go new file mode 100644 index 0000000000..b4b524b41b --- /dev/null +++ b/workerlifecycle.go @@ -0,0 +1,69 @@ +package frankenphp + +import "github.com/dunglas/frankenphp/internal/state" + +// workerLifecycle is the part of a worker thread that HTTP and background +// workers share: the thread, its worker, and the states both walk through +// between two runs of the script. Handlers embed it and supply what differs, +// see workerHandler. +type workerLifecycle struct { + state *state.ThreadState + thread *phpThread + worker *worker +} + +// workerHandler is a threadHandler running a worker script, plus the two +// steps the shared lifecycle delegates +type workerHandler interface { + threadHandler + // startScript prepares a run and returns the script to execute, or an + // empty string to stop the thread + startScript() string + // resetForReboot clears what a handler counts per run of the thread + resetForReboot() +} + +func newWorkerLifecycle(thread *phpThread, worker *worker) workerLifecycle { + return workerLifecycle{state: thread.state, thread: thread, worker: worker} +} + +// beforeScriptExecution returns the name of the script to run, or an empty +// string to stop the thread +func (l *workerLifecycle) beforeScriptExecution(handler workerHandler) string { + switch l.state.Get() { + case state.TransitionRequested: + l.detach() + + return l.thread.transitionToNewHandler() + case state.Ready, state.TransitionComplete: + l.thread.updateContext(true) + if l.worker.onThreadReady != nil { + l.worker.onThreadReady(l.thread.threadIndex) + } + + return handler.startScript() + case state.Rebooting, state.ForceRebooting: + return "" + case state.RebootReady: + handler.resetForReboot() + l.state.Set(state.Ready) + + return handler.beforeScriptExecution() + case state.ShuttingDown: + l.detach() + + // signal to stop + return "" + default: + panic("unexpected state: " + l.state.Name()) + } +} + +// detach takes the thread off its worker, on the paths that stop running its +// script for good +func (l *workerLifecycle) detach() { + if l.worker.onThreadShutdown != nil { + l.worker.onThreadShutdown(l.thread.threadIndex) + } + l.worker.detachThread(l.thread) +} From 910ed5e605f1e68eabb59ca0428b19670eae362e Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Fri, 11 Sep 2026 12:05:08 +0200 Subject: [PATCH 02/20] fix: a blocking receive on the handle marks a background worker ready stream_socket_recvfrom() and the other transport receives never reach the read op: they go through the stream's transport API, which the handle inherited unchanged from the socket ops. A script parking that way was therefore never reported ready and Init() waited for it forever. The set_option op is now wrapped too, reporting the wait on a receive. The new fixture hangs Init() without it. --- bgworker_test.go | 27 +++++++++++++++++++++++++++ frankenphp.c | 14 ++++++++++++++ testdata/bgworker/recv.php | 12 ++++++++++++ 3 files changed, 53 insertions(+) create mode 100644 testdata/bgworker/recv.php diff --git a/bgworker_test.go b/bgworker_test.go index 8699f40f35..e48a582ea6 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -292,6 +292,33 @@ func TestBackgroundWorkerParksOnRead(t *testing.T) { } } +// TestBackgroundWorkerParksOnReceive checks that a blocking receive counts +// as a wait as well: it reaches the stream through the transport API rather +// than the read op, so Init() would hang if only reads reported readiness. +func TestBackgroundWorkerParksOnReceive(t *testing.T) { + sentinel := filepath.Join(t.TempDir(), "bg-recv.sentinel") + + require.NoError(t, frankenphp.Init( + frankenphp.WithWorkers("bg-recv", "testdata/bgworker/recv.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(2), + )) + requireFileEventually(t, sentinel, "background worker parked on a receive did not start") + + done := make(chan struct{}) + go func() { + frankenphp.Shutdown() + close(done) + }() + select { + case <-done: + case <-time.After(10 * time.Second): + t.Fatal("Shutdown did not return within 10s: the receive did not observe EOF") + } +} + // TestBackgroundWorkerRestartDrainsParkedScript checks that RestartWorkers() // wakes a parked background script through the drain and re-runs it. func TestBackgroundWorkerRestartDrainsParkedScript(t *testing.T) { diff --git a/frankenphp.c b/frankenphp.c index 18bdeffdfe..5e6fa23e0b 100644 --- a/frankenphp.c +++ b/frankenphp.c @@ -1205,6 +1205,19 @@ static int frankenphp_worker_handle_cast(php_stream *stream, int castas, return php_stream_socket_ops.cast(stream, castas, ret); } +/* stream_socket_recvfrom() and the other transport receives do not go + * through the read op, they reach the stream through its transport API, and + * a blocking receive is a wait on the handle too */ +static int frankenphp_worker_handle_set_option(php_stream *stream, int option, + int value, void *ptrparam) { + if (option == PHP_STREAM_OPTION_XPORT_API && ptrparam != NULL && + ((php_stream_xport_param *)ptrparam)->op == STREAM_XPORT_OP_RECV) { + frankenphp_worker_handle_waited(); + } + + return php_stream_socket_ops.set_option(stream, option, value, ptrparam); +} + static int frankenphp_worker_handle_close(php_stream *stream, int close_handle) { (void)close_handle; @@ -1329,6 +1342,7 @@ PHP_MINIT_FUNCTION(frankenphp) { frankenphp_worker_handle_ops.read = frankenphp_worker_handle_read; frankenphp_worker_handle_ops.cast = frankenphp_worker_handle_cast; frankenphp_worker_handle_ops.close = frankenphp_worker_handle_close; + frankenphp_worker_handle_ops.set_option = frankenphp_worker_handle_set_option; register_frankenphp_symbols(module_number); #ifndef PHP_WIN32 diff --git a/testdata/bgworker/recv.php b/testdata/bgworker/recv.php new file mode 100644 index 0000000000..1e9f711345 --- /dev/null +++ b/testdata/bgworker/recv.php @@ -0,0 +1,12 @@ + Date: Fri, 11 Sep 2026 12:05:51 +0200 Subject: [PATCH 03/20] fix: never inherit the reserved worker variables FRANKENPHP_WORKER and FRANKENPHP_WORKER_BACKGROUND say what is running the script, so FrankenPHP owns them. Dropping the background flag from the worker's own env was not enough: $_SERVER is built from the process environment first, then the php_server env, then the worker's, so a value from any layer below survived and an HTTP worker answered the documented isset() check. They are now removed after all the layers are merged, for the kind of thread that must not carry them. The test declares the flag in the worker env, the server env and the process environment. --- bgworker_test.go | 8 +++++--- frankenphp.c | 15 +++++++++++++++ 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/bgworker_test.go b/bgworker_test.go index e48a582ea6..12331c3a2d 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -360,12 +360,14 @@ func TestGetWorkerHandleOutsideBackgroundWorker(t *testing.T) { // FRANKENPHP_WORKER_BACKGROUND flag. func TestWorkerNameInServerVars(t *testing.T) { sentinel := filepath.Join(t.TempDir(), "flag.txt") - server, err := frankenphp.NewServer(testDataDir) + t.Setenv("FRANKENPHP_WORKER_BACKGROUND", "1") + server, err := frankenphp.NewServer(testDataDir, frankenphp.WithServerEnv(map[string]string{"FRANKENPHP_WORKER_BACKGROUND": "1"})) require.NoError(t, err) initServers(t, frankenphp.WithServer(server), - // the flag is reserved: an env setting it must not make an HTTP - // worker look like a background one + // both names are reserved: neither the worker env here, nor the + // server env, nor the process environment set below may make an + // HTTP worker look like a background one frankenphp.WithWorkers("web", testDataDir+"worker-name.php", 1, frankenphp.WithWorkerServerScope(server), frankenphp.WithWorkerEnv(map[string]string{"FRANKENPHP_WORKER_BACKGROUND": "1"}), diff --git a/frankenphp.c b/frankenphp.c index 5e6fa23e0b..6e0c43dfdf 100644 --- a/frankenphp.c +++ b/frankenphp.c @@ -1653,6 +1653,21 @@ static void frankenphp_register_variables(zval *track_vars_array) { /* import environment and CGI variables from the request context in go */ go_register_server_variables(frankenphp_thread_index(), track_vars_array); + /* FRANKENPHP_WORKER and FRANKENPHP_WORKER_BACKGROUND say what is running + * the script, so FrankenPHP owns them: a value inherited from the process + * environment, from a php_server or from a worker would otherwise make a + * script take the wrong branch. The worker's own values were merged above, + * the layers below are dropped here. */ + if (!is_worker_thread) { + zend_hash_str_del(Z_ARRVAL_P(track_vars_array), "FRANKENPHP_WORKER", + sizeof("FRANKENPHP_WORKER") - 1); + } + if (!is_background_worker) { + zend_hash_str_del(Z_ARRVAL_P(track_vars_array), + "FRANKENPHP_WORKER_BACKGROUND", + sizeof("FRANKENPHP_WORKER_BACKGROUND") - 1); + } + /* Some variables are already present in SG(request_info) */ frankenphp_register_variables_from_request_info(track_vars_array); } From 470a96905eb62283ac527c08784448924b83d43e Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Fri, 11 Sep 2026 12:07:23 +0200 Subject: [PATCH 04/20] fix: cap the restart backoff, and EOF a handle a forked child still holds Two small ones on the restart path. The quadratic backoff multiplied before capping, which overflows a duration of nanoseconds past some 300k consecutive failures and sleeps for a negative one; a background worker crashing past its ready point counts without bound and reaches that in a few days of retries. The cap now comes first, for the same schedule. Closing the Go side of a handle only lands as EOF on the script's end while no other process holds a copy, and a pcntl_fork() child inherits every descriptor of the process, the pairs of the other threads included. Shutting the write direction down first sends the FIN regardless, so a parked script still wakes up instead of waiting out the force-kill. Also drops a platform conditional: php_network.h maps closesocket to close outside Windows. --- frankenphp.c | 14 ++++++++++---- threadworker.go | 10 ++++++++-- threadworker_test.go | 20 ++++++++++++++++++++ 3 files changed, 38 insertions(+), 6 deletions(-) create mode 100644 threadworker_test.go diff --git a/frankenphp.c b/frankenphp.c index 6e0c43dfdf..d1bca8c830 100644 --- a/frankenphp.c +++ b/frankenphp.c @@ -368,11 +368,8 @@ static void frankenphp_worker_close_sock(php_socket_t s) { if (s == SOCK_ERR) { return; } -#ifdef PHP_WIN32 + /* php_network.h maps closesocket to close outside Windows */ closesocket(s); -#else - close(s); -#endif } /* keep the pair out of processes the script may spawn: a child holding the @@ -463,6 +460,15 @@ void frankenphp_worker_close_stop_sock(intptr_t s) { if (s < 0) { return; } + /* Closing this end only lands as EOF on the script's end while no other + * process holds a copy of it, and a pcntl_fork() child inherits every + * descriptor of the process, including the pairs of the other threads. + * Shutting the write direction down sends the FIN regardless. */ +#ifdef PHP_WIN32 + shutdown((php_socket_t)s, SD_SEND); +#else + shutdown((php_socket_t)s, SHUT_WR); +#endif frankenphp_worker_close_sock((php_socket_t)s); } diff --git a/threadworker.go b/threadworker.go index 233a58a65a..066c94f0c1 100644 --- a/threadworker.go +++ b/threadworker.go @@ -148,9 +148,15 @@ func tearDownWorkerScript(handler *workerThread, exitStatus int) { // restartBackoff is the wait before a worker script is re-run after a // failure: quadratic in the number of consecutive failures, capped at one -// second; shared by HTTP and background workers +// second; shared by HTTP and background workers. The cap comes before the +// multiplication, which overflows a duration past some 300k failures, a +// count a crash loop reaches on its own in a few days func restartBackoff(failures int) time.Duration { - return min(time.Duration(failures*failures*100)*time.Millisecond, time.Second) + if failures >= 4 { + return time.Second + } + + return time.Duration(failures*failures*100) * time.Millisecond } // waitForWorkerRequest is called during frankenphp_handle_request in the php worker script. diff --git a/threadworker_test.go b/threadworker_test.go new file mode 100644 index 0000000000..24e5a2e410 --- /dev/null +++ b/threadworker_test.go @@ -0,0 +1,20 @@ +package frankenphp + +import ( + "math" + "testing" + "time" + + "github.com/stretchr/testify/assert" +) + +func TestRestartBackoff(t *testing.T) { + assert.Equal(t, time.Duration(0), restartBackoff(0)) + assert.Equal(t, 100*time.Millisecond, restartBackoff(1)) + assert.Equal(t, 400*time.Millisecond, restartBackoff(2)) + assert.Equal(t, 900*time.Millisecond, restartBackoff(3)) + assert.Equal(t, time.Second, restartBackoff(4)) + // a crash loop counts without bound, the quadratic must not overflow + assert.Equal(t, time.Second, restartBackoff(303701)) + assert.Equal(t, time.Second, restartBackoff(math.MaxInt32)) +} From 967ac030785874e2937c6f6d1ed0999525cd5af3 Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Fri, 11 Sep 2026 12:09:24 +0200 Subject: [PATCH 05/20] fix: resolve max_threads auto on the HTTP budget An explicit max_threads gets the reservation added on top, so the HTTP capacity it describes is preserved. The automatic limit did not: it resolved from a num_threads that already included the background threads, so declaring three of them turned a limit of 4 into 10, and a memory-derived estimate could be swallowed whole, leaving no room for HTTP autoscaling at all. The main thread now knows what part of its count is reserved, resolves the estimate on the rest, and adds the reservation back, like the explicit path. --- bgworker_test.go | 22 ++++++++++++++++++++++ frankenphp.go | 6 +++--- phpmainthread.go | 36 ++++++++++++++++++++++-------------- phpmainthread_test.go | 8 ++++---- threadbackgroundworker.go | 8 ++++++-- types_test.go | 2 +- 6 files changed, 58 insertions(+), 24 deletions(-) diff --git a/bgworker_test.go b/bgworker_test.go index 12331c3a2d..60f89dec71 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -458,6 +458,28 @@ func TestBackgroundWorkerThreadsComeOnTop(t *testing.T) { requireFileEventually(t, sentinel, "background worker did not start with num_threads 1") } +// TestBackgroundWorkerThreadsComeOnTopOfAutoMaxThreads checks that the +// automatic limit is an HTTP one too: the reservation is added to what it +// resolves, instead of eating into it +func TestBackgroundWorkerThreadsComeOnTopOfAutoMaxThreads(t *testing.T) { + sentinel := filepath.Join(t.TempDir(), "bg-auto.sentinel") + initServers(t, + frankenphp.WithWorkers("bg-auto", "testdata/bgworker/basic.php", 2, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(2), + frankenphp.WithMaxThreads(-1), + frankenphp.WithPhpIni(map[string]string{"memory_limit": "-1"}), + ) + requireFileEventually(t, sentinel, "background worker did not start") + + // an unlimited memory_limit falls back to twice the HTTP threads, so + // 2*2 HTTP plus the 2 reserved, not (2+2)*2 + state := frankenphp.DebugState() + assert.Equal(t, 6, len(state.ThreadDebugStates)+state.ReservedThreadCount) +} + // TestBackgroundWorkerBootFailuresThenSucceeds checks that boot failures below // max_consecutive_failures are retried with the backoff and Init() still // succeeds once a run reaches its ready point. diff --git a/frankenphp.go b/frankenphp.go index 63c1242799..b8928eed59 100644 --- a/frankenphp.go +++ b/frankenphp.go @@ -354,11 +354,11 @@ func Init(options ...Option) error { maxThreads := opt.maxThreads if maxThreads > 0 { - // in auto mode (maxThreads < 0), the resolved value is floored to the - // thread count, background threads included maxThreads += backgroundThreads } - mainThread, err := initPHPThreads(opt.numThreads+backgroundThreads, maxThreads, opt.phpIni) + // in auto mode (maxThreads < 0), the main thread adds the reservation to + // the limit it resolves, see setAutomaticMaxThreads() + mainThread, err := initPHPThreads(opt.numThreads+backgroundThreads, maxThreads, backgroundThreads, opt.phpIni) if err != nil { shutdown() return err diff --git a/phpmainthread.go b/phpmainthread.go index 0878762341..db48197b22 100644 --- a/phpmainthread.go +++ b/phpmainthread.go @@ -20,12 +20,16 @@ import ( // represents the main PHP thread // the thread needs to keep running as long as all other threads are running type phpMainThread struct { - state *state.ThreadState - done chan struct{} - numThreads int - maxThreads int - phpIni map[string]string - isRebooting atomic.Bool + state *state.ThreadState + done chan struct{} + numThreads int + maxThreads int + // backgroundThreads is the part of numThreads and maxThreads that + // background workers reserve: it takes no part in the HTTP budget, + // see calculateMaxThreads() + backgroundThreads int + phpIni map[string]string + isRebooting atomic.Bool } var ( @@ -41,13 +45,14 @@ var ( // initPHPThreads starts the main PHP thread, // a fixed number of inactive PHP threads // and reserves a fixed number of possible PHP threads -func initPHPThreads(numThreads int, numMaxThreads int, phpIni map[string]string) (*phpMainThread, error) { +func initPHPThreads(numThreads int, numMaxThreads int, backgroundThreads int, phpIni map[string]string) (*phpMainThread, error) { mainThread = &phpMainThread{ - state: state.NewThreadState(), - done: make(chan struct{}), - numThreads: numThreads, - maxThreads: numMaxThreads, - phpIni: phpIni, + state: state.NewThreadState(), + done: make(chan struct{}), + numThreads: numThreads, + maxThreads: numMaxThreads, + backgroundThreads: backgroundThreads, + phpIni: phpIni, } // initialize the first thread @@ -263,18 +268,21 @@ func go_frankenphp_main_thread_is_ready() { // max_threads = auto // setAutomaticMaxThreads estimates the amount of threads based on php.ini and system memory_limit // If unable to get the system's memory limit, simply double num_threads +// The estimate is an HTTP one, like an explicit max_threads: the threads +// background workers reserve are added to it rather than taken out of it func (mainThread *phpMainThread) setAutomaticMaxThreads() { if mainThread.maxThreads >= 0 { return } + httpThreads := mainThread.numThreads - mainThread.backgroundThreads perThreadMemoryLimit := int64(C.frankenphp_get_current_memory_limit()) totalSysMemory := memory.TotalSysMemory() if perThreadMemoryLimit <= 0 || totalSysMemory == 0 { - mainThread.maxThreads = mainThread.numThreads * 2 + mainThread.maxThreads = httpThreads*2 + mainThread.backgroundThreads return } maxAllowedThreads := totalSysMemory / uint64(perThreadMemoryLimit) - mainThread.maxThreads = int(maxAllowedThreads) + mainThread.maxThreads = int(maxAllowedThreads) + mainThread.backgroundThreads if globalLogger.Enabled(globalCtx, slog.LevelDebug) { globalLogger.LogAttrs(globalCtx, slog.LevelDebug, "Automatic thread limit", slog.Int("perThreadMemoryLimitMB", int(perThreadMemoryLimit/1024/1024)), slog.Int("maxThreads", mainThread.maxThreads)) diff --git a/phpmainthread_test.go b/phpmainthread_test.go index 08212466d9..8d78e25e74 100644 --- a/phpmainthread_test.go +++ b/phpmainthread_test.go @@ -26,7 +26,7 @@ func setupGlobals(t *testing.T) { } func TestStartAndStopTheMainThreadWithOneInactiveThread(t *testing.T) { - _, err := initPHPThreads(1, 1, nil) // boot 1 thread + _, err := initPHPThreads(1, 1, 0, nil) // boot 1 thread assert.NoError(t, err) assert.Len(t, phpThreads, 1) @@ -41,7 +41,7 @@ func TestStartAndStopTheMainThreadWithOneInactiveThread(t *testing.T) { func TestTransitionRegularThreadToWorkerThread(t *testing.T) { setupGlobals(t) - _, err := initPHPThreads(1, 1, nil) + _, err := initPHPThreads(1, 1, 0, nil) assert.NoError(t, err) // transition to regular thread @@ -66,7 +66,7 @@ func TestTransitionRegularThreadToWorkerThread(t *testing.T) { func TestTransitionAThreadBetween2DifferentWorkers(t *testing.T) { setupGlobals(t) - _, err := initPHPThreads(1, 1, nil) + _, err := initPHPThreads(1, 1, 0, nil) assert.NoError(t, err) firstWorker := getDummyWorker(t, "transition-worker-1.php") secondWorker := getDummyWorker(t, "transition-worker-2.php") @@ -228,7 +228,7 @@ func TestQueuedRequestSurvivesReload(t *testing.T) { func TestFinishBootingAWorkerScript(t *testing.T) { setupGlobals(t) - _, err := initPHPThreads(1, 1, nil) + _, err := initPHPThreads(1, 1, 0, nil) assert.NoError(t, err) // boot the worker diff --git a/threadbackgroundworker.go b/threadbackgroundworker.go index 38967fa31b..8841455036 100644 --- a/threadbackgroundworker.go +++ b/threadbackgroundworker.go @@ -142,9 +142,13 @@ func (handler *backgroundWorkerThread) setupScript() error { handler.isBootingScript = true metrics.StartWorker(handler.worker.qualifiedName) + // the run's logger and context, not the globals: Stop() does not wait + // for a callback that already started, and a shutdown finishing + // meanwhile resets those + logger, ctx, name, threadIndex := fc.logger, fc.ctx, handler.worker.qualifiedName, handler.thread.threadIndex handler.bootTimer = time.AfterFunc(backgroundBootWarnDelay, func() { - if globalLogger.Enabled(globalCtx, slog.LevelWarn) { - globalLogger.LogAttrs(globalCtx, slog.LevelWarn, "background worker has not waited on its handle yet, Init() and Shutdown() wait for it, see frankenphp_get_worker_handle()", slog.String("worker", handler.worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex)) + if logger.Enabled(ctx, slog.LevelWarn) { + logger.LogAttrs(ctx, slog.LevelWarn, "background worker has not waited on its handle yet, Init() and Shutdown() wait for it, see frankenphp_get_worker_handle()", slog.String("worker", name), slog.Int("thread", threadIndex)) } }) diff --git a/types_test.go b/types_test.go index a08f90725e..7d547e9681 100644 --- a/types_test.go +++ b/types_test.go @@ -14,7 +14,7 @@ func testOnDummyPHPThread(t *testing.T, test func()) { t.Helper() globalLogger = slog.Default() - _, err := initPHPThreads(1, 1, nil) // boot 1 thread + _, err := initPHPThreads(1, 1, 0, nil) // boot 1 thread assert.NoError(t, err) handler := convertToTaskThread(phpThreads[0]) From 2384f90d93a88383d85143ae256b833253b1a004 Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Fri, 11 Sep 2026 12:12:26 +0200 Subject: [PATCH 06/20] chore: simplifications and wording from the review resetForReboot() was redundant, setupWorkerScript() already resets the request count before every run, so the lifecycle interface is down to the one step that differs. A worker always has a server now, so the extension dispatch has a single path. zend_alter_ini_entry_chars() takes the literal, and php_network.h maps closesocket to close outside Windows. The ready_workers help text and docs said fetching the handle marks a background worker ready; the first wait on its stream does, as the validation test asserts. A parked fixture no longer disables max_execution_time itself, which is how the engine disabling it went untested, and a new test parks past a one-second limit with max_input_time set, the case where php_execute_script() re-arms it. --- bgworker_test.go | 26 ++++++++++++++++++++++++++ caddy/caddy_test.go | 12 ++++++------ docs/metrics.md | 2 +- frankenphp.c | 4 +--- metrics.go | 4 ++-- metrics_test.go | 2 +- testdata/bgworker/basic.php | 8 +++----- testdata/bgworker/no-time-limit.php | 9 +++++++++ threadbackgroundworker.go | 3 --- threadworker.go | 4 ---- workerextension.go | 17 +++-------------- workerlifecycle.go | 7 ++----- 12 files changed, 54 insertions(+), 44 deletions(-) create mode 100644 testdata/bgworker/no-time-limit.php diff --git a/bgworker_test.go b/bgworker_test.go index 60f89dec71..e43925eca5 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -480,6 +480,32 @@ func TestBackgroundWorkerThreadsComeOnTopOfAutoMaxThreads(t *testing.T) { assert.Equal(t, 6, len(state.ThreadDebugStates)+state.ReservedThreadCount) } +// TestBackgroundWorkerHasNoExecutionTimeout checks that a script parking +// past max_execution_time is not cut short, without the fixture disabling +// the limit itself. max_input_time is set because php_execute_script() +// re-arms the limit from the ini when it is. +func TestBackgroundWorkerHasNoExecutionTimeout(t *testing.T) { + countFile := filepath.Join(t.TempDir(), "runs") + initServers(t, + frankenphp.WithWorkers("bg-timeout", "testdata/bgworker/no-time-limit.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_COUNT_FILE": countFile}), + ), + frankenphp.WithNumThreads(2), + frankenphp.WithPhpIni(map[string]string{"max_execution_time": "1", "max_input_time": "1"}), + ) + + runs := func() int { + b, _ := os.ReadFile(countFile) + + return bytes.Count(b, []byte("\n")) + } + require.Eventually(t, func() bool { return runs() == 1 }, 5*time.Second, 25*time.Millisecond, "background worker did not start") + // well past the limit it must not enforce + time.Sleep(2500 * time.Millisecond) + assert.Equal(t, 1, runs(), "the worker was restarted, so its run was cut short by max_execution_time") +} + // TestBackgroundWorkerBootFailuresThenSucceeds checks that boot failures below // max_consecutive_failures are retried with the backoff and Init() still // succeeds once a run reaches its ready point. diff --git a/caddy/caddy_test.go b/caddy/caddy_test.go index 29004d10b7..b53abfa9a6 100644 --- a/caddy/caddy_test.go +++ b/caddy/caddy_test.go @@ -878,7 +878,7 @@ func TestWorkerMetrics(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="` + workerName + `"} 10 - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="` + workerName + `"} 2 ` @@ -1035,7 +1035,7 @@ func TestNamedWorkerMetrics(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="my_app"} 10 - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="my_app"} 2 ` @@ -1131,7 +1131,7 @@ func TestAutoWorkerConfig(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="` + workerName + `"} 10 - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="` + workerName + `"} ` + workers + ` ` @@ -1499,7 +1499,7 @@ func TestMultiWorkersMetrics(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="service1"} 10 - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="service1"} 2 frankenphp_ready_workers{worker="service2"} 3 @@ -1653,7 +1653,7 @@ func TestWorkerRestart(t *testing.T) { // Check metrics expectedMetrics := ` - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="service"} 1 # HELP frankenphp_total_workers Total number of PHP workers for this worker @@ -1681,7 +1681,7 @@ func TestWorkerRestart(t *testing.T) { // frankenphp_ready_workers should be back to 1 even after worker restarts expectedMetrics = ` - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="service"} 1 # HELP frankenphp_total_workers Total number of PHP workers for this worker diff --git a/docs/metrics.md b/docs/metrics.md index 6239ae8829..6b8c42b0fc 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -19,7 +19,7 @@ When [Caddy metrics](https://caddyserver.com/docs/metrics) are enabled, FrankenP - `frankenphp_busy_workers{worker="[worker_name]"}`: The number of workers currently processing a request. - `frankenphp_worker_request_time{worker="[worker_name]"}`: The time spent processing requests by all workers. - `frankenphp_worker_request_count{worker="[worker_name]"}`: The number of requests processed by all workers. -- `frankenphp_ready_workers{worker="[worker_name]"}`: The number of workers that have reached their ready point at least once: `frankenphp_handle_request()` for HTTP workers, `frankenphp_get_worker_handle()` for background workers. +- `frankenphp_ready_workers{worker="[worker_name]"}`: The number of workers that have reached their ready point at least once: `frankenphp_handle_request()` for HTTP workers, the first wait on the stream of `frankenphp_get_worker_handle()` for background workers. - `frankenphp_worker_crashes{worker="[worker_name]"}`: The number of times a worker has unexpectedly terminated. - `frankenphp_worker_restarts{worker="[worker_name]"}`: The number of times a worker has been deliberately restarted. - `frankenphp_worker_queue_depth{worker="[worker_name]"}`: The number of queued requests. diff --git a/frankenphp.c b/frankenphp.c index d1bca8c830..fa186c101c 100644 --- a/frankenphp.c +++ b/frankenphp.c @@ -477,10 +477,8 @@ void frankenphp_worker_close_stop_sock(intptr_t s) { static void frankenphp_disable_execution_timeout(void) { zend_string *key = zend_string_init("max_execution_time", sizeof("max_execution_time") - 1, 0); - zend_string *value = zend_string_init("0", 1, 0); - zend_alter_ini_entry(key, value, PHP_INI_USER, PHP_INI_STAGE_RUNTIME); + zend_alter_ini_entry_chars(key, "0", 1, PHP_INI_USER, PHP_INI_STAGE_RUNTIME); zend_string_release(key); - zend_string_release(value); } void frankenphp_update_local_thread_context(bool is_worker) { diff --git a/metrics.go b/metrics.go index d51a3726a8..03fe56b6c3 100644 --- a/metrics.go +++ b/metrics.go @@ -11,7 +11,7 @@ import ( const ( StopReasonCrash = iota StopReasonRestart - StopReasonBootFailure // worker exited before reaching its ready point: frankenphp_handle_request, or frankenphp_get_worker_handle for background workers + StopReasonBootFailure // worker exited before reaching its ready point: frankenphp_handle_request, or the first wait on the handle for background workers ) type StopReason int @@ -177,7 +177,7 @@ func (m *PrometheusMetrics) TotalWorkers(string, int) { m.readyWorkers = prometheus.NewGaugeVec(prometheus.GaugeOpts{ Namespace: ns, Name: "ready_workers", - Help: "Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers", + Help: "Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers", }, basicLabels) m.mustRegister(m.readyWorkers) } diff --git a/metrics_test.go b/metrics_test.go index e5ca5e3c71..8c06a5686a 100644 --- a/metrics_test.go +++ b/metrics_test.go @@ -185,7 +185,7 @@ func TestPrometheusMetrics_TestStopReasonCrash(t *testing.T) { name: "Testing ReadyWorkers", c: m.readyWorkers, metadata: ` - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers # TYPE frankenphp_ready_workers gauge `, expect: ` diff --git a/testdata/bgworker/basic.php b/testdata/bgworker/basic.php index 6e779a11cb..2530c6ee58 100644 --- a/testdata/bgworker/basic.php +++ b/testdata/bgworker/basic.php @@ -1,10 +1,8 @@ Date: Fri, 11 Sep 2026 12:14:49 +0200 Subject: [PATCH 07/20] test: cover the parked run and the handle cache Two paths the suite took for granted. A worker parked on its handle must survive default_socket_timeout as well as max_execution_time, so the fixture that disables neither now runs with both set to one second. And a run gets one handle: the second fetch is the same stream, a fetch after closing it is a fresh one, and the drain still reaches the script through that one. --- bgworker_test.go | 44 +++++++++++++++++++++++------ testdata/bgworker/no-time-limit.php | 9 +++--- testdata/bgworker/refetch.php | 16 +++++++++++ 3 files changed, 57 insertions(+), 12 deletions(-) create mode 100644 testdata/bgworker/refetch.php diff --git a/bgworker_test.go b/bgworker_test.go index e43925eca5..7f8e8bbdc0 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -480,11 +480,12 @@ func TestBackgroundWorkerThreadsComeOnTopOfAutoMaxThreads(t *testing.T) { assert.Equal(t, 6, len(state.ThreadDebugStates)+state.ReservedThreadCount) } -// TestBackgroundWorkerHasNoExecutionTimeout checks that a script parking -// past max_execution_time is not cut short, without the fixture disabling -// the limit itself. max_input_time is set because php_execute_script() -// re-arms the limit from the ini when it is. -func TestBackgroundWorkerHasNoExecutionTimeout(t *testing.T) { +// TestBackgroundWorkerParkingIsNotInterrupted checks that a script parked +// on its handle is not cut short by the two limits it never disables +// itself: max_execution_time, which php_execute_script() re-arms from the +// ini when max_input_time is set, and default_socket_timeout, which the +// handle overrides with an infinite read timeout. +func TestBackgroundWorkerParkingIsNotInterrupted(t *testing.T) { countFile := filepath.Join(t.TempDir(), "runs") initServers(t, frankenphp.WithWorkers("bg-timeout", "testdata/bgworker/no-time-limit.php", 1, @@ -492,7 +493,7 @@ func TestBackgroundWorkerHasNoExecutionTimeout(t *testing.T) { frankenphp.WithWorkerEnv(map[string]string{"BG_COUNT_FILE": countFile}), ), frankenphp.WithNumThreads(2), - frankenphp.WithPhpIni(map[string]string{"max_execution_time": "1", "max_input_time": "1"}), + frankenphp.WithPhpIni(map[string]string{"max_execution_time": "1", "max_input_time": "1", "default_socket_timeout": "1"}), ) runs := func() int { @@ -501,9 +502,36 @@ func TestBackgroundWorkerHasNoExecutionTimeout(t *testing.T) { return bytes.Count(b, []byte("\n")) } require.Eventually(t, func() bool { return runs() == 1 }, 5*time.Second, 25*time.Millisecond, "background worker did not start") - // well past the limit it must not enforce + // well past both limits time.Sleep(2500 * time.Millisecond) - assert.Equal(t, 1, runs(), "the worker was restarted, so its run was cut short by max_execution_time") + assert.Equal(t, 1, runs(), "the worker was restarted, so a limit interrupted its park") +} + +// TestBackgroundWorkerHandleClosedAndFetchedAgain checks the handle cache: +// a run gets one stream, closing it yields a fresh one on the next fetch, +// and the drain still reaches the script through it. +func TestBackgroundWorkerHandleClosedAndFetchedAgain(t *testing.T) { + sentinel := filepath.Join(t.TempDir(), "refetch.txt") + + require.NoError(t, frankenphp.Init( + frankenphp.WithWorkers("bg-refetch", "testdata/bgworker/refetch.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(2), + )) + assert.Equal(t, "same then fresh", requireFileContentEventually(t, sentinel)) + + done := make(chan struct{}) + go func() { + frankenphp.Shutdown() + close(done) + }() + select { + case <-done: + case <-time.After(10 * time.Second): + t.Fatal("Shutdown did not return within 10s: the re-fetched handle missed the drain") + } } // TestBackgroundWorkerBootFailuresThenSucceeds checks that boot failures below diff --git a/testdata/bgworker/no-time-limit.php b/testdata/bgworker/no-time-limit.php index 476336c52d..c7356e1605 100644 --- a/testdata/bgworker/no-time-limit.php +++ b/testdata/bgworker/no-time-limit.php @@ -1,9 +1,10 @@ Date: Sun, 13 Sep 2026 12:50:15 +0200 Subject: [PATCH 08/20] docs: flag frankenphp_get_worker_handle() experimental in the stub The directive, the Go option and the docs section carry the flag, the function did not. --- frankenphp.stub.php | 14 +++++++------- frankenphp_arginfo.h | 4 ++-- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/frankenphp.stub.php b/frankenphp.stub.php index bf1587cc64..f935a48393 100644 --- a/frankenphp.stub.php +++ b/frankenphp.stub.php @@ -56,13 +56,13 @@ function mercure_publish(string|array $topics, string $data = '', bool $private function frankenphp_log(string $message, int $level = 0, array $context = []): void {} /** - * Returns a stop-signal stream for the current background worker. The - * stream reaches EOF when FrankenPHP drains the worker, so the script can - * park on stream_select() and exit its loop gracefully. Every call of a run - * returns the same stream, a fresh one over the same socket once the script - * closed it. The worker counts as ready, and its startup as successful, once - * it waits on the stream (stream_select() or a blocking read). Only callable - * from inside a background worker. + * EXPERIMENTAL: returns a stop-signal stream for the current background + * worker. The stream reaches EOF when FrankenPHP drains the worker, so the + * script can park on stream_select() and exit its loop gracefully. Every + * call of a run returns the same stream, a fresh one over the same socket + * once the script closed it. The worker counts as ready, and its startup as + * successful, once it waits on the stream (stream_select() or a blocking + * read). Only callable from inside a background worker. * * @return resource */ diff --git a/frankenphp_arginfo.h b/frankenphp_arginfo.h index 8223be08ba..b2b6fe1a36 100644 --- a/frankenphp_arginfo.h +++ b/frankenphp_arginfo.h @@ -1,5 +1,5 @@ -/* This is a generated file, edit frankenphp.stub.php instead. - * Stub hash: 2f36fc81e0981975adabf170febaaa863653817d */ +/* This is a generated file, edit the .stub.php file instead. + * Stub hash: 23bcea159c151578e7cb9458d2dd9595d8aab621 */ ZEND_BEGIN_ARG_WITH_RETURN_TYPE_INFO_EX(arginfo_frankenphp_handle_request, 0, 1, _IS_BOOL, 0) ZEND_ARG_TYPE_INFO(0, callback, IS_CALLABLE, 0) From 20a576233c6c91a952fee3b6b58f213fb4f835f7 Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Sun, 13 Sep 2026 22:36:15 +0200 Subject: [PATCH 09/20] feat: frankenphp_worker_tick(), the explicit ready point of background workers Readiness rode on intercepting every PHP path that waits on the handle: the read op, the select cast, the transport receive missed at first, and whatever a future PHP adds. The contract was also read three different ways during review. It is now a call the script makes, the background analog of frankenphp_handle_request(): the first frankenphp_worker_tick() of a run marks the worker ready, every call returns false once the worker is drained, and the read, cast and transport hooks are gone. The tick never blocks and never hands out work. It consumes whatever the runtime wrote on the handle to wake the script up, so the script only ever selects on the handle, alone or with its own streams, and the protocol on it stays private. --- bgworker_test.go | 33 ++++++- caddy/caddy_test.go | 12 +-- docs/config.md | 4 +- docs/metrics.md | 2 +- docs/worker.md | 15 +-- frankenphp.c | 118 ++++++++++++++---------- frankenphp.stub.php | 28 ++++-- frankenphp_arginfo.h | 6 +- metrics.go | 4 +- metrics_test.go | 2 +- testdata/bgworker/basic.php | 18 ++-- testdata/bgworker/count.php | 11 ++- testdata/bgworker/crash-after-ready.php | 10 +- testdata/bgworker/crash.php | 11 ++- testdata/bgworker/early-return.php | 6 +- testdata/bgworker/fail-then-succeed.php | 11 ++- testdata/bgworker/fetch-no-tick.php | 7 ++ testdata/bgworker/fetch-no-wait.php | 7 -- testdata/bgworker/flag.php | 11 ++- testdata/bgworker/named.php | 11 ++- testdata/bgworker/no-time-limit.php | 11 ++- testdata/bgworker/pool.php | 11 ++- testdata/bgworker/read.php | 7 +- testdata/bgworker/recv.php | 7 +- testdata/bgworker/refetch.php | 1 + testdata/bgworker/stuck.php | 14 +-- testdata/bgworker/tick.php | 17 ++++ testdata/handle-outside.php | 12 ++- threadbackgroundworker.go | 52 +++++------ 29 files changed, 270 insertions(+), 189 deletions(-) create mode 100644 testdata/bgworker/fetch-no-tick.php delete mode 100644 testdata/bgworker/fetch-no-wait.php create mode 100644 testdata/bgworker/tick.php diff --git a/bgworker_test.go b/bgworker_test.go index 7f8e8bbdc0..37dc5ce4ae 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -180,18 +180,18 @@ func TestBackgroundWorkerValidation(t *testing.T) { ), frankenphp.WithNumThreads(2), ) - require.ErrorContains(t, err, "frankenphp_get_worker_handle") + require.ErrorContains(t, err, "without calling frankenphp_worker_tick()") }) - t.Run("fetching the handle without waiting on it fails startup", func(t *testing.T) { + t.Run("fetching the handle without ticking fails startup", func(t *testing.T) { err := frankenphp.Init( - frankenphp.WithWorkers("bg-no-wait", "testdata/bgworker/fetch-no-wait.php", 1, + frankenphp.WithWorkers("bg-no-tick", "testdata/bgworker/fetch-no-tick.php", 1, frankenphp.WithWorkerBackground(), frankenphp.WithWorkerMaxFailures(2), ), frankenphp.WithNumThreads(2), ) - require.ErrorContains(t, err, "waiting on its handle") + require.ErrorContains(t, err, "without calling frankenphp_worker_tick()") }) t.Run("max_threads is rejected", func(t *testing.T) { @@ -352,7 +352,30 @@ func TestGetWorkerHandleOutsideBackgroundWorker(t *testing.T) { body := serverGet(t, server, "http://example.com/handle-outside.php") - assert.Contains(t, body, "can only be called from a background worker") + assert.Contains(t, body, "frankenphp_get_worker_handle() can only be called from a background worker") + assert.Contains(t, body, "frankenphp_worker_tick() can only be called from a background worker") +} + +// TestBackgroundWorkerTick checks the contract of frankenphp_worker_tick(): +// true while the worker runs, false once it is drained, and still false on +// the next call +func TestBackgroundWorkerTick(t *testing.T) { + sentinel := filepath.Join(t.TempDir(), "ticks.txt") + + require.NoError(t, frankenphp.Init( + frankenphp.WithWorkers("bg-tick", "testdata/bgworker/tick.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(2), + )) + assert.Equal(t, "true true", requireFileContentEventually(t, sentinel)) + + // Shutdown() waits for the script to leave, so the file is final after it + frankenphp.Shutdown() + b, err := os.ReadFile(sentinel) + require.NoError(t, err) + assert.Equal(t, "true true false false", string(b)) } // TestWorkerNameInServerVars checks that every worker sees its declared name diff --git a/caddy/caddy_test.go b/caddy/caddy_test.go index b53abfa9a6..43ac10513e 100644 --- a/caddy/caddy_test.go +++ b/caddy/caddy_test.go @@ -878,7 +878,7 @@ func TestWorkerMetrics(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="` + workerName + `"} 10 - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_worker_tick for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="` + workerName + `"} 2 ` @@ -1035,7 +1035,7 @@ func TestNamedWorkerMetrics(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="my_app"} 10 - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_worker_tick for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="my_app"} 2 ` @@ -1131,7 +1131,7 @@ func TestAutoWorkerConfig(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="` + workerName + `"} 10 - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_worker_tick for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="` + workerName + `"} ` + workers + ` ` @@ -1499,7 +1499,7 @@ func TestMultiWorkersMetrics(t *testing.T) { # TYPE frankenphp_worker_request_count counter frankenphp_worker_request_count{worker="service1"} 10 - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_worker_tick for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="service1"} 2 frankenphp_ready_workers{worker="service2"} 3 @@ -1653,7 +1653,7 @@ func TestWorkerRestart(t *testing.T) { // Check metrics expectedMetrics := ` - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_worker_tick for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="service"} 1 # HELP frankenphp_total_workers Total number of PHP workers for this worker @@ -1681,7 +1681,7 @@ func TestWorkerRestart(t *testing.T) { // frankenphp_ready_workers should be back to 1 even after worker restarts expectedMetrics = ` - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_worker_tick for background workers # TYPE frankenphp_ready_workers gauge frankenphp_ready_workers{worker="service"} 1 # HELP frankenphp_total_workers Total number of PHP workers for this worker diff --git a/docs/config.md b/docs/config.md index 1a009c49b7..193a591793 100644 --- a/docs/config.md +++ b/docs/config.md @@ -111,7 +111,7 @@ You can also explicitly configure FrankenPHP using the [global option](https://c watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. name # Sets the name of the worker, used in logs and metrics and exposed as $_SERVER['FRANKENPHP_WORKER']. Must be unique among global workers. Default: absolute path of the worker file. max_consecutive_failures # Sets the maximum number of consecutive failures before the worker is considered unhealthy, -1 means the worker will always restart. Default: 6. - background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it waits on the stream returned by frankenphp_get_worker_handle() (stream_select() or a blocking read); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] is set in it, test its presence rather than its value. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. + background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] is set in it, test its presence rather than its value. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. } } } @@ -199,7 +199,7 @@ php_server [] { watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. env # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables. Environment variables for this worker are also inherited from the php_server parent, but can be overwritten here. match # match the worker to a path pattern. Overrides try_files and can only be used in the php_server directive. - background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it waits on the stream returned by frankenphp_get_worker_handle() (stream_select() or a blocking read); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] is set in it, test its presence rather than its value. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. + background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] is set in it, test its presence rather than its value. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. } worker # Can also use the short form like in the global frankenphp block. } diff --git a/docs/metrics.md b/docs/metrics.md index 6b8c42b0fc..8def3085da 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -19,7 +19,7 @@ When [Caddy metrics](https://caddyserver.com/docs/metrics) are enabled, FrankenP - `frankenphp_busy_workers{worker="[worker_name]"}`: The number of workers currently processing a request. - `frankenphp_worker_request_time{worker="[worker_name]"}`: The time spent processing requests by all workers. - `frankenphp_worker_request_count{worker="[worker_name]"}`: The number of requests processed by all workers. -- `frankenphp_ready_workers{worker="[worker_name]"}`: The number of workers that have reached their ready point at least once: `frankenphp_handle_request()` for HTTP workers, the first wait on the stream of `frankenphp_get_worker_handle()` for background workers. +- `frankenphp_ready_workers{worker="[worker_name]"}`: The number of workers that have reached their ready point at least once: `frankenphp_handle_request()` for HTTP workers, `frankenphp_worker_tick()` for background workers. - `frankenphp_worker_crashes{worker="[worker_name]"}`: The number of times a worker has unexpectedly terminated. - `frankenphp_worker_restarts{worker="[worker_name]"}`: The number of times a worker has been deliberately restarted. - `frankenphp_worker_queue_depth{worker="[worker_name]"}`: The number of queued requests. diff --git a/docs/worker.md b/docs/worker.md index b9fe70674a..c2e83b66d6 100644 --- a/docs/worker.md +++ b/docs/worker.md @@ -216,25 +216,26 @@ php_server { } ``` -The script must wait on the stream returned by `frankenphp_get_worker_handle()`, which reaches EOF when FrankenPHP drains the worker on shutdown, reboot or restart. The first wait on it marks the worker ready: the server start waits for that point, and an exit before it counts as a failure. Polling `feof()` is not a wait, block in `stream_select()` or in a read: +The script calls `frankenphp_worker_tick()` once it is set up. The first call marks the worker ready: the server start waits for that point, and an exit before it counts as a failure. Every call returns `false` once FrankenPHP drains the worker, on shutdown, reboot or restart, and `true` otherwise. It never blocks and never hands out work: like `frankenphp_handle_request()`, it is where the runtime and the script meet. Between two calls, the script waits on the stream returned by `frankenphp_get_worker_handle()`, alone or together with its own streams. The stream becomes readable when FrankenPHP needs the script's attention, and `frankenphp_worker_tick()` consumes whatever was written on it, so the script does not read the stream itself. ```php 0) { - // drained: return, FrankenPHP re-runs or stops the script - break; - } + stream_select($read, $write, $except, 1); doSomeWork(); } + +// drained: return, FrankenPHP re-runs or stops the script ``` +With an event loop, register the stream as readable and call `frankenphp_worker_tick()` from the callback, stopping the loop when it returns `false`. + `$_SERVER['FRANKENPHP_WORKER']` holds the declared name, as it does in HTTP workers, and `$_SERVER['FRANKENPHP_WORKER_BACKGROUND']` is set, so a script serving both roles can tell them apart with `isset()`. Its value is unspecified, only its presence is part of the contract. `FRANKENPHP_WORKER` used to hold `1` in HTTP workers for the same reason: a script comparing it to that value must test its presence instead. Background threads come on top of `num_threads` and `max_threads`; they don't autoscale, so `max_threads` is not allowed on them. From Go, declare one with `WithWorkerBackground()`. ## Superglobals behavior diff --git a/frankenphp.c b/frankenphp.c index fa186c101c..d0550e82a4 100644 --- a/frankenphp.c +++ b/frankenphp.c @@ -135,9 +135,9 @@ static THREAD_LOCAL bool is_background_worker = false; * exposed via frankenphp_get_worker_handle(); [1] is transferred to the Go * side, which closes it to signal a drain. */ static THREAD_LOCAL php_socket_t worker_stop_socks[2] = {SOCK_ERR, SOCK_ERR}; -/* set on the first wait on the handle of the current run, see - * frankenphp_worker_handle_ops */ -static THREAD_LOCAL bool worker_handle_waited = false; +/* set by the first frankenphp_worker_tick() of the current run, the ready + * point of a background worker */ +static THREAD_LOCAL bool worker_ticked = false; /* the stream of the current run, see frankenphp_get_worker_handle(); the * cache holds a ref, and the resource list of the run frees it at request * shutdown, so the pointer is only reset, never released, between runs */ @@ -440,7 +440,7 @@ static int frankenphp_worker_open_stop_pair(void) { * re-arms it, see php_thread(). */ intptr_t frankenphp_set_background_worker_and_get_stop_sock(void) { is_background_worker = true; - worker_handle_waited = false; + worker_ticked = false; worker_handle_res = NULL; frankenphp_worker_close_stop_socks(); @@ -1177,51 +1177,11 @@ PHP_FUNCTION(frankenphp_log) { } /* Ops of the streams returned by frankenphp_get_worker_handle(): the socket - * ops, except that the first wait on the handle, a select cast or a read, - * reports the worker ready, and that closing a stream leaves the socket - * alone: it belongs to the thread, every handle of a run shares it, and it - * is closed at the next run setup or on thread exit. Waiting is the - * background analog of an HTTP worker reaching frankenphp_handle_request(): - * it comes after the script's bootstrap by construction, where merely - * fetching the handle does not. Initialized in MINIT. */ + * ops, except that closing a stream leaves the socket alone: it belongs to + * the thread, every handle of a run shares it, and it is closed at the next + * run setup or on thread exit. Initialized in MINIT. */ static php_stream_ops frankenphp_worker_handle_ops; -static void frankenphp_worker_handle_waited(void) { - if (!worker_handle_waited) { - worker_handle_waited = true; - go_frankenphp_background_worker_ready(frankenphp_thread_index()); - } -} - -static ssize_t frankenphp_worker_handle_read(php_stream *stream, char *buf, - size_t count) { - frankenphp_worker_handle_waited(); - - return php_stream_socket_ops.read(stream, buf, count); -} - -static int frankenphp_worker_handle_cast(php_stream *stream, int castas, - void **ret) { - if (castas == PHP_STREAM_AS_FD_FOR_SELECT) { - frankenphp_worker_handle_waited(); - } - - return php_stream_socket_ops.cast(stream, castas, ret); -} - -/* stream_socket_recvfrom() and the other transport receives do not go - * through the read op, they reach the stream through its transport API, and - * a blocking receive is a wait on the handle too */ -static int frankenphp_worker_handle_set_option(php_stream *stream, int option, - int value, void *ptrparam) { - if (option == PHP_STREAM_OPTION_XPORT_API && ptrparam != NULL && - ((php_stream_xport_param *)ptrparam)->op == STREAM_XPORT_OP_RECV) { - frankenphp_worker_handle_waited(); - } - - return php_stream_socket_ops.set_option(stream, option, value, ptrparam); -} - static int frankenphp_worker_handle_close(php_stream *stream, int close_handle) { (void)close_handle; @@ -1275,7 +1235,6 @@ PHP_FUNCTION(frankenphp_get_worker_handle) { * default_socket_timeout wake-ups */ ((php_netstream_data_t *)stream->abstract)->timeout.tv_sec = -1; - /* report the worker ready on its first wait on the stream */ stream->ops = &frankenphp_worker_handle_ops; php_stream_to_zval(stream, return_value); @@ -1283,6 +1242,66 @@ PHP_FUNCTION(frankenphp_get_worker_handle) { GC_ADDREF(worker_handle_res); } +/* The ready point of a background worker and its liveness check, the + * background analog of frankenphp_handle_request(): the first call of a run + * reports the worker ready, and every call returns false once FrankenPHP + * drains it. It never blocks and never hands out work: the script waits on + * its handle, alone or with its own streams, and calls this when the handle + * is readable. Whatever the runtime writes on the handle to wake the script + * up is consumed here, so the script never has to read the handle and the + * protocol on it stays private. */ +PHP_FUNCTION(frankenphp_worker_tick) { + ZEND_PARSE_PARAMETERS_NONE(); + + if (!is_background_worker) { + zend_throw_exception(spl_ce_RuntimeException, + "frankenphp_worker_tick() can only be called from a " + "background worker", + 0); + RETURN_THROWS(); + } + + if (worker_stop_socks[0] == SOCK_ERR) { + zend_throw_exception(spl_ce_RuntimeException, + "the background worker stop socket is not available", + 0); + RETURN_THROWS(); + } + + if (!worker_ticked) { + worker_ticked = true; + go_frankenphp_background_worker_ready(frankenphp_thread_index()); + } + + /* consume the wake-ups without blocking; EOF is the drain */ + char buf[64]; + for (;;) { + if (php_pollfd_for_ms(worker_stop_socks[0], PHP_POLLREADABLE, 0) <= 0) { + /* nothing pending, or a transient poll error: still running */ + RETURN_TRUE; + } + +#ifdef PHP_WIN32 + int n = recv(worker_stop_socks[0], buf, (int)sizeof(buf), 0); +#else + ssize_t n = recv(worker_stop_socks[0], buf, sizeof(buf), 0); +#endif + if (n == 0) { + /* the Go side closed its end: drained */ + RETURN_FALSE; + } + if (n < 0) { + int err = php_socket_errno(); + if (err == EINTR || PHP_IS_TRANSIENT_ERROR(err)) { + RETURN_TRUE; + } + + /* a broken socket carries no drain anymore, stop the loop */ + RETURN_FALSE; + } + } +} + /* {{{ thread-safe opcache reset */ PHP_FUNCTION(frankenphp_opcache_reset) { go_schedule_opcache_reset(frankenphp_thread_index()); @@ -1343,10 +1362,7 @@ static const zend_function_entry frankenphp_test_hook_functions[] = { PHP_MINIT_FUNCTION(frankenphp) { frankenphp_worker_handle_ops = php_stream_socket_ops; frankenphp_worker_handle_ops.label = "FrankenPHP worker handle"; - frankenphp_worker_handle_ops.read = frankenphp_worker_handle_read; - frankenphp_worker_handle_ops.cast = frankenphp_worker_handle_cast; frankenphp_worker_handle_ops.close = frankenphp_worker_handle_close; - frankenphp_worker_handle_ops.set_option = frankenphp_worker_handle_set_option; register_frankenphp_symbols(module_number); #ifndef PHP_WIN32 diff --git a/frankenphp.stub.php b/frankenphp.stub.php index f935a48393..36bbd5fe7e 100644 --- a/frankenphp.stub.php +++ b/frankenphp.stub.php @@ -56,14 +56,28 @@ function mercure_publish(string|array $topics, string $data = '', bool $private function frankenphp_log(string $message, int $level = 0, array $context = []): void {} /** - * EXPERIMENTAL: returns a stop-signal stream for the current background - * worker. The stream reaches EOF when FrankenPHP drains the worker, so the - * script can park on stream_select() and exit its loop gracefully. Every - * call of a run returns the same stream, a fresh one over the same socket - * once the script closed it. The worker counts as ready, and its startup as - * successful, once it waits on the stream (stream_select() or a blocking - * read). Only callable from inside a background worker. + * EXPERIMENTAL: returns the handle of the current background worker, a + * stream to wait on, alone or with the script's own streams: it becomes + * readable when FrankenPHP needs the script's attention, its drain + * included, and frankenphp_worker_tick() then tells whether the worker + * still runs. Every call of a run returns the same stream, a fresh one over + * the same socket once the script closed it. Only callable from inside a + * background worker. * * @return resource */ function frankenphp_get_worker_handle() {} + +/** + * EXPERIMENTAL: the ready point and liveness check of a background worker, + * the background analog of frankenphp_handle_request(). The first call of a + * run marks the worker ready: the server start waits for it, and an exit + * before it counts as a failure. It returns false once FrankenPHP drains the + * worker, on shutdown, reboot or restart, so the script can leave its loop, + * and true otherwise. It never blocks and never hands out work: the script + * waits on the stream returned by frankenphp_get_worker_handle() and calls + * this when it is readable. Whatever FrankenPHP wrote on that stream is + * consumed here, the script does not have to read it. Only callable from + * inside a background worker. + */ +function frankenphp_worker_tick(): bool {} diff --git a/frankenphp_arginfo.h b/frankenphp_arginfo.h index b2b6fe1a36..75fd885c16 100644 --- a/frankenphp_arginfo.h +++ b/frankenphp_arginfo.h @@ -1,5 +1,5 @@ /* This is a generated file, edit the .stub.php file instead. - * Stub hash: 23bcea159c151578e7cb9458d2dd9595d8aab621 */ + * Stub hash: 0c68b8a074015c8d7ee667272181469768c2f9c2 */ ZEND_BEGIN_ARG_WITH_RETURN_TYPE_INFO_EX(arginfo_frankenphp_handle_request, 0, 1, _IS_BOOL, 0) ZEND_ARG_TYPE_INFO(0, callback, IS_CALLABLE, 0) @@ -44,6 +44,8 @@ ZEND_END_ARG_INFO() ZEND_BEGIN_ARG_INFO_EX(arginfo_frankenphp_get_worker_handle, 0, 0, 0) ZEND_END_ARG_INFO() +#define arginfo_frankenphp_worker_tick arginfo_frankenphp_finish_request + ZEND_FUNCTION(frankenphp_handle_request); ZEND_FUNCTION(headers_send); ZEND_FUNCTION(frankenphp_finish_request); @@ -52,6 +54,7 @@ ZEND_FUNCTION(frankenphp_response_headers); ZEND_FUNCTION(mercure_publish); ZEND_FUNCTION(frankenphp_log); ZEND_FUNCTION(frankenphp_get_worker_handle); +ZEND_FUNCTION(frankenphp_worker_tick); static const zend_function_entry ext_functions[] = { ZEND_FE(frankenphp_handle_request, arginfo_frankenphp_handle_request) @@ -66,6 +69,7 @@ static const zend_function_entry ext_functions[] = { ZEND_FE(mercure_publish, arginfo_mercure_publish) ZEND_FE(frankenphp_log, arginfo_frankenphp_log) ZEND_FE(frankenphp_get_worker_handle, arginfo_frankenphp_get_worker_handle) + ZEND_FE(frankenphp_worker_tick, arginfo_frankenphp_worker_tick) ZEND_FE_END }; diff --git a/metrics.go b/metrics.go index 03fe56b6c3..4011db0d41 100644 --- a/metrics.go +++ b/metrics.go @@ -11,7 +11,7 @@ import ( const ( StopReasonCrash = iota StopReasonRestart - StopReasonBootFailure // worker exited before reaching its ready point: frankenphp_handle_request, or the first wait on the handle for background workers + StopReasonBootFailure // worker exited before reaching its ready point: frankenphp_handle_request, or frankenphp_worker_tick for background workers ) type StopReason int @@ -177,7 +177,7 @@ func (m *PrometheusMetrics) TotalWorkers(string, int) { m.readyWorkers = prometheus.NewGaugeVec(prometheus.GaugeOpts{ Namespace: ns, Name: "ready_workers", - Help: "Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers", + Help: "Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_worker_tick for background workers", }, basicLabels) m.mustRegister(m.readyWorkers) } diff --git a/metrics_test.go b/metrics_test.go index 8c06a5686a..7d721f0189 100644 --- a/metrics_test.go +++ b/metrics_test.go @@ -185,7 +185,7 @@ func TestPrometheusMetrics_TestStopReasonCrash(t *testing.T) { name: "Testing ReadyWorkers", c: m.readyWorkers, metadata: ` - # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, the first wait on the stream of frankenphp_get_worker_handle for background workers + # HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_worker_tick for background workers # TYPE frankenphp_ready_workers gauge `, expect: ` diff --git a/testdata/bgworker/basic.php b/testdata/bgworker/basic.php index 2530c6ee58..9565cef551 100644 --- a/testdata/bgworker/basic.php +++ b/testdata/bgworker/basic.php @@ -1,7 +1,7 @@ $_SERVER['FRANKENPHP_WORKER'] ?? null, 'background' => isset($_SERVER['FRANKENPHP_WORKER_BACKGROUND']) ? 'set' : 'unset', ], true)); -$stream = frankenphp_get_worker_handle(); -$read = [$stream]; -$write = null; -$except = null; -stream_select($read, $write, $except, null); +$handle = frankenphp_get_worker_handle(); +while (frankenphp_worker_tick()) { + $read = [$handle]; + $write = $except = null; + stream_select($read, $write, $except, null); +} diff --git a/testdata/bgworker/named.php b/testdata/bgworker/named.php index 4b12f88961..435589ac47 100644 --- a/testdata/bgworker/named.php +++ b/testdata/bgworker/named.php @@ -12,8 +12,9 @@ @touch($_SERVER['BG_SENTINEL_DIR'] . DIRECTORY_SEPARATOR . $name); } -$stream = frankenphp_get_worker_handle(); -$read = [$stream]; -$write = null; -$except = null; -stream_select($read, $write, $except, null); +$handle = frankenphp_get_worker_handle(); +while (frankenphp_worker_tick()) { + $read = [$handle]; + $write = $except = null; + stream_select($read, $write, $except, null); +} diff --git a/testdata/bgworker/no-time-limit.php b/testdata/bgworker/no-time-limit.php index c7356e1605..d4228e6ef0 100644 --- a/testdata/bgworker/no-time-limit.php +++ b/testdata/bgworker/no-time-limit.php @@ -2,9 +2,10 @@ // Long-lived bg worker relying on the engine for its parking: it does not // call set_time_limit(0), so max_execution_time must not cut the run -// short, and it does not set a stream timeout, so default_socket_timeout -// must not end the read either. Counts its runs in BG_COUNT_FILE, so -// anything that interrupts the park shows up as a second line. +// short, and it parks in a blocking read without a stream timeout, so +// default_socket_timeout must not end the read either. Counts its runs in +// BG_COUNT_FILE, so anything that interrupts the park shows up as a second +// line. file_put_contents($_SERVER['BG_COUNT_FILE'], "run\n", FILE_APPEND); -$stream = frankenphp_get_worker_handle(); -fgets($stream); +frankenphp_worker_tick(); +fgets(frankenphp_get_worker_handle()); diff --git a/testdata/bgworker/pool.php b/testdata/bgworker/pool.php index 9b5a9096da..8446459393 100644 --- a/testdata/bgworker/pool.php +++ b/testdata/bgworker/pool.php @@ -4,8 +4,9 @@ // its own under BG_SENTINEL_DIR, then parks on its own handle. set_time_limit(0); @touch($_SERVER['BG_SENTINEL_DIR'] . DIRECTORY_SEPARATOR . bin2hex(random_bytes(8))); -$stream = frankenphp_get_worker_handle(); -$read = [$stream]; -$write = null; -$except = null; -stream_select($read, $write, $except, null); +$handle = frankenphp_get_worker_handle(); +while (frankenphp_worker_tick()) { + $read = [$handle]; + $write = $except = null; + stream_select($read, $write, $except, null); +} diff --git a/testdata/bgworker/read.php b/testdata/bgworker/read.php index c57e38b341..3ef3a66f4c 100644 --- a/testdata/bgworker/read.php +++ b/testdata/bgworker/read.php @@ -1,11 +1,10 @@ getMessage(); +foreach (['frankenphp_get_worker_handle', 'frankenphp_worker_tick'] as $function) { + try { + $function(); + echo "$function: no exception\n"; + } catch (\RuntimeException $e) { + echo $e->getMessage(), "\n"; + } } diff --git a/threadbackgroundworker.go b/threadbackgroundworker.go index 83b8db7b8d..89454305b5 100644 --- a/threadbackgroundworker.go +++ b/threadbackgroundworker.go @@ -18,8 +18,9 @@ import ( // it with a quadratic backoff when it crashes. Background workers share the // PHP runtime with HTTP threads but never receive HTTP requests. The script // can park on the stream returned by frankenphp_get_worker_handle(), which -// reaches EOF when the thread is drained, to exit gracefully on shutdown, -// reboot or handler transition. +// reaches EOF when the thread is drained, and frankenphp_worker_tick() then +// returns false, so it exits gracefully on shutdown, reboot or handler +// transition. type backgroundWorkerThread struct { workerLifecycle @@ -31,15 +32,14 @@ type backgroundWorkerThread struct { // touched on the PHP thread. crashCount int - // isBootingScript is true until the current run waits on its handle, a - // stream_select() or a read on the stream returned by - // frankenphp_get_worker_handle(), the background analog of an HTTP - // worker reaching frankenphp_handle_request(). Only touched on the PHP - // thread (setup, the C callback during execution, teardown). + // isBootingScript is true until the current run calls + // frankenphp_worker_tick(), the background analog of an HTTP worker + // reaching frankenphp_handle_request(). Only touched on the PHP thread + // (setup, the C callback during execution, teardown). isBootingScript bool - // bootTimer warns when a run has not waited on its handle after - // backgroundBootWarnDelay; only touched on the PHP thread + // bootTimer warns when a run has not called frankenphp_worker_tick() + // after backgroundBootWarnDelay; only touched on the PHP thread bootTimer *time.Timer // stopSock holds the Go side's end of this thread's stop socket pair @@ -50,9 +50,9 @@ type backgroundWorkerThread struct { stopSock atomic.Int64 } -// backgroundBootWarnDelay is how long a run may go without waiting on its -// handle before a warning: Init() and Shutdown() wait for that point, so a -// script that never gets there hangs both silently +// backgroundBootWarnDelay is how long a run may go without calling +// frankenphp_worker_tick() before a warning: Init() and Shutdown() wait for +// that point, so a script that never gets there hangs both silently const backgroundBootWarnDelay = 10 * time.Second func convertToBackgroundWorkerThread(thread *phpThread, worker *worker) { @@ -145,7 +145,7 @@ func (handler *backgroundWorkerThread) setupScript() error { logger, ctx, name, threadIndex := fc.logger, fc.ctx, handler.worker.qualifiedName, handler.thread.threadIndex handler.bootTimer = time.AfterFunc(backgroundBootWarnDelay, func() { if logger.Enabled(ctx, slog.LevelWarn) { - logger.LogAttrs(ctx, slog.LevelWarn, "background worker has not waited on its handle yet, Init() and Shutdown() wait for it, see frankenphp_get_worker_handle()", slog.String("worker", name), slog.Int("thread", threadIndex)) + logger.LogAttrs(ctx, slog.LevelWarn, "background worker has not called frankenphp_worker_tick() yet, Init() and Shutdown() wait for it", slog.String("worker", name), slog.Int("thread", threadIndex)) } }) @@ -153,8 +153,8 @@ func (handler *backgroundWorkerThread) setupScript() error { fc.logger.LogAttrs(fc.ctx, slog.LevelDebug, "starting background worker", slog.String("worker", handler.worker.qualifiedName), slog.Int("thread", handler.thread.threadIndex)) } - // the thread stays in TransitionComplete until the script waits on its - // handle, see go_frankenphp_background_worker_ready + // the thread stays in TransitionComplete until the script calls + // frankenphp_worker_tick(), see go_frankenphp_background_worker_ready return nil } @@ -170,9 +170,9 @@ func (handler *backgroundWorkerThread) afterScriptExecution(exitStatus int) { handler.stopBootTimer() handler.state.MarkAsWaiting(false) - // cooperative exit: the script waited on its handle and returned cleanly, - // re-run it, unless the thread is being drained (beforeScriptExecution - // checks the state) + // cooperative exit: the script ticked and returned cleanly, re-run it, + // unless the thread is being drained (beforeScriptExecution checks the + // state) if exitStatus == 0 && !handler.isBootingScript { handler.crashCount = 0 metrics.StopWorker(worker.qualifiedName, StopReasonRestart) @@ -203,8 +203,8 @@ func (handler *backgroundWorkerThread) afterScriptExecution(exitStatus int) { return } - // boot failure: the script exited before waiting on its handle, a clean - // exit included, which would otherwise respawn in a tight loop. + // boot failure: the script exited before calling frankenphp_worker_tick(), + // a clean exit included, which would otherwise respawn in a tight loop. // StopReasonBootFailure skips the ready-gauge decrement, matching the // ReadyWorker call that never happened metrics.StopWorker(worker.qualifiedName, StopReasonBootFailure) @@ -217,7 +217,7 @@ func (handler *backgroundWorkerThread) afterScriptExecution(exitStatus int) { pastCap := worker.maxConsecutiveFailures >= 0 && handler.failureCount >= worker.maxConsecutiveFailures if pastCap && startupFailChan != nil && !watcherIsEnabled { if exitStatus == 0 { - startupFailChan <- fmt.Errorf("background worker %s exits without waiting on its handle, see frankenphp_get_worker_handle()", worker.fileName) + startupFailChan <- fmt.Errorf("background worker %s exits without calling frankenphp_worker_tick()", worker.fileName) } else { startupFailChan <- fmt.Errorf("too many consecutive failures: background worker %s keeps crashing", worker.fileName) } @@ -226,9 +226,9 @@ func (handler *backgroundWorkerThread) afterScriptExecution(exitStatus int) { } logLevel := slog.LevelWarn - logMsg := "background worker failed before waiting on its handle, restarting" + logMsg := "background worker failed before calling frankenphp_worker_tick(), restarting" if exitStatus == 0 { - logMsg = "background worker exited without waiting on its handle, restarting" + logMsg = "background worker exited without calling frankenphp_worker_tick(), restarting" } if pastCap { logLevel = slog.LevelError @@ -250,8 +250,8 @@ func (handler *backgroundWorkerThread) stopBootTimer() { //export go_frankenphp_background_worker_ready func go_frankenphp_background_worker_ready(threadIndex C.uintptr_t) { - // called on the PHP thread on the first wait on the handle; the handler - // is a backgroundWorkerThread because frankenphp_get_worker_handle() + // called on the PHP thread by the first frankenphp_worker_tick() of a + // run; the handler is a backgroundWorkerThread because that function // throws on every other thread kind if handler, ok := phpThreads[threadIndex].handler.(*backgroundWorkerThread); ok && handler.isBootingScript { handler.isBootingScript = false @@ -264,7 +264,7 @@ func go_frankenphp_background_worker_ready(threadIndex C.uintptr_t) { // like an HTTP worker reaching frankenphp_handle_request(), the thread // is ready only now: initWorkers() waits for this state, so a script - // that fails before waiting on its handle still fails Init() + // that fails before its first tick still fails Init() if handler.state.Is(state.TransitionComplete) { handler.state.Set(state.Ready) } From 2348eba64c075213ac344363cee65f2f3b910425 Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Mon, 14 Sep 2026 07:16:37 +0200 Subject: [PATCH 10/20] feat: FRANKENPHP_WORKER_BACKGROUND holds the name, HTTP workers untouched FRANKENPHP_WORKER stays what it always was, "1" in HTTP workers, and is not set in background workers, where FRANKENPHP_WORKER_BACKGROUND holds the declared name instead. A script serving both roles tests which of the two is set. The removal of inherited values is gone with the change that motivated it: nothing about HTTP workers moves in this PR anymore. --- bgworker_test.go | 29 +++++++++++------------------ docs/config.md | 8 ++++---- docs/worker.md | 2 +- frankenphp.c | 15 --------------- testdata/bgworker/flag.php | 6 +++--- testdata/bgworker/named.php | 6 +++--- testdata/worker-name.php | 4 ++-- worker.go | 16 ++++++---------- 8 files changed, 30 insertions(+), 56 deletions(-) diff --git a/bgworker_test.go b/bgworker_test.go index 37dc5ce4ae..4ebcf2740d 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -95,8 +95,8 @@ func TestBackgroundWorkerCrashRestarts(t *testing.T) { // TestBackgroundWorkerOnServer scopes a background worker to a Server. It // proves that the worker inherits the server env (the sentinel directory is -// declared on the server, not on the worker), that FRANKENPHP_WORKER holds -// the worker name, and that the worker does not intercept HTTP requests +// declared on the server, not on the worker), that FRANKENPHP_WORKER_BACKGROUND +// holds the worker name, and that the worker does not intercept HTTP requests // served by the same server. func TestBackgroundWorkerOnServer(t *testing.T) { tmp := t.TempDir() @@ -123,7 +123,7 @@ func TestBackgroundWorkerOnServer(t *testing.T) { frankenphp.WithNumThreads(3), ) - // named.php touches "/": the script sees + // named.php touches "/": the script sees // the declared name, not the server-qualified one used by metrics and logs requireFileEventually(t, filepath.Join(tmp, "jobs"), "background worker did not touch its per-name sentinel") requireFileEventually(t, globalSentinel, "the global worker sharing the name did not start") @@ -378,23 +378,16 @@ func TestBackgroundWorkerTick(t *testing.T) { assert.Equal(t, "true true false false", string(b)) } -// TestWorkerNameInServerVars checks that every worker sees its declared name -// in FRANKENPHP_WORKER and that only background workers get the -// FRANKENPHP_WORKER_BACKGROUND flag. +// TestWorkerNameInServerVars checks that an HTTP worker sees FRANKENPHP_WORKER +// as it always did, and that a background worker sees its declared name in +// FRANKENPHP_WORKER_BACKGROUND and no FRANKENPHP_WORKER. func TestWorkerNameInServerVars(t *testing.T) { sentinel := filepath.Join(t.TempDir(), "flag.txt") - t.Setenv("FRANKENPHP_WORKER_BACKGROUND", "1") - server, err := frankenphp.NewServer(testDataDir, frankenphp.WithServerEnv(map[string]string{"FRANKENPHP_WORKER_BACKGROUND": "1"})) + server, err := frankenphp.NewServer(testDataDir) require.NoError(t, err) initServers(t, frankenphp.WithServer(server), - // both names are reserved: neither the worker env here, nor the - // server env, nor the process environment set below may make an - // HTTP worker look like a background one - frankenphp.WithWorkers("web", testDataDir+"worker-name.php", 1, - frankenphp.WithWorkerServerScope(server), - frankenphp.WithWorkerEnv(map[string]string{"FRANKENPHP_WORKER_BACKGROUND": "1"}), - ), + frankenphp.WithWorkers("web", testDataDir+"worker-name.php", 1, frankenphp.WithWorkerServerScope(server)), frankenphp.WithWorkers("jobs", "testdata/bgworker/flag.php", 1, frankenphp.WithWorkerBackground(), frankenphp.WithWorkerServerScope(server), @@ -403,11 +396,11 @@ func TestWorkerNameInServerVars(t *testing.T) { frankenphp.WithNumThreads(3), ) - assert.Equal(t, "web http", serverGet(t, server, "http://example.com/worker-name.php")) + assert.Equal(t, "1 http", serverGet(t, server, "http://example.com/worker-name.php")) flag := requireFileContentEventually(t, sentinel) - assert.Contains(t, flag, "'worker' => 'jobs'") - assert.Contains(t, flag, "'background' => 'set'") + assert.Contains(t, flag, "'worker' => 'unset'") + assert.Contains(t, flag, "'background' => 'jobs'") } // TestBackgroundWorkerPool checks that num > 1 threads share the name, each diff --git a/docs/config.md b/docs/config.md index 193a591793..9929ce22ea 100644 --- a/docs/config.md +++ b/docs/config.md @@ -109,9 +109,9 @@ You can also explicitly configure FrankenPHP using the [global option](https://c num # Sets the number of PHP threads to start, defaults to 2x the number of available CPUs. env # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables. watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. - name # Sets the name of the worker, used in logs and metrics and exposed as $_SERVER['FRANKENPHP_WORKER']. Must be unique among global workers. Default: absolute path of the worker file. + name # Sets the name of the worker, used in logs and metrics. Must be unique among global workers. Default: absolute path of the worker file. max_consecutive_failures # Sets the maximum number of consecutive failures before the worker is considered unhealthy, -1 means the worker will always restart. Default: 6. - background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] is set in it, test its presence rather than its value. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. + background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] holds its name, and FRANKENPHP_WORKER is not set in it. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. } } } @@ -195,11 +195,11 @@ php_server [] { worker { # Creates a worker specific to this server. Can be specified more than once for multiple workers. file # Sets the path to the worker script, can be relative to the php_server root num # Sets the number of PHP threads to start, defaults to 2x the number of available - name # Sets the name for the worker, used in logs and metrics and exposed as $_SERVER['FRANKENPHP_WORKER']. Must be unique within this php_server. In logs and metrics, the worker is reported as ":". Default: absolute path of the worker file. + name # Sets the name for the worker, used in logs and metrics. Must be unique within this php_server. In logs and metrics, the worker is reported as ":". Default: absolute path of the worker file. watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. env # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables. Environment variables for this worker are also inherited from the php_server parent, but can be overwritten here. match # match the worker to a path pattern. Overrides try_files and can only be used in the php_server directive. - background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] is set in it, test its presence rather than its value. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. + background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] holds its name, and FRANKENPHP_WORKER is not set in it. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. } worker # Can also use the short form like in the global frankenphp block. } diff --git a/docs/worker.md b/docs/worker.md index c2e83b66d6..22f37a85fb 100644 --- a/docs/worker.md +++ b/docs/worker.md @@ -236,7 +236,7 @@ while (frankenphp_worker_tick()) { With an event loop, register the stream as readable and call `frankenphp_worker_tick()` from the callback, stopping the loop when it returns `false`. -`$_SERVER['FRANKENPHP_WORKER']` holds the declared name, as it does in HTTP workers, and `$_SERVER['FRANKENPHP_WORKER_BACKGROUND']` is set, so a script serving both roles can tell them apart with `isset()`. Its value is unspecified, only its presence is part of the contract. `FRANKENPHP_WORKER` used to hold `1` in HTTP workers for the same reason: a script comparing it to that value must test its presence instead. Background threads come on top of `num_threads` and `max_threads`; they don't autoscale, so `max_threads` is not allowed on them. From Go, declare one with `WithWorkerBackground()`. +`$_SERVER['FRANKENPHP_WORKER_BACKGROUND']` holds the declared name. `FRANKENPHP_WORKER`, the variable of HTTP workers, is not set, so a script serving both roles tests which of the two is set. Background threads come on top of `num_threads` and `max_threads`; they don't autoscale, so `max_threads` is not allowed on them. From Go, declare one with `WithWorkerBackground()`. ## Superglobals behavior diff --git a/frankenphp.c b/frankenphp.c index d0550e82a4..37e729a29d 100644 --- a/frankenphp.c +++ b/frankenphp.c @@ -1673,21 +1673,6 @@ static void frankenphp_register_variables(zval *track_vars_array) { /* import environment and CGI variables from the request context in go */ go_register_server_variables(frankenphp_thread_index(), track_vars_array); - /* FRANKENPHP_WORKER and FRANKENPHP_WORKER_BACKGROUND say what is running - * the script, so FrankenPHP owns them: a value inherited from the process - * environment, from a php_server or from a worker would otherwise make a - * script take the wrong branch. The worker's own values were merged above, - * the layers below are dropped here. */ - if (!is_worker_thread) { - zend_hash_str_del(Z_ARRVAL_P(track_vars_array), "FRANKENPHP_WORKER", - sizeof("FRANKENPHP_WORKER") - 1); - } - if (!is_background_worker) { - zend_hash_str_del(Z_ARRVAL_P(track_vars_array), - "FRANKENPHP_WORKER_BACKGROUND", - sizeof("FRANKENPHP_WORKER_BACKGROUND") - 1); - } - /* Some variables are already present in SG(request_info) */ frankenphp_register_variables_from_request_info(track_vars_array); } diff --git a/testdata/bgworker/flag.php b/testdata/bgworker/flag.php index d9303d15af..24eb70a85f 100644 --- a/testdata/bgworker/flag.php +++ b/testdata/bgworker/flag.php @@ -1,11 +1,11 @@ $_SERVER['FRANKENPHP_WORKER'] ?? null, - 'background' => isset($_SERVER['FRANKENPHP_WORKER_BACKGROUND']) ? 'set' : 'unset', + 'worker' => $_SERVER['FRANKENPHP_WORKER'] ?? 'unset', + 'background' => $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] ?? 'unset', ], true)); $handle = frankenphp_get_worker_handle(); while (frankenphp_worker_tick()) { diff --git a/testdata/bgworker/named.php b/testdata/bgworker/named.php index 435589ac47..70bbfaa88c 100644 --- a/testdata/bgworker/named.php +++ b/testdata/bgworker/named.php @@ -2,12 +2,12 @@ // Long-lived bg worker that touches a per-name sentinel under // $_SERVER['BG_SENTINEL_DIR'] so tests can confirm the right instance -// ran. The bg worker's $_SERVER['FRANKENPHP_WORKER'] value is the -// declared name, so the same fixture serves multiple distinct names +// ran. The bg worker's $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] value is +// the declared name, so the same fixture serves multiple distinct names // across scopes. set_time_limit(0); -$name = $_SERVER['FRANKENPHP_WORKER'] ?? 'unknown'; +$name = $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] ?? 'unknown'; if (!empty($_SERVER['BG_SENTINEL_DIR'])) { @touch($_SERVER['BG_SENTINEL_DIR'] . DIRECTORY_SEPARATOR . $name); } diff --git a/testdata/worker-name.php b/testdata/worker-name.php index b9760c8dfc..b9f15a047d 100644 --- a/testdata/worker-name.php +++ b/testdata/worker-name.php @@ -1,7 +1,7 @@ Date: Mon, 14 Sep 2026 07:16:58 +0200 Subject: [PATCH 11/20] feat: wake a background worker once at start A script that registers its handle with an event loop and runs it only ticks when the handle is readable, so it never became ready before the drain and Init() waited for it. One wake-up written at run setup makes such a loop tick on its own: readiness then means the loop serviced the handle once. The first frankenphp_worker_tick() consumes it, and a script parked in a blocking read without ticking now fails its boot fast instead of hanging the start. --- bgworker_test.go | 16 ++++++++++++++++ docs/worker.md | 2 +- frankenphp.c | 11 +++++++++++ testdata/bgworker/loop.php | 15 +++++++++++++++ 4 files changed, 43 insertions(+), 1 deletion(-) create mode 100644 testdata/bgworker/loop.php diff --git a/bgworker_test.go b/bgworker_test.go index 4ebcf2740d..df9dfd1c40 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -356,6 +356,22 @@ func TestGetWorkerHandleOutsideBackgroundWorker(t *testing.T) { assert.Contains(t, body, "frankenphp_worker_tick() can only be called from a background worker") } +// TestBackgroundWorkerLoopTicksOnItsOwn checks the wake-up sent at start: a +// script that only ticks when its handle is readable, the shape of a script +// driven by an event loop, becomes ready without an explicit first call. +// Without the wake-up, Init() would wait for that call until the drain. +func TestBackgroundWorkerLoopTicksOnItsOwn(t *testing.T) { + sentinel := filepath.Join(t.TempDir(), "loop.sentinel") + initServers(t, + frankenphp.WithWorkers("bg-loop", "testdata/bgworker/loop.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(2), + ) + requireFileEventually(t, sentinel, "background worker did not start") +} + // TestBackgroundWorkerTick checks the contract of frankenphp_worker_tick(): // true while the worker runs, false once it is drained, and still false on // the next call diff --git a/docs/worker.md b/docs/worker.md index 22f37a85fb..f3f6b5289d 100644 --- a/docs/worker.md +++ b/docs/worker.md @@ -216,7 +216,7 @@ php_server { } ``` -The script calls `frankenphp_worker_tick()` once it is set up. The first call marks the worker ready: the server start waits for that point, and an exit before it counts as a failure. Every call returns `false` once FrankenPHP drains the worker, on shutdown, reboot or restart, and `true` otherwise. It never blocks and never hands out work: like `frankenphp_handle_request()`, it is where the runtime and the script meet. Between two calls, the script waits on the stream returned by `frankenphp_get_worker_handle()`, alone or together with its own streams. The stream becomes readable when FrankenPHP needs the script's attention, and `frankenphp_worker_tick()` consumes whatever was written on it, so the script does not read the stream itself. +The script calls `frankenphp_worker_tick()` once it is set up. The first call marks the worker ready: the server start waits for that point, and an exit before it counts as a failure. Every call returns `false` once FrankenPHP drains the worker, on shutdown, reboot or restart, and `true` otherwise. It never blocks and never hands out work: like `frankenphp_handle_request()`, it is where the runtime and the script meet. Between two calls, the script waits on the stream returned by `frankenphp_get_worker_handle()`, alone or together with its own streams. The stream becomes readable when FrankenPHP needs the script's attention, and `frankenphp_worker_tick()` consumes whatever was written on it, so the script does not read the stream itself. It is readable once right after the script starts, so a loop that services it ticks by itself and the worker is ready as soon as its loop runs. ```php Date: Mon, 14 Sep 2026 07:16:58 +0200 Subject: [PATCH 12/20] docs: event loop example for background workers --- docs/worker.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/docs/worker.md b/docs/worker.md index f3f6b5289d..9836b05422 100644 --- a/docs/worker.md +++ b/docs/worker.md @@ -234,7 +234,27 @@ while (frankenphp_worker_tick()) { // drained: return, FrankenPHP re-runs or stops the script ``` -With an event loop, register the stream as readable and call `frankenphp_worker_tick()` from the callback, stopping the loop when it returns `false`. +With an event loop, register the stream as readable and call `frankenphp_worker_tick()` from the callback. With [Revolt](https://revolt.run), the loop of amphp: + +```php + Date: Mon, 14 Sep 2026 07:17:13 +0200 Subject: [PATCH 13/20] refactor: one lifecycle abstraction instead of two The shared lifecycle keeps its struct, embedded by both worker handlers; the interface that named the one step they supply is gone, that step is a parameter. --- threadbackgroundworker.go | 2 +- threadworker.go | 2 +- workerlifecycle.go | 22 +++++++--------------- 3 files changed, 9 insertions(+), 17 deletions(-) diff --git a/threadbackgroundworker.go b/threadbackgroundworker.go index 89454305b5..e790bd11b6 100644 --- a/threadbackgroundworker.go +++ b/threadbackgroundworker.go @@ -81,7 +81,7 @@ func (handler *backgroundWorkerThread) drain() { } func (handler *backgroundWorkerThread) beforeScriptExecution() string { - return handler.workerLifecycle.beforeScriptExecution(handler) + return handler.workerLifecycle.beforeScriptExecution(handler.startScript) } // startScript keeps trying to start the script: unlike an HTTP worker, whose diff --git a/threadworker.go b/threadworker.go index f1808ff323..e57b117b67 100644 --- a/threadworker.go +++ b/threadworker.go @@ -30,7 +30,7 @@ func convertToWorkerThread(thread *phpThread, worker *worker) { } func (handler *workerThread) beforeScriptExecution() string { - return handler.workerLifecycle.beforeScriptExecution(handler) + return handler.workerLifecycle.beforeScriptExecution(handler.startScript) } // startScript runs the worker script; it always has one to run diff --git a/workerlifecycle.go b/workerlifecycle.go index a0b0906f1f..4d1809ee22 100644 --- a/workerlifecycle.go +++ b/workerlifecycle.go @@ -4,30 +4,22 @@ import "github.com/dunglas/frankenphp/internal/state" // workerLifecycle is the part of a worker thread that HTTP and background // workers share: the thread, its worker, and the states both walk through -// between two runs of the script. Handlers embed it and supply what differs, -// see workerHandler. +// between two runs of the script. Handlers embed it and pass the one step +// that differs, how a run starts. type workerLifecycle struct { state *state.ThreadState thread *phpThread worker *worker } -// workerHandler is a threadHandler running a worker script, plus the step -// the shared lifecycle delegates -type workerHandler interface { - threadHandler - // startScript prepares a run and returns the script to execute, or an - // empty string to stop the thread - startScript() string -} - func newWorkerLifecycle(thread *phpThread, worker *worker) workerLifecycle { return workerLifecycle{state: thread.state, thread: thread, worker: worker} } // beforeScriptExecution returns the name of the script to run, or an empty -// string to stop the thread -func (l *workerLifecycle) beforeScriptExecution(handler workerHandler) string { +// string to stop the thread; startScript prepares a run and returns the +// script to execute, or an empty string to stop the thread +func (l *workerLifecycle) beforeScriptExecution(startScript func() string) string { switch l.state.Get() { case state.TransitionRequested: l.detach() @@ -39,13 +31,13 @@ func (l *workerLifecycle) beforeScriptExecution(handler workerHandler) string { l.worker.onThreadReady(l.thread.threadIndex) } - return handler.startScript() + return startScript() case state.Rebooting, state.ForceRebooting: return "" case state.RebootReady: l.state.Set(state.Ready) - return handler.beforeScriptExecution() + return l.beforeScriptExecution(startScript) case state.ShuttingDown: l.detach() From 9733fcf0122de81df8ed83e0d13b3af23af8d769 Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Mon, 14 Sep 2026 07:17:26 +0200 Subject: [PATCH 14/20] fix: never block or race on the startup failure channel The channel was set to nil once Init() had decided, read without synchronization by the handlers, and the send blocked on a buffer sized to the thread count. A background worker can tick, exit and fail its next boot while Init() finishes, which an HTTP worker cannot since it blocks once ready: that exit could race the nil write, or block its thread on a full buffer. The channel now stays, an atomic startup flag gates the sends, and the send never blocks. --- threadbackgroundworker.go | 16 +++++++++------- threadworker.go | 4 ++-- worker.go | 25 ++++++++++++++++++++++++- 3 files changed, 35 insertions(+), 10 deletions(-) diff --git a/threadbackgroundworker.go b/threadbackgroundworker.go index e790bd11b6..323f183d4e 100644 --- a/threadbackgroundworker.go +++ b/threadbackgroundworker.go @@ -99,8 +99,7 @@ func (handler *backgroundWorkerThread) startScript() string { // fail fast during startup so Init() surfaces the error to the // operator; past startup, back off and retry like a crash - if startupFailChan != nil { - startupFailChan <- err + if reportStartupFailure(err) { handler.thread.state.Set(state.ShuttingDown) return handler.beforeScriptExecution() @@ -215,14 +214,17 @@ func (handler *backgroundWorkerThread) afterScriptExecution(exitStatus int) { // restarting with a louder log line: silently giving up would leave // the server in a broken half-state with no clear way to recover. pastCap := worker.maxConsecutiveFailures >= 0 && handler.failureCount >= worker.maxConsecutiveFailures - if pastCap && startupFailChan != nil && !watcherIsEnabled { + if pastCap && !watcherIsEnabled { + var err error if exitStatus == 0 { - startupFailChan <- fmt.Errorf("background worker %s exits without calling frankenphp_worker_tick()", worker.fileName) + err = fmt.Errorf("background worker %s exits without calling frankenphp_worker_tick()", worker.fileName) } else { - startupFailChan <- fmt.Errorf("too many consecutive failures: background worker %s keeps crashing", worker.fileName) + err = fmt.Errorf("too many consecutive failures: background worker %s keeps crashing", worker.fileName) + } + if reportStartupFailure(err) { + handler.thread.state.Set(state.ShuttingDown) + return } - handler.thread.state.Set(state.ShuttingDown) - return } logLevel := slog.LevelWarn diff --git a/threadworker.go b/threadworker.go index e57b117b67..c37e1fb674 100644 --- a/threadworker.go +++ b/threadworker.go @@ -118,8 +118,8 @@ func tearDownWorkerScript(handler *workerThread, exitStatus int) { return } - if worker.maxConsecutiveFailures >= 0 && startupFailChan != nil && !watcherIsEnabled && handler.failureCount >= worker.maxConsecutiveFailures { - startupFailChan <- fmt.Errorf("too many consecutive failures: worker %s has not reached frankenphp_handle_request()", worker.fileName) + if worker.maxConsecutiveFailures >= 0 && !watcherIsEnabled && handler.failureCount >= worker.maxConsecutiveFailures && + reportStartupFailure(fmt.Errorf("too many consecutive failures: worker %s has not reached frankenphp_handle_request()", worker.fileName)) { handler.thread.state.Set(state.ShuttingDown) return } diff --git a/worker.go b/worker.go index 7cb72aceab..44e4a020f1 100644 --- a/worker.go +++ b/worker.go @@ -47,6 +47,10 @@ var ( workers []*worker watcherIsEnabled bool startupFailChan chan error + // startupPhase is true while initWorkers() waits for the workers to + // boot, the only time a boot failure must reach startupFailChan: past + // that point the handlers log and keep restarting on their own + startupPhase atomic.Bool ) func initWorkers(opts []workerOpt) error { @@ -92,6 +96,7 @@ func initWorkers(opts []workerOpt) error { } startupFailChan = make(chan error, totalThreadsToStart) + startupPhase.Store(true) for _, w := range workers { for range w.num { @@ -109,6 +114,7 @@ func initWorkers(opts []workerOpt) error { } workersReady.Wait() + startupPhase.Store(false) select { case err := <-startupFailChan: @@ -116,12 +122,29 @@ func initWorkers(opts []workerOpt) error { return fmt.Errorf("failed to initialize workers: %w", err) default: // all workers started successfully - startupFailChan = nil } return nil } +// reportStartupFailure hands a boot failure to initWorkers() while it waits +// for the workers, so Init() fails, and reports whether it did: past that +// point the failure is dropped, the handler has logged it and keeps +// restarting. It never blocks, the buffer holds one error per thread and a +// thread failing repeatedly in the startup window must not hang on it +func reportStartupFailure(err error) bool { + if !startupPhase.Load() { + return false + } + + select { + case startupFailChan <- err: + default: + } + + return true +} + func newWorker(o workerOpt) (*worker, error) { // Order is important! // This order ensures that FrankenPHP started from inside a symlinked directory will properly resolve any paths. From b9e4dd973fd22030c25ca1d4364ca4121da0becc Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Mon, 14 Sep 2026 07:30:23 +0200 Subject: [PATCH 15/20] feat: max_execution_time bounds the bootstrap of a background worker Until its first frankenphp_worker_tick(), a run is under the limit like any request: a setup that outlives it ends as a boot failure, with the backoff and the cap. The first tick disarms the timer, and nothing re-arms it past that point, so the loop has no time limit, like the CLI. This replaces the per-request ini override, which exempted the bootstrap too. --- bgworker_test.go | 36 +++++++++++++++++++++++++++++---- docs/config.md | 4 ++-- docs/worker.md | 2 +- frankenphp.c | 24 +++++----------------- frankenphp.stub.php | 3 ++- frankenphp_arginfo.h | 2 +- testdata/bgworker/slow-boot.php | 11 ++++++++++ 7 files changed, 54 insertions(+), 28 deletions(-) create mode 100644 testdata/bgworker/slow-boot.php diff --git a/bgworker_test.go b/bgworker_test.go index df9dfd1c40..fedaf5f372 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -512,11 +512,39 @@ func TestBackgroundWorkerThreadsComeOnTopOfAutoMaxThreads(t *testing.T) { assert.Equal(t, 6, len(state.ThreadDebugStates)+state.ReservedThreadCount) } +// TestBackgroundWorkerBootstrapIsBounded checks that max_execution_time +// applies until the first frankenphp_worker_tick(): a setup that outlives +// it is ended as a boot failure, which fails Init() past the cap. +func TestBackgroundWorkerBootstrapIsBounded(t *testing.T) { + if !frankenphp.Config().ZendMaxExecutionTimers && runtime.GOOS != "windows" { + t.Skip("max_execution_time needs Zend max execution timers on this platform") + } + + countFile := filepath.Join(t.TempDir(), "boots") + err := frankenphp.Init( + frankenphp.WithWorkers("bg-slow", "testdata/bgworker/slow-boot.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerMaxFailures(0), + frankenphp.WithWorkerEnv(map[string]string{"BG_COUNT_FILE": countFile}), + ), + frankenphp.WithNumThreads(2), + frankenphp.WithPhpIni(map[string]string{"max_execution_time": "1"}), + ) + if err == nil { + frankenphp.Shutdown() + } + require.ErrorContains(t, err, "keeps crashing") + + b, _ := os.ReadFile(countFile) + assert.Equal(t, 1, bytes.Count(b, []byte("\n")), "the limit should have ended the one and only boot") +} + // TestBackgroundWorkerParkingIsNotInterrupted checks that a script parked -// on its handle is not cut short by the two limits it never disables -// itself: max_execution_time, which php_execute_script() re-arms from the -// ini when max_input_time is set, and default_socket_timeout, which the -// handle overrides with an infinite read timeout. +// on its handle, past its first tick, is not cut short by the two limits it +// never disables itself: max_execution_time, which the first tick disarms +// after php_execute_script() re-armed it from the ini, and +// default_socket_timeout, which the handle overrides with an infinite read +// timeout. func TestBackgroundWorkerParkingIsNotInterrupted(t *testing.T) { countFile := filepath.Join(t.TempDir(), "runs") initServers(t, diff --git a/docs/config.md b/docs/config.md index 9929ce22ea..5c055400b0 100644 --- a/docs/config.md +++ b/docs/config.md @@ -111,7 +111,7 @@ You can also explicitly configure FrankenPHP using the [global option](https://c watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. name # Sets the name of the worker, used in logs and metrics. Must be unique among global workers. Default: absolute path of the worker file. max_consecutive_failures # Sets the maximum number of consecutive failures before the worker is considered unhealthy, -1 means the worker will always restart. Default: 6. - background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] holds its name, and FRANKENPHP_WORKER is not set in it. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. + background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure, and max_execution_time applies until that call, not after. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] holds its name, and FRANKENPHP_WORKER is not set in it. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. } } } @@ -199,7 +199,7 @@ php_server [] { watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. env # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables. Environment variables for this worker are also inherited from the php_server parent, but can be overwritten here. match # match the worker to a path pattern. Overrides try_files and can only be used in the php_server directive. - background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] holds its name, and FRANKENPHP_WORKER is not set in it. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. + background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure, and max_execution_time applies until that call, not after. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] holds its name, and FRANKENPHP_WORKER is not set in it. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. } worker # Can also use the short form like in the global frankenphp block. } diff --git a/docs/worker.md b/docs/worker.md index 9836b05422..67d803a98b 100644 --- a/docs/worker.md +++ b/docs/worker.md @@ -216,7 +216,7 @@ php_server { } ``` -The script calls `frankenphp_worker_tick()` once it is set up. The first call marks the worker ready: the server start waits for that point, and an exit before it counts as a failure. Every call returns `false` once FrankenPHP drains the worker, on shutdown, reboot or restart, and `true` otherwise. It never blocks and never hands out work: like `frankenphp_handle_request()`, it is where the runtime and the script meet. Between two calls, the script waits on the stream returned by `frankenphp_get_worker_handle()`, alone or together with its own streams. The stream becomes readable when FrankenPHP needs the script's attention, and `frankenphp_worker_tick()` consumes whatever was written on it, so the script does not read the stream itself. It is readable once right after the script starts, so a loop that services it ticks by itself and the worker is ready as soon as its loop runs. +The script calls `frankenphp_worker_tick()` once it is set up. The first call marks the worker ready: the server start waits for that point, and an exit before it counts as a failure. Until that call, `max_execution_time` applies as in any request, 30 seconds by default: a setup that outlives it is ended and counts as a boot failure, so raise the limit in `php_ini`, or call `set_time_limit()` in the script, when the setup legitimately takes longer. From the first call on, the run has no time limit, like the CLI. Every call returns `false` once FrankenPHP drains the worker, on shutdown, reboot or restart, and `true` otherwise. It never blocks and never hands out work: like `frankenphp_handle_request()`, it is where the runtime and the script meet. Between two calls, the script waits on the stream returned by `frankenphp_get_worker_handle()`, alone or together with its own streams. The stream becomes readable when FrankenPHP needs the script's attention, and `frankenphp_worker_tick()` consumes whatever was written on it, so the script does not read the stream itself. It is readable once right after the script starts, so a loop that services it ticks by itself and the worker is ready as soon as its loop runs. ```php Date: Mon, 14 Sep 2026 07:49:21 +0200 Subject: [PATCH 16/20] docs: stop the whole loop on drain in the Revolt example Cancelling the handle's watcher only removes that one callback, and run() keeps going while any other referenced watcher exists. A drained worker has to leave its loop, which is the driver's stop(). --- docs/worker.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/worker.md b/docs/worker.md index 67d803a98b..55d9bdcc12 100644 --- a/docs/worker.md +++ b/docs/worker.md @@ -242,10 +242,10 @@ With an event loop, register the stream as readable and call `frankenphp_worker_ use Revolt\EventLoop; $handle = frankenphp_get_worker_handle(); -EventLoop::onReadable($handle, function (string $id): void { +EventLoop::onReadable($handle, function (): void { if (!frankenphp_worker_tick()) { - // drained: the loop ends once nothing else is pending - EventLoop::cancel($id); + // drained: stop the loop, the script returns and FrankenPHP moves on + EventLoop::getDriver()->stop(); } }); From f30e49d60f2137342cbf5066157b89e2971d55a8 Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Mon, 14 Sep 2026 08:07:14 +0200 Subject: [PATCH 17/20] test: only assert the bounded bootstrap where the timers are known to fire The limit is PHP's: on Windows CI the busy bootstrap ran its full five seconds without the timer ending it, so the test now runs only with the Zend max execution timers of ZTS builds on Linux, where it passes. --- bgworker_test.go | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/bgworker_test.go b/bgworker_test.go index fedaf5f372..2a99f9448d 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -514,10 +514,12 @@ func TestBackgroundWorkerThreadsComeOnTopOfAutoMaxThreads(t *testing.T) { // TestBackgroundWorkerBootstrapIsBounded checks that max_execution_time // applies until the first frankenphp_worker_tick(): a setup that outlives -// it is ended as a boot failure, which fails Init() past the cap. +// it is ended as a boot failure, which fails Init() past the cap. The limit +// itself is PHP's, so the test only runs where its timers are known to +// fire under FrankenPHP, the max execution timers of ZTS builds on Linux. func TestBackgroundWorkerBootstrapIsBounded(t *testing.T) { - if !frankenphp.Config().ZendMaxExecutionTimers && runtime.GOOS != "windows" { - t.Skip("max_execution_time needs Zend max execution timers on this platform") + if !frankenphp.Config().ZendMaxExecutionTimers { + t.Skip("max_execution_time is only reliable with Zend max execution timers") } countFile := filepath.Join(t.TempDir(), "boots") From 8b22cdecef1a0096c5f9c4a44738e03810a32ddd Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Mon, 14 Sep 2026 21:18:36 +0200 Subject: [PATCH 18/20] feat: num defaults to one thread for background workers A background worker serves no requests, so it does not scale with the CPUs like an HTTP one and nearly every declaration wrote "num 1". It is now the default, and declaring "background" is enough; a pool still asks for the threads it wants. --- bgworker_test.go | 32 ++++++++++++++++++++++++-------- caddy/config_test.go | 6 +++--- caddy/workerconfig.go | 3 --- docs/config.md | 4 ++-- docs/worker.md | 2 +- frankenphp.go | 14 ++++++-------- 6 files changed, 36 insertions(+), 25 deletions(-) diff --git a/bgworker_test.go b/bgworker_test.go index 2a99f9448d..f395ff4c6f 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -7,6 +7,7 @@ import ( "os" "path/filepath" "runtime" + "strings" "testing" "time" @@ -144,14 +145,6 @@ func TestBackgroundWorkerValidation(t *testing.T) { require.ErrorContains(t, err, "must have an explicit name") }) - t.Run("num must be >= 1", func(t *testing.T) { - err := frankenphp.Init( - frankenphp.WithWorkers("bg-zero", "testdata/bgworker/basic.php", 0, frankenphp.WithWorkerBackground()), - frankenphp.WithNumThreads(2), - ) - require.ErrorContains(t, err, "must declare num >= 1") - }) - t.Run("names are unique within a server", func(t *testing.T) { // a global and a server-scoped worker may share a name (see // TestBackgroundWorkerOnServer), two workers of one server may not @@ -490,6 +483,29 @@ func TestBackgroundWorkerThreadsComeOnTop(t *testing.T) { requireFileEventually(t, sentinel, "background worker did not start with num_threads 1") } +// TestBackgroundWorkerDefaultsToOneThread checks that num is optional: a +// background worker serves no requests, so it gets one thread unless a +// pool is asked for, rather than the CPU count of an HTTP worker. +func TestBackgroundWorkerDefaultsToOneThread(t *testing.T) { + sentinel := filepath.Join(t.TempDir(), "bg-default.sentinel") + initServers(t, + frankenphp.WithWorkers("bg-default", "testdata/bgworker/basic.php", 0, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(1), + ) + requireFileEventually(t, sentinel, "background worker did not start without an explicit num") + + background := 0 + for _, thread := range frankenphp.DebugState().ThreadDebugStates { + if strings.Contains(thread.Name, "Background Worker") { + background++ + } + } + assert.Equal(t, 1, background) +} + // TestBackgroundWorkerThreadsComeOnTopOfAutoMaxThreads checks that the // automatic limit is an HTTP one too: the reservation is added to what it // resolves, instead of eating into it diff --git a/caddy/config_test.go b/caddy/config_test.go index d71b307bb4..933769fd78 100644 --- a/caddy/config_test.go +++ b/caddy/config_test.go @@ -323,7 +323,7 @@ func TestWorkerBackgroundRequiresName(t *testing.T) { require.ErrorContains(t, err, `background workers must have an explicit "name"`) } -func TestWorkerBackgroundRequiresNum(t *testing.T) { +func TestWorkerBackgroundWithoutNumParses(t *testing.T) { d := caddyfile.NewTestDispenser(` { php_server { @@ -336,8 +336,8 @@ func TestWorkerBackgroundRequiresNum(t *testing.T) { }`) module := &FrankenPHPModule{} - err := module.UnmarshalCaddyfile(d) - require.ErrorContains(t, err, `background workers must declare "num" >= 1`) + // num is optional, it defaults to one thread when the workers start + require.NoError(t, module.UnmarshalCaddyfile(d)) } func TestWorkerBackgroundRejectsMatch(t *testing.T) { diff --git a/caddy/workerconfig.go b/caddy/workerconfig.go index 7f5c15b253..326b99a045 100644 --- a/caddy/workerconfig.go +++ b/caddy/workerconfig.go @@ -159,9 +159,6 @@ func unmarshalWorker(d *caddyfile.Dispenser) (workerConfig, error) { if len(wc.MatchPath) != 0 { return wc, d.Err(`"match" is not supported for background workers`) } - if wc.Num < 1 { - return wc, d.Err(`background workers must declare "num" >= 1`) - } } if frankenphp.EmbeddedAppPath != "" && filepath.IsLocal(wc.FileName) { diff --git a/docs/config.md b/docs/config.md index 5c055400b0..f3f17ebee3 100644 --- a/docs/config.md +++ b/docs/config.md @@ -106,7 +106,7 @@ You can also explicitly configure FrankenPHP using the [global option](https://c php_ini # Set a php.ini directive. Can be used several times to set multiple directives. worker { file # Sets the path to the worker script. - num # Sets the number of PHP threads to start, defaults to 2x the number of available CPUs. + num # Sets the number of PHP threads to start, defaults to 2x the number of available CPUs, 1 for background workers. env # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables. watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. name # Sets the name of the worker, used in logs and metrics. Must be unique among global workers. Default: absolute path of the worker file. @@ -194,7 +194,7 @@ php_server [] { request_body_timeout # Sets an idle timeout on request body reads: a stalled (slow POST) client is cut off while a steady upload of any size succeeds. Default: 60s. Set to 0 to disable. worker { # Creates a worker specific to this server. Can be specified more than once for multiple workers. file # Sets the path to the worker script, can be relative to the php_server root - num # Sets the number of PHP threads to start, defaults to 2x the number of available + num # Sets the number of PHP threads to start, defaults to 2x the number of available CPUs, 1 for background workers. name # Sets the name for the worker, used in logs and metrics. Must be unique within this php_server. In logs and metrics, the worker is reported as ":". Default: absolute path of the worker file. watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. env # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables. Environment variables for this worker are also inherited from the php_server parent, but can be overwritten here. diff --git a/docs/worker.md b/docs/worker.md index 55d9bdcc12..09473d1f10 100644 --- a/docs/worker.md +++ b/docs/worker.md @@ -203,7 +203,7 @@ frankenphp { This feature is experimental. -A background worker runs its script in a loop outside the HTTP request cycle, on its own PHP thread. It is declared like any worker, with the `background` option; `name` is required and `num` must be at least 1: +A background worker runs its script in a loop outside the HTTP request cycle, on its own PHP thread. It is declared like any worker, with the `background` option; `name` is required and `num` defaults to one thread: ```caddyfile php_server { diff --git a/frankenphp.go b/frankenphp.go index b8928eed59..cbe951f822 100644 --- a/frankenphp.go +++ b/frankenphp.go @@ -181,15 +181,13 @@ func calculateMaxThreads(opt *opt) (numWorkers, backgroundThreads int, _ error) for i, w := range opt.workers { if w.isBackgroundWorker { - if w.num < 1 { - name := w.name - if name == "" { - name = w.fileName - } - - return 0, 0, fmt.Errorf("background worker %q must declare num >= 1", name) + if w.num <= 0 { + // one thread unless a pool is asked for: a background + // worker serves no requests, so it does not scale with + // the CPUs like an HTTP one + opt.workers[i].num = 1 } - backgroundThreads += w.num + backgroundThreads += opt.workers[i].num continue } From e7024a6e9bf42fbfcc2cb267eb435963a0be4904 Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Mon, 14 Sep 2026 21:18:36 +0200 Subject: [PATCH 19/20] chore: review polish The missing stop socket of a background worker is an invariant, not a runtime error: the pair is opened before the script starts and closed at the next run setup, so the two guards are asserts now. A thread reaching the ready callback without a background handler would wait out Init() silently, so it panics instead. The context of a background run is not a dummy request, and the field says so. The scope of a name collision is a local variable rather than a method, and the Caddyfile reference keeps the short version of the "background" line, the long one lives in the worker documentation. --- docs/config.md | 4 ++-- frankenphp.c | 16 ++++------------ server.go | 15 ++++++--------- threadbackgroundworker.go | 21 ++++++++++++++------- 4 files changed, 26 insertions(+), 30 deletions(-) diff --git a/docs/config.md b/docs/config.md index f3f17ebee3..eccca79694 100644 --- a/docs/config.md +++ b/docs/config.md @@ -111,7 +111,7 @@ You can also explicitly configure FrankenPHP using the [global option](https://c watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. name # Sets the name of the worker, used in logs and metrics. Must be unique among global workers. Default: absolute path of the worker file. max_consecutive_failures # Sets the maximum number of consecutive failures before the worker is considered unhealthy, -1 means the worker will always restart. Default: 6. - background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure, and max_execution_time applies until that call, not after. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] holds its name, and FRANKENPHP_WORKER is not set in it. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. + background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker, requiring "name" and a script calling frankenphp_worker_tick() at least once } } } @@ -199,7 +199,7 @@ php_server [] { watch # Sets the path to watch for file changes. Can be specified more than once for multiple paths. env # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables. Environment variables for this worker are also inherited from the php_server parent, but can be overwritten here. match # match the worker to a path pattern. Overrides try_files and can only be used in the php_server directive. - background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it calls frankenphp_worker_tick(); an exit before that counts as a failure, and max_execution_time applies until that call, not after. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] holds its name, and FRANKENPHP_WORKER is not set in it. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it. + background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker, requiring "name" and a script calling frankenphp_worker_tick() at least once } worker # Can also use the short form like in the global frankenphp block. } diff --git a/frankenphp.c b/frankenphp.c index f40dfd5965..6b94de6f54 100644 --- a/frankenphp.c +++ b/frankenphp.c @@ -1203,12 +1203,9 @@ PHP_FUNCTION(frankenphp_get_worker_handle) { RETURN_THROWS(); } - if (worker_stop_socks[0] == SOCK_ERR) { - zend_throw_exception(spl_ce_RuntimeException, - "the background worker stop socket is not available", - 0); - RETURN_THROWS(); - } + /* the pair is opened before the script starts and closed at the next run + * setup or on thread exit, so a run always has one */ + ZEND_ASSERT(worker_stop_socks[0] != SOCK_ERR); /* One stream per run: the same resource is returned until the script * closes it, so fetching the handle in a loop does not grow the resource @@ -1263,12 +1260,7 @@ PHP_FUNCTION(frankenphp_worker_tick) { RETURN_THROWS(); } - if (worker_stop_socks[0] == SOCK_ERR) { - zend_throw_exception(spl_ce_RuntimeException, - "the background worker stop socket is not available", - 0); - RETURN_THROWS(); - } + ZEND_ASSERT(worker_stop_socks[0] != SOCK_ERR); if (!worker_ticked) { worker_ticked = true; diff --git a/server.go b/server.go index 332eeafa5f..70f23a335f 100644 --- a/server.go +++ b/server.go @@ -158,18 +158,15 @@ func (s *Server) Name() string { } // addWorker registers a worker scoped to this server -// scope names the worker set in errors: a server, or the global workers -func (s *Server) scope() string { +func (s *Server) addWorker(w *worker) error { + // the fallback server holds the workers declared without a scope + scope := "two workers in a server" if s == fallbackServer { - return "two global workers" + scope = "two global workers" } - return "two workers in a server" -} - -func (s *Server) addWorker(w *worker) error { if s.workersByName[w.name] != nil { - return fmt.Errorf("%s cannot have the same name: %q", s.scope(), w.name) + return fmt.Errorf("%s cannot have the same name: %q", scope, w.name) } s.workersByName[w.name] = w @@ -184,7 +181,7 @@ func (s *Server) addWorker(w *worker) error { } if s.workersByPath[w.fileName] != nil { - return fmt.Errorf("%s cannot have the same filename: %q", s.scope(), w.fileName) + return fmt.Errorf("%s cannot have the same filename: %q", scope, w.fileName) } s.workersByPath[w.fileName] = w diff --git a/threadbackgroundworker.go b/threadbackgroundworker.go index 323f183d4e..4d260b569d 100644 --- a/threadbackgroundworker.go +++ b/threadbackgroundworker.go @@ -24,8 +24,9 @@ import ( type backgroundWorkerThread struct { workerLifecycle - dummyFrankenPHPContext *frankenPHPContext - failureCount int // number of consecutive failed runs + // context of the current run, a background worker serves no request + context *frankenPHPContext + failureCount int // number of consecutive failed runs // crashCount is the number of runs that crashed past their ready point // in a row, paces their restarts; a cooperative exit resets it. Only @@ -67,7 +68,7 @@ func (handler *backgroundWorkerThread) name() string { } func (handler *backgroundWorkerThread) frankenPHPContext() *frankenPHPContext { - return handler.dummyFrankenPHPContext + return handler.context } // drain closes the Go side's end of the stop socket pair so a script @@ -134,7 +135,7 @@ func (handler *backgroundWorkerThread) setupScript() error { handler.drain() return err } - handler.dummyFrankenPHPContext = fc + handler.context = fc handler.isBootingScript = true metrics.StartWorker(handler.worker.qualifiedName) @@ -164,7 +165,7 @@ func (handler *backgroundWorkerThread) afterScriptExecution(exitStatus int) { // (drain() already took it when the exit was drain-triggered) handler.drain() worker := handler.worker - handler.dummyFrankenPHPContext = nil + handler.context = nil handler.stopBootTimer() handler.state.MarkAsWaiting(false) @@ -254,8 +255,14 @@ func (handler *backgroundWorkerThread) stopBootTimer() { func go_frankenphp_background_worker_ready(threadIndex C.uintptr_t) { // called on the PHP thread by the first frankenphp_worker_tick() of a // run; the handler is a backgroundWorkerThread because that function - // throws on every other thread kind - if handler, ok := phpThreads[threadIndex].handler.(*backgroundWorkerThread); ok && handler.isBootingScript { + // throws on every other thread kind, and a thread reaching this without + // one would wait out Init() instead + handler, ok := phpThreads[threadIndex].handler.(*backgroundWorkerThread) + if !ok { + panic("frankenphp_worker_tick() called on a thread that is not a background worker") + } + + if handler.isBootingScript { handler.isBootingScript = false // the boot succeeded, only consecutive boot failures count handler.failureCount = 0 From 5d5b461892d478a654c8ceb3b40c141e0248b9b7 Mon Sep 17 00:00:00 2001 From: Nicolas Grekas Date: Mon, 14 Sep 2026 21:35:05 +0200 Subject: [PATCH 20/20] test: the tick leaves the handle quiet A loop selecting on the handle blocks rather than spinning, because the tick consumed the wake-ups, the one sent at start included. The fixture polls the handle before and after a tick and the docs say so. --- bgworker_test.go | 17 +++++++++++++++++ docs/worker.md | 2 +- testdata/bgworker/readable.php | 25 +++++++++++++++++++++++++ 3 files changed, 43 insertions(+), 1 deletion(-) create mode 100644 testdata/bgworker/readable.php diff --git a/bgworker_test.go b/bgworker_test.go index f395ff4c6f..63b74f6919 100644 --- a/bgworker_test.go +++ b/bgworker_test.go @@ -365,6 +365,23 @@ func TestBackgroundWorkerLoopTicksOnItsOwn(t *testing.T) { requireFileEventually(t, sentinel, "background worker did not start") } +// TestBackgroundWorkerTickLeavesTheHandleQuiet checks that the tick +// consumes the wake-ups: the handle is readable at start, and not anymore +// once a tick returned, so a loop selecting on it blocks instead of +// spinning. +func TestBackgroundWorkerTickLeavesTheHandleQuiet(t *testing.T) { + sentinel := filepath.Join(t.TempDir(), "readable.txt") + initServers(t, + frankenphp.WithWorkers("bg-readable", "testdata/bgworker/readable.php", 1, + frankenphp.WithWorkerBackground(), + frankenphp.WithWorkerEnv(map[string]string{"BG_SENTINEL": sentinel}), + ), + frankenphp.WithNumThreads(2), + ) + + assert.Equal(t, "start:readable after tick:quiet after second tick:quiet", requireFileContentEventually(t, sentinel)) +} + // TestBackgroundWorkerTick checks the contract of frankenphp_worker_tick(): // true while the worker runs, false once it is drained, and still false on // the next call diff --git a/docs/worker.md b/docs/worker.md index 09473d1f10..1c7aa4e147 100644 --- a/docs/worker.md +++ b/docs/worker.md @@ -216,7 +216,7 @@ php_server { } ``` -The script calls `frankenphp_worker_tick()` once it is set up. The first call marks the worker ready: the server start waits for that point, and an exit before it counts as a failure. Until that call, `max_execution_time` applies as in any request, 30 seconds by default: a setup that outlives it is ended and counts as a boot failure, so raise the limit in `php_ini`, or call `set_time_limit()` in the script, when the setup legitimately takes longer. From the first call on, the run has no time limit, like the CLI. Every call returns `false` once FrankenPHP drains the worker, on shutdown, reboot or restart, and `true` otherwise. It never blocks and never hands out work: like `frankenphp_handle_request()`, it is where the runtime and the script meet. Between two calls, the script waits on the stream returned by `frankenphp_get_worker_handle()`, alone or together with its own streams. The stream becomes readable when FrankenPHP needs the script's attention, and `frankenphp_worker_tick()` consumes whatever was written on it, so the script does not read the stream itself. It is readable once right after the script starts, so a loop that services it ticks by itself and the worker is ready as soon as its loop runs. +The script calls `frankenphp_worker_tick()` once it is set up. The first call marks the worker ready: the server start waits for that point, and an exit before it counts as a failure. Until that call, `max_execution_time` applies as in any request, 30 seconds by default: a setup that outlives it is ended and counts as a boot failure, so raise the limit in `php_ini`, or call `set_time_limit()` in the script, when the setup legitimately takes longer. From the first call on, the run has no time limit, like the CLI. Every call returns `false` once FrankenPHP drains the worker, on shutdown, reboot or restart, and `true` otherwise. It never blocks and never hands out work: like `frankenphp_handle_request()`, it is where the runtime and the script meet. Between two calls, the script waits on the stream returned by `frankenphp_get_worker_handle()`, alone or together with its own streams. The stream becomes readable when FrankenPHP needs the script's attention, and `frankenphp_worker_tick()` consumes whatever was written on it, so the script does not read the stream itself and the stream is quiet again until the next wake-up. It is readable once right after the script starts, so a loop that services it ticks by itself and the worker is ready as soon as its loop runs. ```php 0 ? 'readable' : 'quiet'; +}; + +$seen = ['start:' . $poll()]; +frankenphp_worker_tick(); +$seen[] = 'after tick:' . $poll(); +frankenphp_worker_tick(); +$seen[] = 'after second tick:' . $poll(); +file_put_contents($_SERVER['BG_SENTINEL'], implode(' ', $seen)); + +while (frankenphp_worker_tick()) { + $read = [$handle]; + $write = $except = null; + stream_select($read, $write, $except, null); +}