Log the error when Postgres closes a connection in a transaction

[anurag]

Problem

withAppLock() in app/store.ts checks out a client with db().connect() for one transaction. claimRunApp (workflows host) and claimDelete (gateway) use it. While a client is checked out, pg-pool does not listen for "error" on it (see _acquireClient in pg-pool/index.js). The pool listener of #43 gets only the errors of idle clients.

If Postgres closes the connection during the transaction, for example in a restart, a failover, maintenance, or with pg_terminate_backend, the query fails. Then the socket closes, and the pg client emits "error" (Connection terminated unexpectedly). No listener got the event, so Node stopped the process with exit code 1:

Emitted 'error' event on Client instance at:
    at Client._handleErrorEvent (node_modules/pg/lib/client.js:422:10)

Change

• withAppLock() attaches an "error" listener right after connect(). The listener logs Lost a Postgres connection in a transaction: with console.error, as the pool listener of Log the error when Postgres closes an idle pool connection #43 does, and does not throw the error.
• The finally block removes the listener just before client.release(). release() attaches the listener of the pool again, and the pool removes the closed client. If the listener remains, each transaction on the same client adds one more.
• The error of the query still goes to the caller, so claimRunApp and claimDelete fail as before.

No other code changes.

Test

tests/store.test.ts makes db().connect() give claimDelete a fake client: an EventEmitter with query and release. The lock query fails. Then, in a later macrotask, the client emits "error" while the rollback waits. The test checks that:

• the emit does not throw, and the error is logged,
• the caller gets the error of the query,
• no listener remains after the release, and the release occurs one time.

The test connects to no Postgres. Without the fix, it fails with expected [Function] to not throw an error but 'Error: Connection terminated unexpectedly' was thrown. Without the off() line, it fails with expected 1 to be +0.

npm run check passes (Biome lint, tsc, 401 tests).

End-to-end check

On a throwaway Homebrew Postgres 18 on port 55987, not the docker-compose database. The scripts:

1. The reproduction of the issue. It writes out the pattern of withAppLock(): select pg_sleep(5) in a transaction, and pg_terminate_backend from psql after 1 s. It does not call withAppLock(), so it can only show the crash.
2. Through the real claimDelete and claimRunApp. A second session holds the app lock, so the transaction waits in pg_advisory_xact_lock (pg_stat_activity shows Lock:advisory). After 1 s, pg_terminate_backend ends the vibe-factory backend. Postgres sends 57P01, and then the socket closes.
3. The same with SIGKILL on the backend. The connection closes with no error message from Postgres, so the client emits "error" before the query fails.

Case	Before (2c64c81)	After
1. Script of the issue	exit 1	— (does not call withAppLock())
2. claimDelete, pg_terminate_backend	exit 1	exit 0. The caller gets 57P01. The next claim gives { claimed: true, runIds: ["run-old"] }.
2. claimRunApp, pg_terminate_backend	exit 1	exit 0. The caller gets 57P01. The next claim gives { claimed: true }.
3. claimDelete, SIGKILL	exit 1	exit 0. The caller gets Connection terminated unexpectedly. After crash recovery, the next claim gives { claimed: true, runIds: ["run-old"] }.

After the fix, each case logs one error of 7 lines: the message and the stack. pg-pool adds err.client only in its idle listener, so this log does not include the settings of the connection.

Dependency

#43 is merged. This branch starts from origin/main at 2c64c81.

🤖 Generated with Claude Code