fix(stream): release the JDBC result stream when it is destroyed - #105
Open
LittleGnome wants to merge 1 commit into
Open
fix(stream): release the JDBC result stream when it is destroyed#105LittleGnome wants to merge 1 commit into
LittleGnome wants to merge 1 commit into
Conversation
JdbcStream only closes the Java-side ResultStream from within _read, on the tick after close() has flipped the _closed flag. A consumer that stops early never triggers that path, because no further _read arrives: the result set and the pooled connection behind it are then held until the JVM exits. This happens in ordinary use -- breaking out of a for-await loop, an error thrown further down a pipe() chain, or an explicit destroy() -- and each occurrence permanently removes one connection from the pool. Implements _destroy so the Java stream is closed whenever the readable is torn down, including when the teardown happens before the queryAsStream promise has even resolved. ResultStream.close() returns its connection to the pool unconditionally and read() already closes at end of data, so closing again from _destroy would hand the same connection out twice. A _jdbcStreamClosed flag now tracks whether the Java side still owns the connection, and _destroy is a no-op once it does not. Making ResultStream.close() idempotent on the Java side would let this be simpler, and is a good follow-up. Two further failure modes in the same file: A rejected _jdbcStreamPromise had no handler until a consumer read from the stream. A stream that was created and then abandoned -- the normal outcome when the query itself fails -- therefore surfaced as an unhandled rejection, which terminates the process under Node's default policy on every supported version. The constructor now parks a no-op handler; the real handler in _read still reports the failure to a reading consumer. A close() failure was reported by emitting 'error'. At that point every row has already been delivered and consumers have usually detached their error handling, so a failure to release a connection escalated into an uncaught exception. It is now reported through the logger that the rest of the library already uses, and the logger is passed in at the three construction sites. Adds unit tests covering all three, using a fake Java stream that counts close() calls. Five of the eight fail against the current implementation; the other three guard against the double-close this change could otherwise introduce. No JVM or IBM i is needed, so they run in CI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release the JDBC result stream when it is destroyed
Branch:
fix/jdbc-result-stream-lifecycleThree independent problems in
ts-src/lib/jdbcstream.ts. They are groupedbecause they are all about what happens to a result stream that does not end
the way the happy path assumes, and they overlap in the same twenty lines.
1. A destroyed stream leaks its connection
JdbcStreamonly closes the Java-sideResultStreamfrom inside_read, onthe tick after
close()has set the_closedflag. A consumer that stopsearly never gets there, because no further
_readarrives.The same happens on
stream.destroy()and when a later stage of apipe()chain errors.
ResultStream.close()is what returns the connection to thepool, so each occurrence permanently removes one connection. With
AS400JDBCConnectionPoolthere is nothing that reclaims it afterwards, andonce the pool is drained unrelated queries start blocking.
Fixed by implementing
_destroy, which also covers being destroyed beforethe
queryAsStreampromise has resolved.Why the extra flag
ResultStream.close()returns its connection unconditionally, andResultStream.read()already callsclose()at end of data. So a naive_destroywould close a second time on the normal path — Node calls_destroyafter'end'becauseautoDestroydefaults to true — and hand thesame connection to two callers.
A
_jdbcStreamClosedflag now tracks whether the Java side still owns theconnection, and
_destroydoes nothing once it does not.Follow-up worth considering: making
ResultStream.close()idempotent onthe Java side would make this unnecessary and would also protect the one case
this patch deliberately leaves alone — a failure inside
read()'sinitblock, where the Java code has not closed but this patch assumes it has,
because double-returning a connection is worse than the leak that already
exists there today. Happy to send that as a separate PR; it needs a jar
rebuild, so it did not belong in this one.
2. An abandoned stream can terminate the process
_jdbcStreamPromisegets its rejection handler in_read. A stream that iscreated and never read — the normal outcome when the query itself fails and
the caller never subscribes — leaves the rejection unhandled. On every Node
version this package supports (
engines: >=16), the default policy for thatis to terminate the process.
The constructor now parks a no-op handler on the promise. The real handler in
_readis unchanged, so a consumer that does read still gets the error.3. A close failure crashes consumers that already got their rows
A failing
close()was reported withemit('error'). At that point every rowhas been delivered and
'end'may already have fired, so consumers havetypically stopped listening for errors — and an
'error'with no listener isan uncaught exception. A connection that could not be released turned into a
process-level failure.
It now goes to the
Loggerthe library already has.baseConnectionandconnectionpass their logger in at the three construction sites; the defaultstays the existing no-op logger, so nothing is printed unless a logger is
configured.
Tests
ts-src/unit-test/jdbcstream-spec.ts, 8 cases, using a fake Java stream thatcounts
close()calls. No JVM and no IBM i, so they run in CI.Against the current implementation, 5 of the 8 fail:
The three that already pass are there on purpose: they guard the behaviour
this change could plausibly break, in particular the double-close.
Verification
No Java change, no jar rebuild, no new dependency, no public API change.