perf(jdbc): set a fetch size on statements that return a result set - #108
Open
LittleGnome wants to merge 1 commit into
Open
perf(jdbc): set a fetch size on statements that return a result set#108LittleGnome wants to merge 1 commit into
LittleGnome wants to merge 1 commit into
Conversation
The Toolbox driver uses a small default fetch size, so reading a result set of any size spends most of its wall-clock time on server round-trips rather than in the database. Raising it batches more rows per trip and noticeably shortens larger queries; the cost is a proportionally larger client-side row buffer. Applies a default fetch size of 500 in query, queryAsStream and execute. The remaining prepareStatement call sites -- update, batchUpdate and insertAndGetId -- either return no result set or only a single generated key, so nothing is gained by setting it there. setFetchSize is a hint that drivers may ignore or reject, so a failure to apply it leaves the driver default in place instead of failing the query. The value is overridable with the JT400_FETCH_SIZE environment variable or the jt400.fetchSize system property, for deployments where the default is a poor fit. An unparsable or non-positive value falls back to the default rather than throwing during class initialisation. Verified default 500, both override mechanisms, and the fallback for garbage and negative input. An environment variable is the least invasive way to make this adjustable, but a connection-config key threaded through Props would fit the library's existing configuration style better. Happy to rework it that way. The unit suite passes against the rebuilt jar and does exercise all three code paths, though against HSQLDB rather than DB2 for i. The throughput claim itself needs a real IBM i to confirm. Note on the jar: ant was not available here, so java/lib/jt400wrap.jar was rebuilt with javac and jar directly, matching what build.xml does (-source 8 -target 8, same classpath). Re-running npm run build-java regenerates it with your own toolchain and is the preferred way to take this change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Set a fetch size on statements that return a result set
Branch:
perf/statement-fetch-sizeRationale
The Toolbox driver defaults to a small JDBC fetch size, so reading a result set
of any size spends most of its wall-clock time waiting on server round-trips
rather than in the database. Raising it batches more rows per trip.
This applies a default of 500 in the three places that read a result set:
queryqueryAsStreamexecuteStatementWrapmay be a queryupdatebatchUpdateinsertAndGetIdsetFetchSizeis a hint that a driver may ignore or reject, so a failure toapply it leaves the driver default in place rather than failing the query.
Configurability
Overridable with the
JT400_FETCH_SIZEenvironment variable or thejt400.fetchSizesystem property. An unparsable or non-positive value fallsback to the default instead of throwing during class initialisation.
Verified directly against the built jar:
An environment variable is the least invasive way to make this adjustable,
but a connection-config key threaded through
Propswould fit this library'sconfiguration style better. Happy to rework it that way — it is a small
change either way, and I would rather match your preference than argue for
mine.
If you would prefer the default left alone entirely and only the override
added, that also works; say the word.
Verification
The unit suite does exercise all three modified paths, so the change is proven
not to break them — but against HSQLDB, not DB2 for i. The throughput claim
itself needs a real IBM i to confirm, and the right value for the default may
well be different in your environment than the one I picked.
Note on the jar
antwas not available in my environment, sojava/lib/jt400wrap.jarwasrebuilt with
javacandjardirectly, matching whatbuild.xmldoes(
-source 8 -target 8, same classpath, same class set and bytecode version).Most of the size delta is the different compiler rather than this patch.
Re-running
npm run build-javaregenerates it with your own toolchain and isthe preferred way to take this change.