Skip to content

perf(jdbc): set a fetch size on statements that return a result set - #108

Open
LittleGnome wants to merge 1 commit into
tryggingamidstodin:masterfrom
LittleGnome:perf/statement-fetch-size
Open

perf(jdbc): set a fetch size on statements that return a result set#108
LittleGnome wants to merge 1 commit into
tryggingamidstodin:masterfrom
LittleGnome:perf/statement-fetch-size

Conversation

@LittleGnome

Copy link
Copy Markdown

Set a fetch size on statements that return a result set

Branch: perf/statement-fetch-size

Rationale

The Toolbox driver defaults to a small JDBC fetch size, so reading a result set
of any size spends most of its wall-clock time waiting on server round-trips
rather than in the database. Raising it batches more rows per trip.

This applies a default of 500 in the three places that read a result set:

Method Applied Why
query yes reads a full result set
queryAsStream yes reads a full result set
execute yes the StatementWrap may be a query
update no no result set
batchUpdate no no result set
insertAndGetId no one generated key

setFetchSize is a hint that a driver may ignore or reject, so a failure to
apply it leaves the driver default in place rather than failing the query.

Configurability

Overridable with the JT400_FETCH_SIZE environment variable or the
jt400.fetchSize system property. An unparsable or non-positive value falls
back to the default instead of throwing during class initialisation.

Verified directly against the built jar:

default                        -> 500
-Djt400.fetchSize=1234         -> 1234
JT400_FETCH_SIZE=77            -> 77
JT400_FETCH_SIZE=abc           -> 500
JT400_FETCH_SIZE=-5            -> 500

An environment variable is the least invasive way to make this adjustable,
but a connection-config key threaded through Props would fit this library's
configuration style better. Happy to rework it that way
— it is a small
change either way, and I would rather match your preference than argue for
mine.

If you would prefer the default left alone entirely and only the override
added, that also works; say the word.

Verification

npm run build && npm test       -> 32 passing
npm run test-cjs                -> 32 passing

The unit suite does exercise all three modified paths, so the change is proven
not to break them — but against HSQLDB, not DB2 for i. The throughput claim
itself needs a real IBM i to confirm
, and the right value for the default may
well be different in your environment than the one I picked.

Note on the jar

ant was not available in my environment, so java/lib/jt400wrap.jar was
rebuilt with javac and jar directly, matching what build.xml does
(-source 8 -target 8, same classpath, same class set and bytecode version).
Most of the size delta is the different compiler rather than this patch.

Re-running npm run build-java regenerates it with your own toolchain and is
the preferred way to take this change.

The Toolbox driver uses a small default fetch size, so reading a result
set of any size spends most of its wall-clock time on server round-trips
rather than in the database. Raising it batches more rows per trip and
noticeably shortens larger queries; the cost is a proportionally larger
client-side row buffer.

Applies a default fetch size of 500 in query, queryAsStream and execute.
The remaining prepareStatement call sites -- update, batchUpdate and
insertAndGetId -- either return no result set or only a single generated
key, so nothing is gained by setting it there.

setFetchSize is a hint that drivers may ignore or reject, so a failure to
apply it leaves the driver default in place instead of failing the query.

The value is overridable with the JT400_FETCH_SIZE environment variable
or the jt400.fetchSize system property, for deployments where the default
is a poor fit. An unparsable or non-positive value falls back to the
default rather than throwing during class initialisation. Verified
default 500, both override mechanisms, and the fallback for garbage and
negative input.

An environment variable is the least invasive way to make this
adjustable, but a connection-config key threaded through Props would fit
the library's existing configuration style better. Happy to rework it
that way.

The unit suite passes against the rebuilt jar and does exercise all three
code paths, though against HSQLDB rather than DB2 for i. The throughput
claim itself needs a real IBM i to confirm.

Note on the jar: ant was not available here, so java/lib/jt400wrap.jar
was rebuilt with javac and jar directly, matching what build.xml does
(-source 8 -target 8, same classpath). Re-running npm run build-java
regenerates it with your own toolchain and is the preferred way to take
this change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant