Releases: Query-farm/vgi-java
Release list
v0.20.0
Moves to vgirpc 0.17.0 and ports the worker side of the per-partition result cache. Integration suite green on all three lanes (launch, shm, http) plus landing conformance.
- vgirpc 0.17.0 — client-authoritative HTTP codec negotiation,
identityas a first-class accept token, and a configurable producible-encodings set (Config.supportedEncodings; an empty list means "never compress"). Two behaviour changes reach anyone serving this library's worker over HTTP:DEFAULT_ZSTD_LEVELdrops 3 → 1, and a request body whoseContent-Encodingnames a codec the server does not advertise is now a 415 carrying the advertised set rather than being decoded anyway. Hence the minor bump.VGI_HTTP_DISABLE_ZSTDstill works — it is now a preset on the default set. - Per-partition result cache (upstream vgi
7443263) —CacheControl.partitionScope(boolean)emitsvgi.cache.partition_scope, so aSINGLE_VALUE_PARTITIONSfunction's result is additionally cached split by partition value and a later=/INscan on the partition column(s) serves those partitions without the worker. - Four example-worker fixtures backing
cache/partition_scope{,_ops,_shapes}.test:cache_partition_scope(single worker, auto partition values),cache_partition_parallel(work-queue fan-out plus a NULL partition),cache_partition_multicol((region, year)cross-product enumeration), andcache_partition_proj(projection pushdown with explicit partition values). All four apply pushed filters themselves, since DuckDB does not re-apply a pushed predicate above the scan.
v0.19.0
Port of the vgi-python 0.16 delta. Aligns the Java SDK with vgi-python 0.16.0 and the vgi C++ extension. Integration suite: 255 pass / 0 fail across launch, http, and shm lanes.
- Blended ("UNNEST-style") table-in-out: RowTransformFunction interface (subtyping = blended signal), FunctionInfo.input_from_args, arity/varargs overload resolution; fixtures geo_encode, row_sum, blended_drop, projectable_blended, hostile_provenance.
- Per-output-row provenance for batched correlated LATERAL: RowTransformFunction.PARENT_ROW_KEY + parentRows() helper; blended_explode fixture.
- substream_id + parallel per-substream finalize: InitRequest.substream_id, TableInOutFunction.hasFinalize()/finish()/maxWorkers(); FINALIZE-phase producer over materialized finish(); table-in-out max_workers no longer hardcoded to 1 (unblocks the C++ exchange cache's parallel_safe requirement).
- Result-cache surfaces: cache-control on buffered finalize emits, scalar cacheControl() (CACHE_CONTROL), conditional-revalidation validators; cached_* fixtures.
- fix(blended): exchange states are named classes so init-baked args survive HTTP state tokens (anonymous-class captured locals are synthetic fields, which StateSerializer skips).
v0.18.0
Breaking wire change: TableInfo.required_filters (CNF)
Ports the vgi-python 0.15.0 / vgi C++ extension change. The trailing TableInfo
field required_field_filter_paths (flat list<utf8>, AND-only) is renamed to
required_filters and retyped to list<list<utf8>> — conjunctive normal
form: an AND of OR-groups of dotted column paths. A group is satisfied when any
one member path has a WHERE filter; every group must be satisfied.
[["accession_number"], ["ticker","cik"]] means "accession_number AND one of
(ticker, cik)". Empty = no enforcement. No backward compatibility.
Worker-facing API
CatalogTable.builder(...).requiredFilters(List<List<String>>)and
CatalogTable.withRequiredFilters(List<List<String>>)replace the old
requiredFieldFilterPaths/withRequiredFieldFilterPaths(which took a flat
List<String>).- Validation rejects empty OR-groups and empty path strings, mirroring the
vgi-python descriptor.
Wire-compatible with vgi-python 0.15.0 and the published vgi DuckDB extension.
v0.17.0
Added
Result-cache worker support (farm.query.vgi.cache.CacheControl). The DuckDB extension caches the complete result of a cacheable table-function scan and serves identical future scans from memory or disk. A worker opts a result in by advertising vgi.cache.* metadata on the first batch it emits:
BatchUtil.emit(schema, rows, out, CacheControl.ttl(300).toMetadata(), filler);The vocabulary mirrors HTTP caching (RFC 9111/9110): ttl / expires, a reuse scope (catalog or transaction), no_store, validators (etag / last_modified), revalidatable, stale_while_revalidate / stale_if_error, and not_modified.
Conditional revalidation flows the other way: when the client holds a stale-but-revalidatable result it sends vgi.cache.if_none_match on the producer's first tick, readable from AnnotatedBatch.customMetadata(). A worker whose data is unchanged answers with a 0-row not_modified batch instead of re-streaming.
No OutputCollector change was required — emit(root, customMetadata) already existed, so CacheControl is purely a metadata renderer.
Worker.registerUnlistedTable(fn) — register a table function that is dispatchable but not advertised in the catalog's function listing, so DuckDB never registers it as a callable table function. For the scan function behind a function-backed CatalogTable that should surface only as a table.
Changed
- Pins
farm.query:vgirpc0.15.0 → 0.16.0, which carries a fix for the stateless HTTP producer path dropping the/initrequest's batch metadata (without it, conditional revalidation silently never fires over HTTP). - The example worker's native-branch and
rff_*fixtures now resolve their scratch paths fromVGI_TEST_BRANCH_DIR(default: the OS temp dir) instead of hardcoding/tmp.
Verified
Full upstream integration suite green on all three transports (launch, shm, http): 10872 assertions across 231 test cases on the http lane, 0 failures.
v0.16.0
Per-schema doc in describe.json; depend on vgirpc 0.15.0 (landing.html v3).
v0.15.0
Surface catalog macros (scalar + table) in describe.json; depend on vgirpc 0.14.0 (refreshed landing.html).
v0.14.0
Standardized VGI worker HTTP landing page: content-negotiated GET / serving the shared landing.html, GET /describe.json (one entry per advertised catalog with per-catalog scoping) + lazy column endpoint, and the VGI logo header. Requires vgirpc 0.13.0.
v0.13.0
Const-argument constraints (choices/ge/le/gt/lt/pattern) are now enforced at bind for every function kind, not just scalar. Enforcement was extracted into a shared ConstraintEnforcer and wired into the table, table-in-out, table-buffering, and aggregate bind paths.
v0.12.0
Bind-time enforcement of const-argument constraints: a const value violating a declared choices/ge/le/gt/lt/pattern constraint now fails the bind (IllegalArgumentException), complementing the per-argument discovery metadata shipped in v0.11.0.
v0.11.0
Per-argument constraint metadata for agent discovery; multi_branch_iceberg example fixture. Backward-compatible feature additions.