Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -458,6 +458,7 @@ jobs:
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: java-native-linux-x86_64
retention-days: 1
path: java-natives/

# The binding with its test-only operations, built beside native-wheel by
Expand Down Expand Up @@ -506,10 +507,12 @@ jobs:
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: native-wheel-test-ops
retention-days: 1
path: native-test-ops/
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: java-test-ops-native
retention-days: 1
path: rust/target/release/libsdm_jni.so

native-wheel-aarch64:
Expand Down Expand Up @@ -539,6 +542,7 @@ jobs:
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: ${{ matrix.artifact }}
retention-days: ${{ matrix.build == 'test-ops' && 1 || '' }}
path: native/
- name: The Java binding, by platform
if: matrix.java == 'shipped'
Expand All @@ -549,6 +553,7 @@ jobs:
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: java-native-linux-aarch64
retention-days: 1
path: java-natives/

# The binding's tests on the wheel that ships, at the E16 floor: the file
Expand Down Expand Up @@ -1499,6 +1504,7 @@ jobs:
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: java-native-${{ matrix.platform }}
retention-days: 1
path: java-natives/

# The one JAR, with every platform's native (the all-natives profile
Expand Down Expand Up @@ -1551,6 +1557,7 @@ jobs:
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: java-jar
retention-days: 1
path: java-jar/

java-test:
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,8 @@ The public API snapshot records this version update; publication remains pending

### Fixed

- The client wheel's notices name the IANA Time Zone Database that its native module embeds, with the release and the tz project's LICENSE.
The installation docs say that the `postgres` and `lakebase` extras install LGPL-3.0-only psycopg, which the wheels do not bundle.
- Explicit empty outputs remain invalid with `EXPLAIN`; unsupported local-runtime outputs are refused before model weights load.
- Gradient identities and merging run in Rust, with generated transport keys distinguished from genuine source keys.
Merged explanations and predictions now use matching row indices, including repeated result parts.
Expand Down
3 changes: 3 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ Compiled dependencies
Python native wheels and Java native libraries include Rust dependencies.
Their licenses and notices are collected in
packages/structured-data-models-client/THIRD-PARTY-NOTICES-RUST.txt.
They embed the IANA Time Zone Database through the jiff-tzdb crate. The tz
project places it in the public domain. Its release and the tz project's
LICENSE are in the same file.
Optional Python connector dependencies are resolved at installation time.

The offline schema visualization includes Mermaid 11.15.0, distributed under
Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,12 @@ See [the local-model quickstart](docs/get-started/quickstart.md).
| `pip install 'structured-data-models-client[snowflake]'` | Reads tables from a warehouse. Also `[sqlite]`, `[duckdb]`, `[postgres]`, `[databricks]`, `[lakebase]`, `[s3]` |
| `pip install 'structured-data-models-client[all]'` | Both models and every warehouse backend |

The `postgres`, `lakebase` and `all` extras install psycopg 3.1 or later.
The `postgres` extra also installs psycopg-binary.
Both are licensed LGPL-3.0-only.
Your installer resolves them from the package index.
NVIDIA's wheels do not bundle them.

[Installation](docs/get-started/installation.md) has the rest, including what
differs on Databricks.

Expand Down
6 changes: 6 additions & 0 deletions docs/get-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@
| `pip install 'structured-data-models-client[databricks-serving]'` | Reaches a model served on Databricks Model Serving, through `SDMClient.for_databricks_serving`. Not in `[all]`: it is a way to reach a model, not a backend. |
| `pip install 'structured-data-models-client[in-process]'` | Runs the models in this Python process with structured-data-models, through `SDMClient.in_process`. Brings PyTorch. **Python 3.11 or later**: on 3.10 the extra installs nothing, and `in_process()` says why. Not in `[all]`, for the same reason as `[databricks-serving]`. |

The `postgres`, `lakebase` and `all` extras install psycopg 3.1 or later.
The `postgres` extra also installs psycopg-binary.
Both are licensed LGPL-3.0-only.
Your installer resolves them from the package index.
NVIDIA's wheels do not bundle them.

**Install distribution names; import module names.**
The three maintained distributions and their imports are listed below.
The client wheel includes the shared Rust core for PQL, sampling and prediction preparation.
Expand Down
3 changes: 3 additions & 0 deletions packages/structured-data-models-client/NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ This distribution includes third-party code:
- The native module, sdm_client._native, statically links Rust crates. Their
licenses and notices are in THIRD-PARTY-NOTICES-RUST.txt, beside this file
in the wheel's .dist-info/licenses directory.
- The native module embeds the IANA Time Zone Database through the jiff-tzdb
crate. The tz project places it in the public domain. Its release and the
tz project's LICENSE are in THIRD-PARTY-NOTICES-RUST.txt.
- The offline schema visualization bundles Mermaid 11.15.0 and its
dependencies, MIT licensed, as sdm_client/assets/mermaid.min.js. Its license
is in sdm_client/assets/MERMAID_LICENSE, its source in
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ The Python wheel and Java JAR carry the Rust core, built from
rust/bindings/sdm-py and rust/bindings/sdm-jni with production features for the targets
x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-apple-darwin, aarch64-apple-darwin, x86_64-pc-windows-msvc. These are the crates
linked into any production build, other than this repository's own, each with the
licence, copyright and notice files it ships, and then the Rust standard
library. Written by scripts/rust_core/rust_notices.py from rust/Cargo.lock;
licence, copyright and notice files it ships, then the IANA Time Zone
Database that jiff-tzdb embeds, and then the Rust standard library.
Written by scripts/rust_core/rust_notices.py from rust/Cargo.lock;
regenerate it with --write rather than editing it.

Crates
Expand Down Expand Up @@ -5115,6 +5116,22 @@ Licence: MIT
(the same text as LICENSE-MIT of dyn-clone 1.0.20 above)


The IANA Time Zone Database
===========================

jiff-tzdb 0.1.9 embeds the IANA Time Zone Database, release 2026e,
compiled to TZif data: https://www.iana.org/time-zones. The tz project states
its terms in the LICENSE file of that release,
https://data.iana.org/time-zones/releases/tzdata2026e.tar.gz:

--- LICENSE ---
Unless specified below, all files in the tz code and data (including
this LICENSE file) are in the public domain.

If the files date.c, newstrftime.3, and strftime.c are present, they
contain material derived from BSD and use the BSD 3-clause license.


The Rust standard library
=========================

Expand Down
74 changes: 71 additions & 3 deletions scripts/rust_core/rust_notices.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@
them is linked, and the workspace's own crates left out because they are
this repository's. For each, the file reproduces the licence, copyright and
notice files the crate ships, which is what its licence asks of a binary
distribution, and the Rust standard library is named after them.
distribution. The IANA Time Zone Database that jiff-tzdb embeds follows them
with the release it embeds and the tz project's own statement of its terms,
and then the Rust standard library is named.

scripts/rust_core/rust_notices.py --write regenerate the file
scripts/rust_core/rust_notices.py --check fail if it is out of date
Expand Down Expand Up @@ -65,11 +67,33 @@
rust/bindings/sdm-py and rust/bindings/sdm-jni with production features for the targets
{', '.join(TARGETS)}. These are the crates
linked into any production build, other than this repository's own, each with the
licence, copyright and notice files it ships, and then the Rust standard
library. Written by scripts/rust_core/rust_notices.py from rust/Cargo.lock;
licence, copyright and notice files it ships, then the IANA Time Zone
Database that jiff-tzdb embeds, and then the Rust standard library.
Written by scripts/rust_core/rust_notices.py from rust/Cargo.lock;
regenerate it with --write rather than editing it.
"""

TZDB_LICENSE_RELEASE = '2026e'
TZDB_LICENSE = """\
Unless specified below, all files in the tz code and data (including
this LICENSE file) are in the public domain.

If the files date.c, newstrftime.3, and strftime.c are present, they
contain material derived from BSD and use the BSD 3-clause license."""

TZDB = """\
The IANA Time Zone Database
===========================

jiff-tzdb {crate} embeds the IANA Time Zone Database, release {release},
compiled to TZif data: https://www.iana.org/time-zones. The tz project states
its terms in the LICENSE file of that release,
https://data.iana.org/time-zones/releases/tzdata{release}.tar.gz:

--- LICENSE ---
{licence}
"""

STD = """\
The Rust standard library
=========================
Expand Down Expand Up @@ -139,6 +163,49 @@ def _text(path: pathlib.Path) -> str:
return '\n'.join(line.rstrip() for line in text.strip().splitlines())


def _tzdb_release(root: pathlib.Path) -> str:
"""The IANA Time Zone Database release the jiff-tzdb crate at `root`
embeds, as its `VERSION` names it.

Stops when the release cannot be read, or when it is not the one whose
LICENSE file :data:`TZDB_LICENSE` reproduces, so that a new release is
attributed only after its terms are read again.
"""
source = root / 'tzname.rs'
text = source.read_text() if source.is_file() else ''
found = re.search(
r'static VERSION: Option<&str> = Some\(r?"(\d{4}[a-z]+)"\);', text
)
if not found:
sys.exit(f'{source} names no IANA Time Zone Database release')
release = found.group(1)
if release != TZDB_LICENSE_RELEASE:
sys.exit(
f'jiff-tzdb embeds tzdata{release}, but TZDB_LICENSE is the '
f'LICENSE of tzdata{TZDB_LICENSE_RELEASE}; read the new LICENSE '
f'and update both'
)
return release


def _tzdb(
packages: dict[tuple[str, str], dict], crates: list[tuple[str, str]]
) -> str:
"""The section for the IANA Time Zone Database, if jiff-tzdb is linked."""
for name, version in crates:
if name == 'jiff-tzdb':
root = pathlib.Path(packages[(name, version)]['manifest_path'])
return (
TZDB.format(
crate=version,
release=_tzdb_release(root.parent),
licence=TZDB_LICENSE,
)
+ '\n\n'
)
return ''


def render() -> str:
packages = _packages()
crates = _linked()
Expand Down Expand Up @@ -203,6 +270,7 @@ def render() -> str:
+ '\n\n\n'
+ '\n\n\n'.join(sections)
+ '\n\n\n'
+ _tzdb(packages, crates)
+ STD.format(toolchain=_toolchain())
)

Expand Down
2 changes: 2 additions & 0 deletions tests/compatibility/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,8 @@ Both workflows share one repository-wide concurrency group, while the required p
They reuse production wheel, JAR and JNI artifacts from successful full CI for the exact candidate commit.
The matrix loads the exact JAR; focused Java API and giant tests compile the exact clean source commit and load the verified packaged JNI, recording source-equivalent execution.
Set the same `ci_run_id` dispatch input to pin identical bytes across reruns.
CI keeps the JAR and JNI artifacts for one day.
A later dispatch needs a CI run from that day.
For the optional registry, use the installed candidate Python interpreter, its pretrained model runtime, the candidate JNI library and Maven:

```sh
Expand Down
25 changes: 25 additions & 0 deletions tests/test_ci_workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -1289,3 +1289,28 @@ def test_contract_installer_preserves_supported_duckdb_runtime(
assert len(active) == 1
assert str(active[0].specifier) == '==' + duckdb_pin
assert '$CELL_PINS duckdb pyyaml' in install


def test_unreleased_binaries_expire_in_a_day() -> None:
uploads = {
step['with']['name']: step['with']
for job in _workflow()['jobs'].values()
for step in job.get('steps', [])
if str(step.get('uses', '')).startswith('actions/upload-artifact@')
}
unreleased = sorted(
name
for name in uploads
if name.startswith('java-') or 'test-ops' in name
)
assert {'java-jar', 'java-test-ops-native', 'native-wheel-test-ops'} <= set(
unreleased
)
assert {
name: uploads[name].get('retention-days') for name in unreleased
} == {name: 1 for name in unreleased}
assert uploads['${{ matrix.artifact }}']['retention-days'] == (
"${{ matrix.build == 'test-ops' && 1 || '' }}"
)
for name in ('native-wheel', 'native-wheel-${{ matrix.platform }}'):
assert 'retention-days' not in uploads[name], name
24 changes: 23 additions & 1 deletion tests/test_rust_notices_terms.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
# All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""Which of a crate's files ``rust_notices.py`` reproduces as its terms.
"""Which of a crate's files ``rust_notices.py`` reproduces as its terms, and
which IANA Time Zone Database release it attributes.

yaml-rust2 ships its two licences as ``MIT-LICENSE`` and
``Apache-LICENSE``, which the pattern missed, so the notices script stopped
Expand Down Expand Up @@ -46,3 +47,24 @@ def test_a_crate_s_terms_files_are_found_by_name(
name: str, terms: bool
) -> None:
assert bool(rust_notices.TERMS.match(name)) is terms


@pytest.mark.parametrize(
('version', 'release'),
[
('Some(r"2026e")', '2026e'),
('Some(r"2027a")', None),
('None', None),
],
)
def test_the_embedded_tzdb_release_is_read_from_jiff_tzdb(
tmp_path: Path, version: str, release: str | None
) -> None:
(tmp_path / 'tzname.rs').write_text(
f'pub(super) static VERSION: Option<&str> = {version};\n'
)
if release is None:
with pytest.raises(SystemExit):
rust_notices._tzdb_release(tmp_path)
else:
assert rust_notices._tzdb_release(tmp_path) == release
Loading