Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,10 @@ The standalone Python PQL and relational-engine distributions are archived and a
[`tests/test_skills.py`](tests/test_skills.py) reads the tables out of
`skills/` and puts them to the client, and with `-m live_nim` it runs every
complete example and compares what it printed to the block pasted beneath it.
A `**Verified**` block is a claim that an endpoint produced exactly that. Do
not write one by hand, and do not edit the numbers in one.
The two in-process references run the same way under `SDM_IN_PROCESS=1` with
`-m in_process`, loading the published weights here. A `**Verified**` block is
a claim that an endpoint, or this process, produced exactly that. Do not write
one by hand, and do not edit the numbers in one.

An example is complete when every name it reads was bound by itself or by an
earlier complete example. One that is not is a fragment: it is checked as code,
Expand Down
4 changes: 4 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,10 @@ SDM_RELATIONAL_URL=http://localhost:8001 \
.venv/bin/python -m pytest packages/structured-data-models-client/tests \
-m live_nim -q

# the published weights, fetched and run in this process; needs the
# in-process extra
SDM_IN_PROCESS=1 .venv/bin/python -m pytest tests/test_skills.py -m in_process -q

# real warehouses, with credentials in the environment
.venv/bin/python -m pytest \
packages/structured-data-models-connectors/tests -m live_warehouse -q
Expand Down
6 changes: 6 additions & 0 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,12 @@ The version is written in several places, and all of them move together:
`tests/test_release_contract.py` fails if `_version.py`, the Rust workspace, `java/pom.xml` or a first-party pin disagree.
Search for the old version afterwards to catch anything else, such as the coordinates in `java/README.md`.

Going from a prerelease to a release also retires the checkout build the
skills' in-process references tell a reader to run, because the extra alone
carries `in_process()` from then on. `python scripts/check_repo_hygiene.py
in-process` fails until both `skills/*/references/in-process.md` drop that
block and pin the extra to the version that first carries it.

Update `CHANGELOG.md`: the entries for the release go under its own heading
with the date you are tagging on, not the date you wrote them, so the tag
carries a changelog that names the version it
Expand Down
12 changes: 11 additions & 1 deletion docs/reference/prediction-output.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,19 @@ has one; a single-table request may have none.
order.** The endpoint is not required to return rows in the order they were
sent, and a request split across several calls certainly will not.

`row_index` is the *position* in the frame passed as `predict`, not that
frame's index label, so drop the index before making it a column. A frame that
was filtered or sorted carries labels that are no longer its positions, and
merging on those pairs a row with another row's answer, or drops it, without
saying so.

```python
answers = result.to_pandas()
joined = to_score.reset_index(names='row_index').merge(answers, on='row_index')
joined = (
to_score.reset_index(drop=True)
.reset_index(names='row_index')
.merge(answers, on='row_index')
)
```

## Columns
Expand Down
14 changes: 12 additions & 2 deletions examples/local/tabular.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,13 @@
"source": [
"Join on the key the result carries, or on `row_index`. **Never on row\n",
"order**. The endpoint is not required to answer in the order you asked, and a\n",
"request split across several calls certainly will not.\n"
"request split across several calls certainly will not.\n",
"\n",
"`row_index` is the *position* in the frame passed as `predict`, not that\n",
"frame's index label, so drop the index before making it a column. A frame\n",
"that was filtered or sorted carries labels that are no longer its positions,\n",
"and merging on those pairs a row with another row's answer, or drops it,\n",
"without saying so.\n"
]
},
{
Expand Down Expand Up @@ -266,7 +272,11 @@
],
"source": [
"answers = result.to_pandas()\n",
"to_score.reset_index(names='row_index').merge(answers, on='row_index')"
"(\n",
" to_score.reset_index(drop=True)\n",
" .reset_index(names='row_index')\n",
" .merge(answers, on='row_index')\n",
")"
]
},
{
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,7 @@ tmp_path_retention_policy = "failed"
markers = [
"live_nim: needs a running NIM",
"live_warehouse: needs a real Snowflake or Databricks warehouse",
"in_process: loads the published weights and runs them in this process",
"socket: binds a local socket; deselect where that is not permitted",
"gpu: needs a CUDA GPU",
"tooling: repository-tool tests run once in dedicated CI jobs, outside the Python version matrix",
Expand Down
181 changes: 179 additions & 2 deletions scripts/check_repo_hygiene.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""Three checks this repository has to pass before it can be published.
"""The checks this repository has to pass before it can be published.

``spdx`` asserts every first-party source file carries a licence header. A file
added without one is otherwise invisible until a release audit finds it.
Expand All @@ -13,6 +13,12 @@
chooses its own roots will miss a tree, so this enumerates from
``git ls-files``.

``in-process`` asserts the skills' in-process references install the way the
client's own version allows. They build from a checkout because no release
carries ``in_process()`` yet, and the release that first does makes that
instruction wrong. Nothing else would catch it: the references are prose, and
the example harness runs them against this tree rather than against the index.

``internal`` asserts no internal-only reference reaches a *shipped* file. The
distinction matters: an internal URL in a provenance record is correct and
required, while the same URL in shipped source is a leak. So the scan covers
Expand Down Expand Up @@ -362,11 +368,179 @@ def check_repository_url() -> int:
return 0


IN_PROCESS_REFERENCES = (
'skills/tabular/references/in-process.md',
'skills/relational/references/in-process.md',
)

# The release that first carries `in_process()`. Until the index has it the
# references have to build from a checkout; from that release on they must not,
# and the extra needs a floor no older than this. It is the feature's release
# rather than "any prerelease", so that 2.1.0rc1 does not read as a client
# without the feature and ask for the checkout back.
FIRST_IN_PROCESS = '2.0.0'
CHECKOUT_INSTALL = 'git clone -b v2-candidate'
# A version, as PEP 440's grammar spells one. That grammar orders the
# suffixes as well as naming them: a development release precedes whatever it
# hangs off, a pre-release precedes the release it names, and a post-release
# follows it. A local label orders after nothing that matters here, so it is
# read and dropped.
PEP_440 = re.compile(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: this check is heavy for what it does. About 90 lines of hand-written PEP 440 parsing, to compare this repo's own __version__ with 2.0.0. Since the script stays stdlib-only, a few lines that recognise a pre-release of 2.0.0 (or anything older) would do the same job and be easier to maintain.

r"""
v?
(?:(?P<epoch>\d+)!)?
(?P<release>\d+(?:\.\d+)*)
(?:[-_.]?(?P<pre_name>alpha|beta|preview|pre|rc|a|b|c)
[-_.]?(?P<pre>\d+)?)?
(?:-(?P<implicit_post>\d+)
|[-_.]?(?P<post_name>post|rev|r)[-_.]?(?P<post>\d+)?)?
(?:[-_.]?(?P<dev_name>dev)[-_.]?(?P<dev>\d+)?)?
(?:\+[a-z0-9]+(?:[-_.][a-z0-9]+)*)?
""",
re.IGNORECASE | re.VERBOSE,
)
PRE_RELEASE_NAMES = {
'alpha': 'a',
'a': 'a',
'beta': 'b',
'b': 'b',
'c': 'rc',
'pre': 'rc',
'preview': 'rc',
'rc': 'rc',
}
BEFORE_ANY = (-1,)
AFTER_ANY = (1,)

IN_PROCESS_FLOOR = re.compile(
r'structured-data-models-client\[in-process\]>=([^\'"\s,\]]+)'
)
CLIENT_VERSION_FILE = (
'packages/structured-data-models-client/src/sdm_client/_version.py'
)


def _version(version: str) -> tuple[object, ...]:
"""What the version sorts as, by PEP 440's rules.

Trailing zeros go, so 2.0 is the 2.0.0 it names. A release with nothing
after it sorts above every pre-release of itself and below every
post-release of itself, and a development release with neither of those in
front of it sorts below the lot.
"""
found = PEP_440.fullmatch(version.strip())
if found is None:
raise ValueError(f'not a version this check can read: {version!r}')

release = [int(part) for part in found['release'].split('.')]
while len(release) > 1 and release[-1] == 0:
release.pop()

posted = (
found['post_name'] is not None or found['implicit_post'] is not None
)
if found['pre_name'] is not None:
pre: tuple[object, ...] = (
0,
PRE_RELEASE_NAMES[found['pre_name'].lower()],
int(found['pre'] or 0),
)
elif found['dev_name'] is not None and not posted:
pre = BEFORE_ANY
else:
pre = AFTER_ANY

post = (
(0, int(found['implicit_post'] or found['post'] or 0))
if posted
else BEFORE_ANY
)
dev = (
(0, int(found['dev'] or 0))
if found['dev_name'] is not None
else AFTER_ANY
)

return (int(found['epoch'] or 0), tuple(release), pre, post, dev)


def check_in_process_install() -> int:
"""The in-process references must install the way this version allows."""
version_file = REPO_ROOT / CLIENT_VERSION_FILE
found = re.search(
r"__version__ = '([^']+)'", version_file.read_text(encoding='utf-8')
)
if found is None:
print(f'no __version__ in {CLIENT_VERSION_FILE}', file=sys.stderr)
return 1
version = found.group(1)
first = _version(FIRST_IN_PROCESS)
unpublished = _version(version) < first

findings = []
for relative in IN_PROCESS_REFERENCES:
body = (REPO_ROOT / relative).read_text(encoding='utf-8')
checkout = CHECKOUT_INSTALL in body
if unpublished and not checkout:
findings.append(
f'{relative}: installs from the index, but {version} is '
f'before {FIRST_IN_PROCESS}, the release that first carries in_process()'
)
if not unpublished and checkout:
findings.append(
f'{relative}: still builds from a checkout, but {version} is '
f'{FIRST_IN_PROCESS} or later'
)
if not unpublished:
floor = IN_PROCESS_FLOOR.search(body)
if floor is None:
findings.append(
f'{relative}: the extra names no version floor, so an '
'older client satisfies it'
)
else:
try:
floored = _version(floor.group(1))
except ValueError:
findings.append(
f'{relative}: the extra floors at {floor.group(1)}, '
'which is not a version pip can read'
)
else:
if floored < first:
findings.append(
f'{relative}: the extra floors at '
f'{floor.group(1)}, which a client without '
f'in_process() satisfies; {FIRST_IN_PROCESS} is '
'the first that carries it'
)

for finding in findings:
print(f'in-process install: {finding}', file=sys.stderr)
if findings:
print(
'\nA reader runs the first install that fits their client. When '

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Include surrounding prose in the release instructions

Nit: following the release instructions literally removes the checkout block and adds a version floor, but leaves both opening paragraphs saying no published release carries in_process(). The hygiene check then passes. Please have its message and RELEASING.md also tell maintainers to update the surrounding prerelease prose.

'a release carries in_process(), drop the checkout block from '
'both references and pin the extra to that version.',
file=sys.stderr,
)
return 1
print(f'the in-process references install the way {version} allows')
return 0


def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
'check',
choices=('spdx', 'internal', 'retired', 'repo-url', 'all'),
choices=(
'spdx',
'internal',
'retired',
'repo-url',
'in-process',
'all',
),
)
args = parser.parse_args()
if args.check == 'spdx':
Expand All @@ -377,11 +551,14 @@ def main() -> int:
return check_retired()
if args.check == 'repo-url':
return check_repository_url()
if args.check == 'in-process':
return check_in_process_install()
return (
check_spdx()
| check_internal()
| check_retired()
| check_repository_url()
| check_in_process_install()
)


Expand Down
10 changes: 8 additions & 2 deletions skills/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ be loaded without the skill it belongs to:
| [`tabular/references/pql.md`](tabular/references/pql.md) | The same language on one table, and what it refuses |
| [`relational/references/warehouses.md`](relational/references/warehouses.md) | Tables that stay in a warehouse, and metric views |
| [`relational/references/beyond-one-answer.md`](relational/references/beyond-one-answer.md) | Rankings, forecasts and explanations |
| [`tabular/references/in-process.md`](tabular/references/in-process.md) | Running kumo-tabular in this process, with no NIM |
| [`relational/references/in-process.md`](relational/references/in-process.md) | Running kumo-relational in this process, with no NIM |

## What these models are

Expand Down Expand Up @@ -61,7 +63,10 @@ only check its imports get. Run with
run and compares what it printed to the block pasted beneath it. An example
that reads a name no example before it binds is a fragment -- a Databricks
session, a Snowflake one -- and those are checked as code but executed
nowhere, so none of them is allowed to show output.
nowhere, so none of them is allowed to show output. The two in-process
references open no connection and load the published weights instead, so they
run under `SDM_IN_PROCESS=1` with `-m in_process`, and their output was pasted
from a CPU.

Where a skill says the client allows something the endpoint refuses, that gap
is the point and is stated as such. If a claim here contradicts the code, the
Expand All @@ -72,4 +77,5 @@ code is right and the skill is stale: say so rather than working around it.
- Training, fine-tuning, checkpoints, model registries. The models do not train.
- The parser behind `PredictiveQuery`. It is part of the client's Rust core
and has no Python API of its own. Write the query as a string.
- Deployment, authentication and endpoint operation. A skill assumes a URL.
- Deployment, authentication and endpoint operation. A skill assumes a URL,
or that the model runs in this process.
10 changes: 9 additions & 1 deletion skills/relational/SKILL.md

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skills/relational/SKILL.md:65

P2: Update the remaining relational installation instructions

The relational reference now includes connectors, but this main install line still omits them. On this tree, following it and materializing pandas frames fails with ModuleNotFoundError: No module named 'sdm_connectors', including for NIM users. Until the import fix discussed in the thread lands, please add compatible connectors here and to the README and installation page.

Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: sdm-relational
description: "Ask kumo-relational a question about an entity and the records linked to it: write the question as a predictive query over a schema of related tables, covering classification, regression, forecasting and ranking the candidates an entity may link to. Triggers: kumo-relational, KumoRelational, PREDICT, FOR EACH, predictive query, PQL, Schema.from_data, Schema.from_source, Relationship, ranking, forecast, link prediction, explanation, Snowflake, Databricks, warehouse, metric view, semantic view, SQL, catalog."
description: "Ask kumo-relational a question about an entity and the records linked to it: write the question as a predictive query over a schema of related tables, covering classification, regression, forecasting and ranking the candidates an entity may link to. Triggers: kumo-relational, KumoRelational, PREDICT, FOR EACH, predictive query, PQL, Schema.from_data, Schema.from_source, Relationship, ranking, forecast, link prediction, explanation, Snowflake, Databricks, warehouse, metric view, semantic view, SQL, catalog, SDMClient.in_process, run the model in this process, no NIM."
---

# Asking kumo-relational
Expand Down Expand Up @@ -121,6 +121,14 @@ print(answer.to_pandas())
`id` is the entity, `row_index` its position in `indices`. Join on the key when
it is there, otherwise on `row_index`, and never on row order.

## Without a NIM

Every example here points at a NIM. With none to point at, the same client
runs the model in this process: `SDMClient.in_process()` in place of the URL,
and the rest unchanged. It is an extra, it loads the published weights, and on
a CPU its numbers differ from a GPU's. All of that is in
[`references/in-process.md`](references/in-process.md).

## Describing the data

A `Schema` says which tables exist, how they are keyed, which column carries
Expand Down
Loading
Loading