Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions .github/workflows/kumo-fm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES.
# All rights reserved.
# SPDX-License-Identifier: Apache-2.0

name: kumo-fm

on: # yamllint disable-line rule:truthy
pull_request:
paths:
- 'packages/experimental/kumo-fm/**'
- 'packages/experimental/kumo-pytorch/**'
- 'packages/diskgraph/**'
- 'uv.lock'
- '.github/workflows/kumo-fm.yml'
push:
branches: [main]
paths:
- 'packages/experimental/kumo-fm/**'
- 'packages/experimental/kumo-pytorch/**'
- 'packages/diskgraph/**'
- 'uv.lock'
- '.github/workflows/kumo-fm.yml'
workflow_dispatch:

permissions:
contents: read

jobs:
diskgraph-wheel:
uses: ./.github/workflows/_diskgraph-wheel.yml

package-and-wheel:
needs: diskgraph-wheel
runs-on: ubuntu-24.04
timeout-minutes: 45
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
- uses: ./.github/actions/kumo-pytorch-setup
with:
diskgraph-wheel-artifact: diskgraph-wheel
- name: Test the source package against the current native stack
run: |
uv sync --locked --inexact --no-install-package diskgraph --no-install-package torch \
--package kumo-fm --extra test
uv run --no-sync --package kumo-fm python -m pytest -q \
packages/experimental/kumo-fm/tests
- name: Build both finished wheels
run: |
uv build --wheel --package kumo-pytorch --out-dir "$RUNNER_TEMP/dist"
uv build --sdist --package kumo-fm --out-dir "$RUNNER_TEMP/dist"
uv build --wheel "$RUNNER_TEMP"/dist/kumo_fm-*.tar.gz --out-dir "$RUNNER_TEMP/dist"
- name: Install finished wheels outside the checkout
working-directory: ${{ runner.temp }}
run: |
uv venv --python 3.12 fresh
uv pip install --python fresh/bin/python --torch-backend cpu \
/tmp/diskgraph-wheel/diskgraph-*.whl dist/kumo_pytorch-*.whl \
dist/kumo_fm-*.whl 'pytest>=8,<10'
uv pip check --python fresh/bin/python
- name: Run tests using only installed runtime packages
run: |
mkdir "$RUNNER_TEMP/stage"
cp -R packages/experimental/kumo-fm/tests \
packages/experimental/kumo-fm/examples "$RUNNER_TEMP/stage/"
cd "$RUNNER_TEMP/stage"
../fresh/bin/python -I -c 'import kumo.fm, kumo.pytorch, diskgraph; print(kumo.fm.__file__)'
../fresh/bin/python -m pytest -q tests
- name: Build matching connector and engine wheels
run: |
for package in kumo-api kumo-pql kumo-settings kumo-core kumo-engine; do
uv build --wheel --package "$package" --out-dir "$RUNNER_TEMP/engine-wheels"
done
- name: Test connector snapshot to FM from installed wheels
working-directory: ${{ runner.temp }}/stage
run: |
uv pip install --python ../fresh/bin/python --torch-backend cpu \
../engine-wheels/*.whl
uv pip check --python ../fresh/bin/python
../fresh/bin/python -I -c 'import kumo.engine, kumo.fm, kumo.pytorch, diskgraph'
../fresh/bin/python -m pytest -q tests/test_connector_snapshot.py
1 change: 1 addition & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ This monorepo contains Kumo's graph machine-learning application, Python package
| Service Python SDK | `packages/kumo-sdk/` | Python client for a configured Kumo service (`kumoai_studio`) |
| Local Python SDK | `packages/kumo-engine/` | In-process SDK surface (`kumo.engine`) for local execution with local or remote data sources |
| Shared runtime | `packages/kumo-core/` | Service-independent data processing, training, evaluation, and prediction used by the service and local SDK |
| Experimental model data | `packages/experimental/kumo-pytorch/`, `packages/experimental/kumo-fm/` | Public columnar sampling and FM-specific Arrow payload assembly |
| Connectors | `kumo/connector/` | Data-warehouse and file-source integrations |
| Data pipeline | `kumo/data/` | Export, cache, and DiskGraph-backed table generation |
| Query language | `packages/kumo-pql/`, `kumo/pquery/` | Parser and semantic validation package; service query and table-generation orchestration |
Expand Down
44 changes: 44 additions & 0 deletions benchmark/fm-payloads/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# FM payload comparison

`runner.py` compares the installed `kumo-fm` wheel with the pinned
fm-dataloader DuckDB reference on the same frozen bundle, workload, and
selected instances. Install the reference checkout only in the benchmark
environment; it is not a `kumo-fm` runtime dependency.

Run correctness first, then warmups and interleaved measured draws:

```bash
python make_fixture.py /tmp/fm-case
python runner.py --bundle /tmp/fm-case/bundle \
--workload /tmp/fm-case/workload.json --index /tmp/fm-case/index \
--draws 5 --output /tmp/fm-report.json
```

For an existing bundle and workload:

```bash
python runner.py --bundle /data/bundle --workload /data/workload.json \
--index /data/fm-index --draws 7 --output /tmp/fm-report.json
```

The workload must be eligible for the public `kumo.pytorch` recent-sampling
path. Uniform/timeless and empty-layer workloads are reported as excluded,
not silently rewritten. Reports include source revisions, dependency
versions, machine metadata, row counts, logical Arrow bytes, body bytes, and
raw per-phase timings. Retain the bundle, workload, wheel hashes, index
provenance, and report together when publishing a comparison.

## Omnistation smoke comparison

[Raw report](reports/omnistation-smoke.json) records five interleaved draws
after two warmups on a four-event, one-hop synthetic graph. Both encodings
decoded identically to the DuckDB reference and produced the same body byte
counts. Median phase times in milliseconds:

| Encoding | DuckDB reference full payload | DiskGraph load and assembly | Encoding retained batch |
| --- | ---: | ---: | ---: |
| Arrays | 40.200 | 3.453 | 0.710 |
| Arrow IPC | 40.141 | 3.477 | 0.424 |

The backend paths and measured phase boundaries differ. This small smoke
case validates parity and timing instrumentation, not general throughput.
30 changes: 30 additions & 0 deletions benchmark/fm-payloads/make_fixture.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES.
# All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""Create the frozen smoke case used by the benchmark and SDM notebook."""

import json
import sys
from pathlib import Path

TESTS = Path(__file__).resolve().parents[2] / "packages/experimental/kumo-fm/tests"
sys.path.insert(0, str(TESTS))

from test_integration import fixture # noqa: E402


def main() -> None:
if len(sys.argv) != 2:
raise SystemExit("usage: python make_fixture.py /path/to/new/case")
root = Path(sys.argv[1]).resolve()
root.mkdir(parents=True, exist_ok=False)
_, workload = fixture(root)
(root / "supported").rename(root / "bundle")
(root / "workload.json").write_text(json.dumps(workload, indent=2) + "\n")
print(f"bundle: {root / 'bundle'}")
print(f"workload: {root / 'workload.json'}")


if __name__ == "__main__":
main()
140 changes: 140 additions & 0 deletions benchmark/fm-payloads/reports/omnistation-smoke.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
{
"bundle_manifest_sha256": "756537788732b93e1c94f24c386cf253d867adc5220e4689d636c1627a88467b",
"workload_sha256": "5c479c8c9438f297c23e5f8e0ce00ff95fc406f36bfe765c94723f75dbed0395",
"wheel_sha256": {
"kumo_fm-0.1.0a1-cp311-cp311-linux_x86_64.whl": "92cd45fb8691db74c86ab53f382a2880231008f2db02ba1de4ae77a259479eb6",
"diskgraph-0.2.0-cp311-abi3-linux_x86_64.whl": "ede0ca14dee5ccc75266afaf55c8ffd90f73a6e6f835448ddeff87655186522e"
},
"snapshot_id": "7a69e955e84d54d6b3b2711be57f9bf1569ada00abbae1a658fb31b0cfdaa76e",
"engine_revision": "539e9a4f1cb7ae7cfa254ea2eac90629a07a032b",
"reference_revision": "b4516394e98929c5b4752745866d5e3138276255",
"checkout_dirty": false,
"machine": {
"platform": "Linux-7.0.0-1010-aws-x86_64-with-glibc2.39",
"processor": "x86_64",
"python": "3.11.15"
},
"versions": {
"kumo-fm": "0.1.0a1",
"kumo-pytorch": "0.1.0a1",
"diskgraph": "0.2.0",
"fm-dataloader": "0.1.0",
"duckdb": "2.0.0",
"pyarrow": "24.0.0",
"numpy": "2.4.6"
},
"cases": [
{
"encoding": "arrays",
"context_rows": 2,
"predict_rows": 4,
"logical_arrow_bytes": 584,
"new_body_bytes": 3374,
"reference_body_bytes": 3374,
"draws": [
{
"reference_full_ms": 41.671234,
"new_load_ms": 3.360692,
"new_encode_ms": 0.694896
},
{
"reference_full_ms": 39.565126,
"new_load_ms": 3.484977,
"new_encode_ms": 0.713286
},
{
"reference_full_ms": 40.508425,
"new_load_ms": 3.447152,
"new_encode_ms": 0.710074
},
{
"reference_full_ms": 40.103816,
"new_load_ms": 3.452723,
"new_encode_ms": 0.979746
},
{
"reference_full_ms": 40.199899,
"new_load_ms": 3.533769,
"new_encode_ms": 0.698663
}
]
},
{
"encoding": "arrow_ipc",
"context_rows": 2,
"predict_rows": 4,
"logical_arrow_bytes": 584,
"new_body_bytes": 8859,
"reference_body_bytes": 8859,
"draws": [
{
"reference_full_ms": 40.14074,
"new_load_ms": 3.622619,
"new_encode_ms": 0.465381
},
{
"reference_full_ms": 41.85428,
"new_load_ms": 3.477252,
"new_encode_ms": 0.424267
},
{
"reference_full_ms": 40.112547,
"new_load_ms": 3.372056,
"new_encode_ms": 0.444124
},
{
"reference_full_ms": 41.194207,
"new_load_ms": 3.428842,
"new_encode_ms": 0.422983
},
{
"reference_full_ms": 40.038979,
"new_load_ms": 3.915465,
"new_encode_ms": 0.420586
}
]
}
],
"warmups": 2,
"draws_per_encoding": 5,
"graph": {
"layers": [
[
{
"relation": "interaction_user",
"direction": "reverse",
"fanout": 1
}
]
],
"table_rows": {
"interactions": 4,
"users": 3
},
"relations": [
"interaction_user"
]
},
"eligibility": "supported",
"exclusion_reasons": [],
"index_marker_sha256": "0ada8dd747a380a67199e9d6138f02178a4f926c2a90ffebea9468fab2d4f087",
"selected_rows": {
"context": {
"interactions": 2,
"users": 2
},
"predict": {
"interactions": 3,
"users": 4
}
},
"selected_edges": {
"context": {
"interaction_user": 2
},
"predict": {
"interaction_user": 3
}
},
"correctness": "passed"
}
Loading
Loading