From 4aceaaa4510403efb1c67ffb77c628e3ac59cb02 Mon Sep 17 00:00:00 2001 From: k-ruthvika Date: Thu, 12 Nov 2026 16:28:27 -0500 Subject: [PATCH 1/5] examples: add Snowflake Kumo quickstart notebooks Signed-off-by: k-ruthvika --- .../kumo_relational_quickstart.ipynb | 1011 +++++++++++++++++ .../snowflake/kumo_tabular_quickstart.ipynb | 414 +++++++ 2 files changed, 1425 insertions(+) create mode 100644 examples/snowflake/kumo_relational_quickstart.ipynb create mode 100644 examples/snowflake/kumo_tabular_quickstart.ipynb diff --git a/examples/snowflake/kumo_relational_quickstart.ipynb b/examples/snowflake/kumo_relational_quickstart.ipynb new file mode 100644 index 000000000..eb0d772c8 --- /dev/null +++ b/examples/snowflake/kumo_relational_quickstart.ipynb @@ -0,0 +1,1011 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "1e4bc018f9b64464", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "# Kumo Relational Quickstart with structured-data-models-client\n", + "\n", + "**Kumo Relational** is a relational foundation model for\n", + "machine learning on enterprise data. With your relational data and a\n", + "Predictive Query Language (PQL) query, you can generate predictions without\n", + "training a task-specific model.\n", + "\n", + "This quickstart uses four tables from Microsoft's **AdventureWorks** OLTP\n", + "sample database: customers, products, sales-order headers, and sales-order\n", + "details. Predictions are submitted to a deployed Kumo Relational NVIDIA NIM through\n", + "`structured-data-models-client`.\n", + "\n", + "Each user is responsible for checking the content of datasets and the applicable licenses and determining if suitable for the intended use." + ] + }, + { + "cell_type": "markdown", + "id": "c333f1688de74b17", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## Introduction\n", + "\n", + "Kumo Relational is grounded in three key world views:\n", + "\n", + "> **1. Enterprise data is a graph.**\n", + "\n", + "Enterprise data is a graph where tables are connected by keys. In this notebook, `sales_order_headers` connects customers to orders, while `sales_order_details` connects each order to its products. Together, these relationships give Kumo Relational the business context surrounding every transaction.\n", + "\n", + "Once we structure enterprise data this way, we can apply pre-trained Relational Graph Transformers to extract insights and patterns.\n", + "\n", + "> **2. With timestamps, we place events on a timeline.**\n", + "\n", + "By placing events on a timeline, we unlock the ability to model how things evolve. We can select any point in time and predict what is likely to happen next from the sequence and patterns in the historical data.\n", + "\n", + "> **3. Machine learning tasks can be described via predictive queries.**\n", + "\n", + "Major machine learning tasks, including regression, classification, forecasting, and recommendation, can be expressed using *Predictive Query Language (PQL)*. A PQL query identifies the outcome to predict, the entity to predict it for, and, for temporal tasks, the future prediction window.\n", + "\n", + "If you know SQL, PQL should feel familiar. Learn more in the [query-writing guide](https://docs.nvidia.com/sdgm/kumo-relational/user-guide/make-predictions/writing-predictive-queries)." + ] + }, + { + "cell_type": "markdown", + "id": "454eac1b23f249df", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "**Let's get started!**" + ] + }, + { + "cell_type": "markdown", + "id": "fc9ae315749f4538", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## Step 1. Install structured-data-models-client\n", + "\n", + "Install the SDK with the relational extra. See [Deploy, Install, and Connect](https://docs.nvidia.com/sdgm/kumo-relational/get-started/sdk-getting-started) for the supported Python versions and platforms." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a24f971d5d724e15", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "%pip install \"structured-data-models-client[relational]\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6aac3912c4a144b8", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "import os\n", + "from pathlib import Path\n", + "\n", + "import pandas as pd\n", + "from sdm_client import (\n", + " KumoRelational,\n", + " PredictiveQuery,\n", + " Relationship,\n", + " Schema,\n", + " SDMClient,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "1e6663f0c2574e5e", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "Build schemas and queries through `sdm_client`'s top-level names: `Schema`, `Relationship`, and `PredictiveQuery`. Submit predictions through `SDMClient`, naming the model with `KumoRelational()`." + ] + }, + { + "cell_type": "markdown", + "id": "40af00956c7342d2", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## Step 2. Configure the Kumo Relational NIM endpoint\n", + "\n", + "Deploy Kumo Relational as an NVIDIA NIM before running this notebook. See\n", + "[Deploy, Install, and Connect](https://docs.nvidia.com/sdgm/kumo-relational/get-started/sdk-getting-started)\n", + "for deployment instructions.\n", + "\n", + "Set `KUMO_RELATIONAL_API_ENDPOINT` to the reachable NIM endpoint. The default below uses a\n", + "local container on port `8000`. Set `KUMO_RELATIONAL_API_KEY` only when an\n", + "authenticating gateway protects the endpoint." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e1bcc8f7a5024ff5", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "NIM_URL = os.environ.get(\n", + " 'KUMO_RELATIONAL_API_ENDPOINT',\n", + " 'http://:8000',\n", + ")\n", + "NIM_API_KEY = os.environ.get('KUMO_RELATIONAL_API_KEY')" + ] + }, + { + "cell_type": "markdown", + "id": "bb2867af180a486f", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## Step 3. Connect to the NIM\n", + "\n", + "An `SDMClient` owns the HTTP connection to one NIM endpoint. Keep credentials\n", + "in environment configuration rather than notebook source." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a1f6054ae0524596", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "client = SDMClient(\n", + " url=NIM_URL,\n", + " api_key=NIM_API_KEY,\n", + ")\n", + "\n", + "print('Ready:', client.health_ready())\n", + "print('Capabilities:', client.capabilities(KumoRelational()))" + ] + }, + { + "cell_type": "markdown", + "id": "f83cc342e84b44f0", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "
\n", + "Security: Never hard-code NIM or gateway credentials in a\n", + "notebook. Use environment variables or your platform's secret manager.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "456e3686d07948be", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## Step 4. Data Preparation\n", + "\n", + "This notebook uses the **AdventureWorks OLTP sample database**, originally\n", + "published by Microsoft as a SQL Server sample. The four source extracts are:\n", + "\n", + "- `Customer.csv`\n", + "- `Product.csv`\n", + "- `SalesOrderHeader.csv`\n", + "- `SalesOrderDetail.csv`\n", + "\n", + "Download the four `*.csv` extracts from the AdventureWorks OLTP install scripts in the\n", + "[Microsoft SQL Server samples repository](https://github.com/microsoft/sql-server-samples/tree/master/samples/databases/adventure-works/oltp-install-script).\n", + "\n", + "> Each user is responsible for checking the content of datasets and the\n", + "> applicable licenses and determining if suitable for the intended use.\n", + "The repository's samples and templates are provided under the\n", + "[MIT License](https://github.com/microsoft/sql-server-samples/blob/master/license.txt).\n", + "\n", + "Set `DATA_LOCATION` to the Snowflake database and schema that hold the\n", + "AdventureWorks tables (`RUTH.ADVENTUREWORKS` in this account). Each table is\n", + "read with the Snowpark session, and only the listed columns that exist in the\n", + "table are kept.\n", + "\n", + "The sales-order header and detail tables remain separate. During preparation,\n", + "only `CustomerID` and `OrderDate` are copied from each header to its detail\n", + "rows using `SalesOrderID`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9b4cbe03709f4914", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "from snowflake.snowpark.context import get_active_session\n", + "\n", + "session = get_active_session()\n", + "\n", + "DATA_LOCATION = '.'\n", + "print('AdventureWorks data:', DATA_LOCATION)\n", + "\n", + "CUSTOMER_COLUMNS = [\n", + " 'CustomerID',\n", + " 'PersonID',\n", + " 'StoreID',\n", + " 'TerritoryID',\n", + " 'AccountNumber',\n", + " 'rowguid',\n", + " 'ModifiedDate',\n", + "]\n", + "\n", + "PRODUCT_COLUMNS = [\n", + " 'ProductID',\n", + " 'Name',\n", + " 'ProductNumber',\n", + " 'MakeFlag',\n", + " 'FinishedGoodsFlag',\n", + " 'Color',\n", + " 'SafetyStockLevel',\n", + " 'ReorderPoint',\n", + " 'StandardCost',\n", + " 'ListPrice',\n", + " 'Size',\n", + " 'SizeUnitMeasureCode',\n", + " 'WeightUnitMeasureCode',\n", + " 'Weight',\n", + " 'DaysToManufacture',\n", + " 'ProductLine',\n", + " 'Class',\n", + " 'Style',\n", + " 'ProductSubcategoryID',\n", + " 'ProductModelID',\n", + " 'SellStartDate',\n", + " 'SellEndDate',\n", + " 'DiscontinuedDate',\n", + " 'rowguid',\n", + " 'ModifiedDate',\n", + "]\n", + "\n", + "SALES_HEADER_COLUMNS = [\n", + " 'SalesOrderID',\n", + " 'RevisionNumber',\n", + " 'OrderDate',\n", + " 'DueDate',\n", + " 'ShipDate',\n", + " 'Status',\n", + " 'OnlineOrderFlag',\n", + " 'SalesOrderNumber',\n", + " 'PurchaseOrderNumber',\n", + " 'AccountNumber',\n", + " 'CustomerID',\n", + " 'SalesPersonID',\n", + " 'TerritoryID',\n", + " 'BillToAddressID',\n", + " 'ShipToAddressID',\n", + " 'ShipMethodID',\n", + " 'CreditCardID',\n", + " 'CreditCardApprovalCode',\n", + " 'CurrencyRateID',\n", + " 'SubTotal',\n", + " 'TaxAmt',\n", + " 'Freight',\n", + " 'TotalDue',\n", + " 'Comment',\n", + " 'rowguid',\n", + " 'ModifiedDate',\n", + "]\n", + "\n", + "SALES_DETAIL_COLUMNS = [\n", + " 'SalesOrderID',\n", + " 'SalesOrderDetailID',\n", + " 'CarrierTrackingNumber',\n", + " 'OrderQty',\n", + " 'ProductID',\n", + " 'SpecialOfferID',\n", + " 'UnitPrice',\n", + " 'UnitPriceDiscount',\n", + " 'LineTotal',\n", + " 'rowguid',\n", + " 'ModifiedDate',\n", + "]\n", + "\n", + "\n", + "def read_headerless_table(filename, columns):\n", + " path = f'{DATA_LOCATION}.\"{Path(filename).stem}\"'\n", + " return session.table(path).to_pandas()\n", + "\n", + "\n", + "customers_df = read_headerless_table('Customer.csv', CUSTOMER_COLUMNS)\n", + "products_df = read_headerless_table('Product.csv', PRODUCT_COLUMNS)\n", + "sales_order_headers_df = read_headerless_table(\n", + " 'SalesOrderHeader.csv', SALES_HEADER_COLUMNS\n", + ")\n", + "sales_order_details_df = read_headerless_table(\n", + " 'SalesOrderDetail.csv', SALES_DETAIL_COLUMNS\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "55240ba7af5f4412", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "Parse the timestamp fields, then enrich each sales-order detail with only its parent order's `CustomerID` and `OrderDate`. The header and detail dataframes remain separate." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5e56b1a7770c46d9", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "timestamp_columns = {\n", + " 'customers': (customers_df, ['ModifiedDate']),\n", + " 'products': (\n", + " products_df,\n", + " ['SellStartDate', 'SellEndDate', 'DiscontinuedDate', 'ModifiedDate'],\n", + " ),\n", + " 'sales_order_headers': (\n", + " sales_order_headers_df,\n", + " ['OrderDate', 'DueDate', 'ShipDate', 'ModifiedDate'],\n", + " ),\n", + " 'sales_order_details': (sales_order_details_df, ['ModifiedDate']),\n", + "}\n", + "\n", + "for _, (table_df, columns) in timestamp_columns.items():\n", + " for column in columns:\n", + " if (\n", + " column in table_df\n", + " ): # skip columns not present in the Snowflake table\n", + " table_df[column] = pd.to_datetime(table_df[column], errors='coerce')\n", + "\n", + "sales_order_details_df = sales_order_details_df.merge(\n", + " sales_order_headers_df[['SalesOrderID', 'CustomerID', 'OrderDate']],\n", + " on='SalesOrderID',\n", + " how='left',\n", + " validate='many_to_one',\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "e383e679775c4356", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## Step 5. Build a schema from DataFrames\n", + "\n", + "`Schema.from_data()` accepts a dictionary of pandas DataFrames and infers\n", + "each table's column types and semantic roles automatically. The AdventureWorks\n", + "relationships are supplied explicitly so the example is reproducible.\n", + "Kumo Relational still infers column data types and semantic metadata, which\n", + "you should review before prediction." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d56d61bdd28e4630", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "schema = Schema.from_data(\n", + " {\n", + " 'customers': customers_df,\n", + " 'products': products_df,\n", + " 'sales_order_headers': sales_order_headers_df,\n", + " 'sales_order_details': sales_order_details_df,\n", + " },\n", + " primary_keys={\n", + " 'customers': 'CustomerID',\n", + " 'products': 'ProductID',\n", + " 'sales_order_headers': 'SalesOrderID',\n", + " 'sales_order_details': 'SalesOrderDetailID',\n", + " },\n", + " time_columns={\n", + " # These two tables contain datetime columns but do not represent\n", + " # event timelines, so no time column is assigned to them.\n", + " 'customers': None,\n", + " 'products': None,\n", + " 'sales_order_headers': 'OrderDate',\n", + " 'sales_order_details': 'OrderDate',\n", + " },\n", + " relationships=[\n", + " Relationship(\n", + " source_table='sales_order_headers',\n", + " source_column='CustomerID',\n", + " target_table='customers',\n", + " target_column='CustomerID',\n", + " ),\n", + " Relationship(\n", + " source_table='sales_order_details',\n", + " source_column='SalesOrderID',\n", + " target_table='sales_order_headers',\n", + " target_column='SalesOrderID',\n", + " ),\n", + " Relationship(\n", + " source_table='sales_order_details',\n", + " source_column='ProductID',\n", + " target_table='products',\n", + " target_column='ProductID',\n", + " ),\n", + " Relationship(\n", + " source_table='sales_order_details',\n", + " source_column='CustomerID',\n", + " target_table='customers',\n", + " target_column='CustomerID',\n", + " ),\n", + " ],\n", + ")\n", + "\n", + "print(schema)\n", + "for name in schema.tables:\n", + " print(schema[name])" + ] + }, + { + "cell_type": "markdown", + "id": "9549e5a69d5e4576", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "Primary-key and relationship columns are automatically assigned the `ID`\n", + "semantic type. Review the remaining columns before prediction:\n", + "\n", + "- Override a column's semantic type with\n", + " `schema[\"table_name\"][\"column_name\"].stype = \"\"`.\n", + "- Available semantic types: `\"numerical\"`, `\"categorical\"`,\n", + " `\"multicategorical\"`, `\"ID\"`, `\"text\"`, `\"timestamp\"`.\n", + "\n", + "For this known AdventureWorks schema, review identifier-like strings before\n", + "prediction. Account numbers, product numbers, order numbers, approval codes,\n", + "`rowguid` columns, and `CarrierTrackingNumber` identify records rather than\n", + "categories, and several exceed the NIM's 10,000-value categorical cardinality\n", + "limit in the full extracts, so this example marks them `\"ID\"`. Product names\n", + "are typed `\"text\"` to reflect their content, though Kumo Relational currently\n", + "ignores `text` features for prediction; see\n", + "[Data Types](https://docs.nvidia.com/sdgm/kumo-relational/user-guide/prepare-data/data-types).\n", + "Order quantity is a measured numerical value." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "343f65d6fa1e4421", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "# Correct semantic types that are specific to this known schema. These values\n", + "# identify business records rather than categories or free-text documents.\n", + "id_columns = {\n", + " 'customers': ['AccountNumber', 'rowguid'],\n", + " 'products': ['ProductNumber', 'rowguid'],\n", + " 'sales_order_headers': [\n", + " 'SalesOrderNumber',\n", + " 'PurchaseOrderNumber',\n", + " 'AccountNumber',\n", + " 'CreditCardApprovalCode',\n", + " 'rowguid',\n", + " ],\n", + " 'sales_order_details': ['CarrierTrackingNumber', 'rowguid'],\n", + "}\n", + "\n", + "for table_name, column_names in id_columns.items():\n", + " for column_name in column_names:\n", + " if (\n", + " column_name in schema[table_name].columns\n", + " ): # skip columns not in the Snowflake table\n", + " schema[table_name][column_name].stype = 'ID'\n", + "\n", + "# Product names carry natural-language meaning; order quantity is measured.\n", + "# This NIM does not read the 'text' stype, so Name is typed categorical.\n", + "schema['products']['Name'].stype = 'categorical'\n", + "schema['sales_order_details']['OrderQty'].stype = 'numerical'" + ] + }, + { + "cell_type": "markdown", + "id": "b8d9bde03c0043dd", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "Call `schema.check_consistent()` after all metadata changes. An incompatible\n", + "semantic type, a missing column, or a primary key that is also configured as\n", + "a time column raises an error." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "76207d2f8c1440bc", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "schema.check_consistent()\n", + "print('Schema OK.')\n", + "for name in schema.tables:\n", + " print(schema[name])" + ] + }, + { + "cell_type": "markdown", + "id": "daf381f78c0c42a3", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## Step 6. Submit prediction requests\n", + "\n", + "The schema is a reusable description of the data. For each prediction:\n", + "\n", + "1. Write the task in PQL.\n", + "2. Build a `PredictiveQuery` from the query text and the schema.\n", + "3. Call `.materialize(indices=...)` to gather a `Request`.\n", + "4. Call `client.predict(request, model=KumoRelational())`.\n", + "\n", + "The result's `to_pandas()` method returns a pandas DataFrame." + ] + }, + { + "cell_type": "markdown", + "id": "2018ab497db1403f", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "**Note:** AdventureWorks is sample data, and the queries and results below are intended for demonstration. Benchmark Kumo Relational on data representative of your own use case before drawing performance conclusions." + ] + }, + { + "cell_type": "markdown", + "id": "9a3b840a14124567", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "#### PQL: How you describe a prediction task\n", + "\n", + "Learn more in the [query-writing guide](https://docs.nvidia.com/sdgm/kumo-relational/user-guide/make-predictions/writing-predictive-queries).\n", + "\n", + "The short reference below covers the entity, target, aggregation,\n", + "prediction-window, and condition syntax used by the examples in this\n", + "quickstart." + ] + }, + { + "cell_type": "markdown", + "id": "77a82381c7c543fb", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "
\n", + "💡 Click here for a short introduction to Predictive Query!\n", + "\n", + "**Entities** can be specified in three ways:\n", + "\n", + "* `FOR .=1` embeds one entity ID in PQL.\n", + "* `FOR . IN (1, 2, 3)` embeds several\n", + " entity IDs in PQL.\n", + "* `FOR EACH .` declares the entity column,\n", + " while `materialize(indices=[1, 2, 3])` supplies the entity IDs separately.\n", + "\n", + "**Preferred:** For application code, use `FOR EACH` and pass primary-key values\n", + "through `materialize()`'s `indices` argument. This keeps the prediction task separate\n", + "from the entity batch and makes the same PQL reusable. Despite its name,\n", + "`indices` contains source primary-key values, not DataFrame row numbers.\n", + "If `indices` is provided, it overrides entity IDs embedded in the PQL.\n", + "\n", + "**Targets** can be specified in two common ways:\n", + "\n", + "* `PREDICT .` imputes a missing value.\n", + "* `PREDICT (, , , )` predicts an aggregate in a future window.\n", + "\n", + "Supported aggregations include `COUNT`, `SUM`, `AVG`, `MIN`, `MAX`, and `LIST_DISTINCT`. For example, `PREDICT COUNT(sales_order_headers.*, 0, 7, days)` predicts the number of orders in the next seven days.\n", + "\n", + "Conditions can turn an aggregate into an event. For example, appending `=0` predicts the probability that no orders occur in the specified window.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "7bdf5317192d401c", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "### Example 1A: Forecast 30-day product demand\n", + "\n", + "Predict how many units of product `ProductID=870` will be ordered in the next 30 days." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6f6a049b1a2245c1", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "query = PredictiveQuery(\n", + " 'PREDICT SUM(sales_order_details.OrderQty, 0, 30, days) FOR EACH products.ProductID',\n", + " schema,\n", + ")\n", + "\n", + "request = query.materialize(indices=[870])\n", + "result = client.predict(request, model=KumoRelational())\n", + "result.to_pandas()" + ] + }, + { + "cell_type": "markdown", + "id": "c909c8c8ce3f413a", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "How to interpret the result:\n", + "\n", + "1. `id`: the product with `ProductID=870`\n", + "2. `prediction`: the predicted number of units ordered during the next 30 days\n", + "\n", + "This result can support inventory and demand planning." + ] + }, + { + "cell_type": "markdown", + "id": "bb42f92f2c2c4126", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "### Example 1B: Forecast from a historical anchor time\n", + "\n", + "By default, predictions use the maximum timestamp in the temporal schema. Set a historical `anchor_time` to simulate what the prediction would have looked like at that point in time. Kumo Relational uses only information available before the anchor time." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fb4d3b45bdfb439e", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "# Pin the anchor instead of letting it be derived. It is taken from the\n", + "# data rather than written as a literal: AdventureWorks is re-dated\n", + "# upstream, so a hardcoded date stops matching the file you just read.\n", + "anchor_time = sales_order_headers_df['OrderDate'].max() - pd.Timedelta(days=60)\n", + "\n", + "request = query.materialize(indices=[870], anchor_time=anchor_time)\n", + "result = client.predict(request, model=KumoRelational())\n", + "result.to_pandas()" + ] + }, + { + "cell_type": "markdown", + "id": "1afe70c0bf7c4654", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "### Example 1C: Forecast for multiple steps" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "43a67a8a138949ec", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "query_multi_step = PredictiveQuery(\n", + " 'PREDICT SUM(sales_order_details.OrderQty, 0, 30, days) '\n", + " 'FORECAST 3 TIMEFRAMES FOR EACH products.ProductID',\n", + " schema,\n", + ")\n", + "\n", + "request = query_multi_step.materialize(indices=[870])\n", + "result = client.predict(request, model=KumoRelational())\n", + "result.to_pandas()" + ] + }, + { + "cell_type": "markdown", + "id": "2458e94f330c4ce0", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "A multi-step forecast returns one row per timeframe.\n", + "`forecast_step` is the one-based sequence number and `prediction` is the\n", + "forecast value for that period." + ] + }, + { + "cell_type": "markdown", + "id": "f4ecd48298c04830", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "### Example 2: Predict customer inactivity\n", + "\n", + "Predict the likelihood that customers `CustomerID=11176` and `CustomerID=11091` will place zero orders in the next 90 days." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "68b73fba28a54daf", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "query_inactivity = PredictiveQuery(\n", + " 'PREDICT COUNT(sales_order_headers.*, 0, 90, days)=0 FOR EACH customers.CustomerID',\n", + " schema,\n", + ")\n", + "\n", + "request = query_inactivity.materialize(indices=[11176, 11091])\n", + "result = client.predict(request, model=KumoRelational())\n", + "result.to_pandas()" + ] + }, + { + "cell_type": "markdown", + "id": "89c2f6703bdc4af7", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "How to interpret the result:\n", + "\n", + "1. `id`: the customer represented by each `CustomerID`\n", + "2. `prediction`: whether the zero-order event is predicted to occur\n", + "3. `probability_*` columns: the estimated likelihood of each outcome\n", + "\n", + "This result can support retention or re-engagement workflows." + ] + }, + { + "cell_type": "markdown", + "id": "f4a9687b1c554efd", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "### Example 3: Recommend products\n", + "\n", + "Rank which of a small candidate set of products customer `CustomerID=11176` is most likely to order in the next 30 days.\n", + "\n", + "Ranking runs as one classification call per candidate, so the candidate set is\n", + "named explicitly with `candidates=` rather than left for the client to infer.\n", + "This example candidates the 15 most frequently ordered products so each has\n", + "enough purchase history for the model to learn from; an arbitrary or rarely\n", + "ordered candidate set can fail with \"nothing to rank\" because its context\n", + "holds only one outcome. See\n", + "[Prediction Types](https://docs.nvidia.com/sdgm/kumo-relational/user-guide/make-predictions/prediction-types#recommendations-ranking)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "62cadc402ad84193", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "candidate_product_ids = (\n", + " sales_order_details_df['ProductID']\n", + " .value_counts()\n", + " .head(15)\n", + " .index.to_series()\n", + " .reset_index(drop=True)\n", + ")\n", + "\n", + "query_rank = PredictiveQuery(\n", + " 'PREDICT LIST_DISTINCT(sales_order_details.ProductID, 0, 30, days) '\n", + " 'RANK TOP 10 FOR EACH customers.CustomerID',\n", + " schema,\n", + ")\n", + "\n", + "request = query_rank.materialize(\n", + " indices=[11176],\n", + " candidates=candidate_product_ids,\n", + " num_context_samples=5000,\n", + ")\n", + "result = client.predict(request, model=KumoRelational())\n", + "print(result.to_pandas())\n", + "print('Unscored candidates:', result.unscored_candidates)" + ] + }, + { + "cell_type": "markdown", + "id": "c591ab8a0d9948f8", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "How to interpret the result:\n", + "\n", + "1. `id`: the customer with `CustomerID=11176`\n", + "2. `rankings`: the ranked candidates, each with a `candidate` product ID and a `score`\n", + "\n", + "A candidate whose sampled context holds only one outcome can't be scored and\n", + "is left out rather than failing the whole ranking; check\n", + "`result.unscored_candidates` for which ones were skipped." + ] + }, + { + "cell_type": "markdown", + "id": "9ccdfb90708143fb", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "### Example 4: Infer a missing product attribute\n", + "\n", + "Predict the missing `Color` value for product `ProductID=870`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a872685251844a6c", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "query_attribute = PredictiveQuery(\n", + " 'PREDICT products.Color FOR EACH products.ProductID',\n", + " schema,\n", + ")\n", + "\n", + "request = query_attribute.materialize(indices=[870])\n", + "result = client.predict(request, model=KumoRelational())\n", + "result.to_pandas()" + ] + }, + { + "cell_type": "markdown", + "id": "8f2a1fe04ce34513", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "How to interpret the result:\n", + "\n", + "1. `id`: the product with `ProductID=870`\n", + "2. `prediction`: the predicted color category\n", + "3. `probability_*` columns: the estimated likelihood of each category\n", + "\n", + "Attribute inference can help enrich incomplete catalog records, but predicted values should be reviewed before being written back to a source system." + ] + }, + { + "cell_type": "markdown", + "id": "863c7f0fa50d4557", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "### Example 5: Predict customer churn\n", + "\n", + "Predict the likelihood that customers `CustomerID=11176` and `CustomerID=11091` will place zero orders in the next 90 days, reusing Example 2's query with one addition. The `WHERE` condition does not filter which entities are scored, both `indices` are still predicted; it filters which rows qualify as labeled context examples, restricting the model's learned comparison to customers who themselves placed at least one order in the previous 90 days." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1574bbb37ce747d6", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "query_churn = PredictiveQuery(\n", + " 'PREDICT COUNT(sales_order_headers.*, 0, 90, days) = 0 '\n", + " 'FOR EACH customers.CustomerID '\n", + " 'WHERE COUNT(sales_order_headers.*, -90, 0, days) > 0',\n", + " schema,\n", + ")\n", + "\n", + "request = query_churn.materialize(indices=[11176, 11091])\n", + "result = client.predict(request, model=KumoRelational())\n", + "result.to_pandas()" + ] + }, + { + "cell_type": "markdown", + "id": "f217317179fc486f", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "How to interpret the result:\n", + "\n", + "1. `id`: the customer represented by each `CustomerID`\n", + "2. `prediction`: whether the customer is predicted to place zero orders during the next 90 days\n", + "3. `probability_*` columns: the estimated likelihood of each outcome\n", + "\n", + "This churn prediction can support retention or re-engagement workflows." + ] + }, + { + "cell_type": "markdown", + "id": "0af9353c28134884", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## Step 7. Close the client\n", + "\n", + "Close the client when the notebook is finished to release its HTTP connection." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6f15f0bff032428e", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "client.close()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "pygments_lexer": "ipython3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/snowflake/kumo_tabular_quickstart.ipynb b/examples/snowflake/kumo_tabular_quickstart.ipynb new file mode 100644 index 000000000..56bc4ba10 --- /dev/null +++ b/examples/snowflake/kumo_tabular_quickstart.ipynb @@ -0,0 +1,414 @@ +{ + "cells": [ + { + "attachments": { + "logo.png": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAIAAAACGCAYAAAAVZ4LWAAAehUlEQVR4nO1deXAb53V/i5MAD4AkeIEgCZISdVkOJSu21cgRaTtOnDgduXWatNOZyHHbpE4ykdv8k5lOLWWmnUkvy9NJp3GT1G4mySRNY3smTRw7iahEPuRDpixblkiJBG+KBEkcJG7s9r0lF1wAi3uxC1L9zWBA7AIgsN/73vd7v/feBx3chHh2/ISVDQS+gn8eZzRw8g/2PPUM3KRg4CbDs5c//xWW5U7iN7cKxziAYa1G89iDe/59CG4y3DQG8NP3/+KzHAsn8U9npucwHDPEaJmH0RBccJNg2xvAs+9/YYCLc49zDDeQ/6uYpzUa5tTNYAjb1gBw4J0syz6Bfx6DYsCBB6/OaY3Z/OSD3ac9sE2x7QxgneAFceC54yAH0BAYLZzYrkRx2xhAgtlzcEJM8GSES8PAYw/ufeo52EbYFgaQD8GTCxtE8dR2iRi2tAEQwcN1/j9BgYFPx/YgilvSAJ5979F+jos/URizLxM4JIrV5lNblShqYIvh0R9D/9jC6/9aEYOP4Jj4wwvuSz+ELQodbDFo4mC9PvfmkanF94b3OD4MNktXP6gET2jq7FpksZ8D7mOwRVHxBsBLtxxn/cN9T50SH4/Egv0XXb8Eg840fGjnMZvJUOsAhbAWXTrvCU62c8AeTT3H6w9x9isareb5rUAUK5YDSDB7FyVufjP81ATLwZnU59eabOcO9j6wX6c1WKBMCMW8wytBF8S5WJrX6Wy9qz41DN0K0nLFGUAu6TYaCw9fmnipf2V1VuIs42219ry9y3HXATkNIc6GpxfXRmZiXOQOqfM1hmawGjs8mfWHyo0YKsYACpVuV1bn4PLUEIQifomzjLe37bZLzuaDR6AEcBD3LgXGhkMx31Gp8yadFSxVHaDTGPJ5s4qUllU3gFKl27nlERiZfRVi8XDaOQ1opvc7P+IuhiguB8eGglHPASR4aZ7EqK2FOmMbGHW1UDAqTFpWzQDklG5j8QhMLl6CKfe7koZARDHfiMEfmTvnC807keClkUotYwRrlQNMelmU5oqQllUxgHJJt2QIIzOvwNzKiOR5IoroEZxSEQMRvKXgmInj2F2p5zQ4ZeuMdn6tlxtqS8uKGoBS0m0QecH7U2dBmigCtFp7hwSiSARvYe2KW4rZ08DXGFr4gae/ywt1iKIiBqCWdEtEcfzGW5ApYtjt/OBrqOR9VOq1Zn0jP+vzInhyQmFpuawGsM7sucdly80XCSKKY2gIqRHDzq7+YY1GmzTzieBZkdnrtSZQDRsRQ6r4VQ6UzQCePPvhjzsad3+/TLn5onDR9XNwe6cTj/f33rsYYVeahMdM3Ajt9bdApSAaD51fWRv5zF8efs0FZULZkkFXp397+5l3vr3qWrhwDlTGDe8ouJbPg8lcnXTcyNQ0iR9fmXwF3rz2LKyGlkFNkP5AYejC2uXdMYg5oYyQzQCI4FFoJz7GYihFiZvfvfe9Ybd3YhgUhts/AWPuVyHK+ECnS0576LRGdPPGtNd41xbh/NWfwNvj/0v5BlAaFIbO+i5CILoykKpB0JJKN5ARJS8BSdItrl3IZE8+uPdbTz76faDa+8fFz8Xw6/yB3gfay524WQuvwIJ/BPSG5EEPhFZhau4qdNj2Q0/rbWgEBgjH/OALz0E47oer428lPV+r0UFrQy/0tNxOWgKUE6IEU9K10QAz6LAeciVzKYwYzKbH5CCKRRtADunWNbl4cXh09rykrFuuxA3N2FnPu6AzSDs2DVcF9aZOMsS0c2QIb47+TFJa1uuqoLPpFkBpGeRGtgSTBg2wydT3tF5jOpbGpWSSlgs2AF7BWws8jq88keu5HgzDxrKEYXImbiaW3gJGy+JFSx/8QqTbbNIyGcJuxxFotvRAqSD9wR0Yc0XZgGS+otrQCBZDB36fHPpDidJy3gZQinS76HXBKF7UoMTsIr2+u+2gq9jEzezKexDhVtPWeAJJtzTwdDELQS5pucZkhT77EaivsUOhIIK3Epx4m9Z4qfNmlJnrjA7UH4xQIFxo/A8XqijmZQBySbf87JrD2RUrPXGzvDod9IanTHp9ulAjl3SbS1puqGuHXfYPrZmN1up83i9bgqkKvVOtwY5eqgZKQaHSclYDIIIXx3UenyRb2VUshrPLjbNr6V1JQ6DEza3dHzVZzM27pF6fTbolCAMvp3RbiLQsdT5bgolmOnkpUh7lRX7SsqQBKCHd0kV1IT+YXc4vcZNrzVRCus0lLXfY9g33tf9eonYgF8Gr07dCjbEFyoyT2YhikgGoId1mn13rRLGtuRuy5eaVlm4zScsEWsp2th8e1xpjtVm9lL45N8GTC1mkZd4AFGirygmKGIh9+4PutHO7um9LO0YEr8HUVVxRhkwYm39LkijWVFuH25t70wafyGitoa0YgicXXKkbYjA/euf4n6Pb/IdK0eyliKLYAOJsHGzm3oKZfblARHF09jVcyq4kjjXUtl1tstkTHMagNaOXcuK9igkmEYgoGhntH39i37/Na86++/Q/Xp195SJUAEi6DbJLSAQzr+PhcBDedf0aFrxjUAlY8I1CdW3ywNYZO5MI7Kz7Orw3+RuIqiAtpwK5nW9+7bJuzPf6bnpMwbNl2v3u0Vn35eldjiOz9sbdt4PCEEu3BkNuErca9MAl16/AUt2EwsxRqKlqAKVBxroanef1B01KSmU9AoklHVv2zcDL7/8QpeUe6G25A0UlZb0BDfxKaPJCMLqMXAqOaDboX0I9ocTN+9O/dVyff2N4d8eHmaa6rg9AmSGWblN1e0KTxQk77YfxYhlRr5/FBMlS0nkhcdNc343x+JGy6/UEGnj6LGSoUgmmDtst0NW8h88teELTGAFsLmVxNgYz7hFY8Ezic/B5TfJLy1JYD0Nn9+PAD6SeYx79Ae2RlI71xM0nHCYDqh1lQDbp1uv1gr1+DyptbUnHSa9f8I/ByOT5tNeUO3GTKcEkwOfxA4aAfIIp6XWRJYwY3oSF5Ym018gpLUshGPVeRPGpUUp/oCTTlw+/OZTRAAQ01LSf3e+8F0UOYx3IgFkPSrdsadItScsUMSiRuCEvNee9DFq99Pl8egPibBQmFt7JLi23fwjqq+WZazTwnvAkF2cjGQW8vA1gA97W+t63d7UfOVisIZDr9EdmQU7pNlvipspQg8vHnSXNLrkSTAJI8xjH0DGbtNzXdhiqi+Q0OOAz7sD18UximRiFGsDGiwpP3JDrvOG/kpHclSrdliNxU44Ekxi5pOV2W19BRFEgeIHo8gDkiaIMQIBWq7+6r/PuUDaiqLR0m2t2ddT+HnS17QajKXNDNMNW8cUh3mh6iFmO3gCSlkcziF8Mw/gcjfveFkvLUlgOjg9tMPuCUuolGYAAStzs777PbDW39AnHhHSnWtJtttml19RAt/UI2FsduKZvunWGM4AvdENy4AnlSDCJkUtalvK62RJM+UAWAxBQg4mbW533dsdgbdwXmtsvNfBKS7fZZhcZws7Gu6HN3h4Ph0PaxfA7ku+hdG9AJmmZILS3VZutTC6Clw9kNQBCJv1bjjWzFGSbXQd7fx/W2Jm04yU1f5YI4jTrhnBJ8rxUXqQYCAZQth1ClG2ryoy2hj7+lm12CaiEBBPpCH3th6EDQ9lsnEYuyFYWbqvauV/82OtfAf/asqqDL0ZdtQ1abJ1Jxxgm+evP3BhFvT4KlQAStqqrazEkLG+OTj4PwFBym008DEfX4MriyzDtfg+6Ww6VTe3KBbF0mxrPp1bDqCEtSyERhup1RKqhnCj7JlFC4oZEjp0ociiVuMmVYGqr7wOz0QKRsIfX7cVYWBmHJe+UYj0BAsQJJp1C+3cptksYZcPO+8o/u8TSrZRuT4LQno6jid6AJl0fVRvxiRsx1hM3VzFxM1G2ngAB3sAcLAVckgmmckPxbeLEs2t3+1GQE4J0K47xBRCzb7D2grk5fU2lHT/oFrNzGDFcSCKK0VgIrs+9CTNLV0qWllNRaBq8HFBln0C5Z5d4zUzltYWEoR1N+zFi2CUpLYciq/xSVkpPgIBcXkpJ6JAIPYNCwGdBBZQ6u5Jy8ym2LIShNPiFgMIw6huk0FEqDCNOc+H6z4rmNFPLF4HTRCS9lILwgp6bpz+0b/wPPPfBT8PzSOD3QAmNH1azA3TGzSiAGjGDIX9eryXxY8EzBh7vEqZWHTlnBUm3i6vXIcwtYV4iPcwk9a7B1ANVuuIz2NQ5TAUpdAuEvWlCUjDsh/mVazidTVBrbgCNNnuPDRvTwaz/Il7xKOn8kC98q0s4USKJx7b64j0PDw6erDPVHvv8ba/ypCfxSabmpgauuV944sXRk/3e0EzB7+u03QHG2s0yKPcKEhvPLBSDTIkbhtPiQIRADek2P2m5Ne1ctgRTPpiaG8HJtGl8xSqBXTWDsLP249+1Ww78dX19faJHgJmbm3OirkzNnseFg+/M/zf8zvUEFGIIchoAQZy40en0EMO3XgzhuswF0p6rpHSbTVo2a5thV+sgNNgsvJcikrccuQKloFQDsFXdAvssn8b7ffxjjm/hh9P21s5voCcKMedeG/pde5u9W683JpWjhOM+eH3qO/D69Hf4UqxckNsABJAh9DkPSA68WtJtrhqE3oZBVmfxybLIF2sAZl0TfKDhc2A3pe9uG2PjFw0a7Zfs9q5zunAodGRsfAxq6yxDLbaWg1qdll84jXh3l/MxuL3jEXhp9BR6hZ+AGoiyq2mDTwTPYuxQLcEkEMVOjBqkEjdh1otCrjp7cOo0Zuivf4R3+angOHYmFAkuIg/ojzIMv74mrNTv8w5cGxvhZudnh8QvIkN4YPc/wxfvfBn6bPeB2ojH4qCJV1dEY4iQuNlhT56VWq3yoR0N/F7rp+ETjqfSBp8GPhwJnguFg+00+EmvS3kfCxnCiN833dRoc9U32BJFCMTOH7rlP2DS8xr8FvkB3auBcDQI1+aH+fBMSWlZCgnp1phCVhllZz8N+F7LZ3i3LwbHcb5oPHohHosewIeSlVmS6xS+0LHgXjyCHmF4ddWf1DXUab0T/rT/R+gV/gmNInMVa9MG6SgXeGkZEzeXJl9SfDMnkm5p86kQ5y67dNtcszfjOSJ497d/Cw41fjlt8HHQccYHOLwfgCzlYlk/Pbrb/pnZaRRa9Ocd7R0OMVG8tfVT/O0NJInkEQR0VQ/CHnRFF6a/D1fgJSg3yiktp0IN6fYjO/4WAg4/vHTt67Cwepk/ZtE7ob/hkQSzFyMej7weicZQLODyKtzNy3wjkegdRBSNVVXnOuydtwpEkfBBxyNwa9unYHTpRWip240zX/mNFsuduFFbuu2yHoY/O/QLPjy31NgkCR4x+2g0xOEaX1BrX0HfhiIGXBa8UhHDLc0PgdooR+KmQqRbHuRxU0EELxINj7MsW9QeS8V8K54oXh8f9a0suyV3AeVY9moszhYuJ8oEIXFzfuTHRe/6SQkm2l2U0cUkG0OUwuKi+614PO5NPU4ELxIJniVmX+zgE4r+ZgJRHBm9Mi0QRTw2zYcbkeCuO9u/0P7Q3qd5aVYtUOKGiOLF8RcgGAzl9Zog5jBcS68Dqw0pnpsXwxo/CH2Rr4I22HLbtesjMDs7MySci8QiQzzBY9mSSU/J35AMAYmiw2QyvdjU1ETrT8IaO+puh0cO/Aredz8Pr059E7zhaVADbt8keFd/KtkTIGBTur2KaWX16hjrdbugMTwAVbGk+kWLf9U3cHXUP9nZ0UGPB0AmyGni1DKTVG1BRZfUC3jQ8Sewt/UBPmJ4ffq7KC37QGmQojiy/AKMe2pgT/PHoKllXT/YTDC9BWrCrN2Qbs3r0u3KyjK4lxYB3fvmkziuE2SGbAaA6lfSh4tGY2A2VYN2YyMkSs2StExRw0vXTsElFaVlkrXNi81waOeD7FLkXY1UnkEp6FHBu7X+c+CsuTvpeH19A5jMJnL9eC3LV6ksG7tBopQkyQUCazDuus5bsRhkCJ9EafnRO8+pKi0H4guo4s2pNvg08HswS/cxFHJSBz8SiYDXt0IMv+wEtKwsh9zX0pIbvF4PNDQ0Qr1100aoP5Ck5QnPq5h6Pq2KtMyolLARxLJqXXKjaQxz3msB6krWKiY0KUJz6YstLNwAn88LjY02qKneTN+SyNHVf3ijBuE0eEPKEUWlNXubcR8csn05beAJfr+PrypC1RWUhKJxTigUgpmZaTCbzdDS3IpfdnO/PEFapvoDMoRMRFHFPfaKBkm3RPCkVNK1NT9FUqpFHqooHIFAAPnBGFCegbyDGLcjSaTU813O9N3od9Q+ALfb/goqEY8c+jmfKBODmP1tjV+Ce+3/kjb4xJF8Pg/Oeg2gogpqgXqPvLgUlu0Xt7MBBST+QlgsFmiotyWEFyFi2N/6EO8N2qqdiTWTOEMloqVmH58lpc938cYPYIflCJ+fTwURvGBojV/j9Qq7+2RwvDKm04DRyXIhnG7MCTUMgYjiysoKroF+sFrrobHBljhHRJEihq0EntPgLRUo58Iqunta49VqAtnABHKfkzu69/CsWzM4OOi5Z/D+kxrg+quMRtV+4YuWArd7Ea6OvA/LK5n1+1i8vM2SxYKWtUxAeRwi0ZDiBC8FMY1O+82dPXudOPhPCwd5nzs7O3mMBfr9H8ZJosPc/Cxq5+qJI4uLNzB8XISWllaoq113SiwbxwsZAnv1Abiv9+/gtelv8k0haqOa7Ybm2D0wNT0BJpMZWlvaEjMcs3Q482OgNvQ6PS6vfB77i1OzrnYdY3ysra3NReeYq6Pv/X11dc3XUl8URIu+sTCPFz2c1z+pq7OgC99cQWizR7qV/OH1erDb7TxTFoOihAvz34O35/4rrbs3XxzacQz88SkoBgZogNbo/VDHplfsoN7hq62rkWVfxRs3bkA4vDkGnZ35q8FUm6jXGdLCXQ44r0Fruq2lpeW6Znpm6mtSpV8mDNWczh7eovUqZsXII6UOPsFksMJAz1fhi4df4cmiUliXbh+GT3Z9G/qa7uFnVyowRauqr6ffIagymsGgN6YNPl8qFgr6fYFlPqukWz+4XvpFFT+pPQIWi5W/kUumtTkpOaEC6P9XVZk3cwyadWmZwsZfXfs6jLhfhHKABn5H7SdhR90DYNCs/0SQcG3IU3o8K4nnaqX61RQADbZBXyUpH/MVQ5Fw42apWMpm0QShRwCFmrOYNj0gLv1qbGzCBEUjT9RWPOr9tCqFUfPz84pKy7mk25qa6iQDwBmoLM1naBcxo2Q5OseyI6EoEjpO+nefJH07MtqjQumXvdU+IBwny2pubuEzVSTtUlijBpSSlkm6JQXPauhOO6eUdJs1GcRQEytP8NJO5Vsqlm1xz9gjQMSsvd3BE8VFXBpSIwYiHjpd+ZfBfKRl6nEstAahkqTbVoyElpaXeWMXg2Y7rfGpEPUCDODDnLtP52R3G6VfjsVl91V7a3uopqY2sT0sEcVOcxev6AVDAX7QdWiRSidZBGkZPxtvCOJSLqEG4Y2NHEM2kHRLrj41Pbv+P9bQ80QVV++0eE3bWu1gQ083Nz/HEzwpckeIxiJn8TOSqx/I8+3zTwZxLLeLiCLyguGujq4mMVGkC083tZGvtJxajCJF8ARUinRLlVWdHV2S54jZR2JRJ45SwTWCBcd3FDEIPQJSXcUCYtHoJN7JXsKUC2JpWYooUsRw64YhECjBRLM+deAFgqe2dIsh27Vqc80OqXOiXoCiq4KLDvA3Ioa0HoE4Gx+ORMK26prqzhiqYGtra6AGBKK4vLzEK4pSRDHCrqUNPMGLWbr1ogz1ZjwZMhkximk7UGs47+jobDfoDfzG0DzBi4SmWY67A0pEqQoPTxTx5m1rbvu13qRvFMINcr+NjY1QW1vLz0ixmqUkyBCIKBKbFkvLhDR3vyHdqqzZ8woqDb6guURj0TvGx68j2a0+29zcpN1g9rL8vIhcEp/F4/c0NlU1pcWapI/X1Vr5SGFh8YZqhkAXc25ultcxWlvsfOQggLaHjYn24VELxOyJSNehs4pGohBIia6Q3xxlWXnb4sum8VKoRIqdZmM/XrRecHb18PWB7iU3z6jVAEnLQuIG2XUoFo9Ugcog70SxvKBu0rWimx9J7Twm5sqpvpatIsjn8/EzLpKSTCLp1NnVzQs4arZckUeKRIOybJVfLHjp1lAFRoMpMfgCKJrhuDivuZQTZc3yUFhGt/r6+qSwjKRyG0nLyNCpalgtaZkB9QyQwjqdxE+RBUNBdP9hPuTUa8rPRRRJ8xEJJGJD8Xlz0+ZWamQI69Jy/Ya0vAqKgmEU3w48k1imlt6gWJ43W3xOs6G9vYN3e8QP1CxGKRcy5ebV6AUQQ3EfKMTnE5PjackkIj6kdtntDslcAl08JXIMxSATp6FjRqNJUr71+b18uRiFnWrxIdUqPYRETlVVFa91ixM5tSgr001okCQICSY2ro6wlAvEaSx1Ft6DEVKZvRhCQkmvV3ejaIJOw3GDGGQ8jebZBSqADEGUyOGQKCamCaWdqdSMLqbSCaZiQEsZGTN5Oam9BdRKKEngeR1UDdMfmsHB+4fuGbzfiSb5MN8joBIoWhgbv+anjRDEO2IQURQPPsZtCjPFwpE6+PzGGeHAeZJEVB78s3gxB3f27D3W3d3N7xecWHjQCJ6mHgH8tKfUMgS8UHW0EQLtiLEksf0Mzp5zLBv3G42V1R5GEQ5+5mGMYobFx6n4cn0bl4BDDt2+BEzgHHoQB35gZ/eeIfEJSb965swZKweh0xzD5P07AiaTabipaVMKlqMqGD/ctL29w43vTYKSDS9p4lcyaemgqKKU3vmOjk4oZWVZW13jv2NMVPptNFads7fam/DDz8XZGG3QWFKzTSlVwQB819fJnd17MxZCSFJPaha5e/D+48gPunFangWVgO7eMTMz1Y+Zr37x4BOIPNIAtrS0KR4ZECchchfG2D2WUvcfDoeOzC/MBXHwBwDUabkDGnj05DrG5Mw2+ISsNBT5gQvvBs6c+cUAEsWTOF3KuxNjnhCSJpRnsFoM6xEDqokrZa5aJi6ix3BOYPbtGK4SsSM1M1Ah2gVOmmf0DHOyu2evK5/n5xV8bhDFgXWiyE2Aigij6/d4PFShlDgmSMs93TuSBCY5QQyfau1Twzr636RwVgDO6hjdgb6evce7u/e48n1RQeoDEcWNiOGUmhEDzfSx8VE+vSyGIC33dPdCTXUNyAHyNDTwqbp9NLq+jQtLCRsVmT0SzYsbzH6gu7tvuNDXFyU/rTeTbkQM0jgbCAafgTKCRQ9AhnB9bDSpJp8gSMsdjk4+7VsMaJkhBS9VvqUuX6/fwxePKCHdGo2Gb2TwutTl+3Bfz77+VGZfCIrWHxNdxUgUGY57ZvNDrYcb0XDYBQqARBfqzCFpea1AaVkKYulWk/LbwiTdYkiH55STbq31DS9ooKpf5HUFgtcv7vItFiVrkRtE8fiFyxe6cOBV4wcUFk5vSMsYhqFbziwtSxFFYU9DSek24EdXr550S5MN705ieH4aBz7U3dOd37aneUC2b3Rw70FVyaEAMoQxlJapFrGluS3pZ+UEaVmIGAhCgkkqN0/MPoZrPa3x6m8VnTAEWVG274VvPKQmUaS0s0AU46JNJYSIgaqS6O/MBM+DxsGpLt3iDHVBGVE2A6DQUW1pWSCKKNOGl5eWkrpFyd0bDcnlgNRWFY4EzsfiUbUrgxNcqpCQrhgokmIrRloWQyx/kg5wY2EBigG6+2l7W/uSuL2NkPLbOkUF9ZOTk4m/U2XxApBTupUbiuZYUVF0bqSeC1IU5TIAAdTe1trcxpAhiNqqHFACSjQAYvandRrzaSFLpxQUpbWVIi0LG2J0dnadLaafTk4UKt3KDVXiGuIHeDfw6zO/OA7rhqBKMQrHsVUqFpqQdHuiGPVOTqga3VSKtKwwJkqRbuVGJYS3+UjL2wG8dEv79JUi3cqNijAAQgZpOQ1xln0ZKhGZs6SySrdyo2IMQAARxY1ilMGUYhS+nm15cfFvoAIhVVdJBI8vyujdd1Jpdp8vKr7U9syZXx7rcHbAju69z60/xgiCYc6ADOjo6DiPblmWWj107fy1JM3DVKP7oxab7cVyizhyQP3C9BwYHPzoc7CFsKHXPwVbBBW3BOSG1lOBEUNFJMKKwZYzgMHB+4ZpZ/NsRFFBkHT7GDF72KKo/HabLChWWhZQEgcgZq+CdCs3trQBCFiXlpnT+G0+UMjrijGAhHS7BQhePtgWBiCgUGm5QAOoCOlWbmwrAxCAhnAyn5/AydMASLo9XknqnZzYglFAbmxKy/AkFI+KlG7lxrb0AGIQUcR1+6RUMUoGD6Babl4NbHsDECBVg5BqAOsEz3TiZhh4ATeNAQggaZkF9jQRRZEBPK9jmBPbhdkXgpvOAARQxOBwOG7R6HQ/285rfC7ctAbw/1jH/wEHnCVcWuW6QwAAAABJRU5ErkJggg==" + } + }, + "cell_type": "markdown", + "id": "4ed0a233", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "![NVIDIA Structured Data Models logo](attachment:logo.png)\n", + "\n", + "# Kumo Tabular on Breast Mass Classification (Kaggle)\n", + "\n", + "**Kumo Tabular** is a structured-data foundation model for predictive AI on a single table. It uses labelled context rows to predict outcomes for new rows without training a task-specific model.\n", + "\n", + "This notebook runs a **binary classification** task with the Mammographic Mass dataset from **Kaggle**, using `structured-data-models-client` to call a deployed **Kumo Tabular NVIDIA NIM**. It demonstrates both a direct `Request` and a Predictive Query Language (PQL) workflow.\n", + "\n", + "**A GPU is not required on the notebook machine.** The model runs on the NIM server. This notebook reads the dataset from an existing Snowflake stage using an active Snowpark session.\n", + "\n", + "The client identifies the model as `KumoTabular()`; its size is determined by the server's deployed checkpoint. Local PyTorch `size`, `device`, and ensemble settings do not configure the NIM.\n", + "\n", + "See the [Kumo Tabular overview](https://docs.nvidia.com/sdgm/kumo-tabular/about-kumo-tabular/overview), [quickstart](https://docs.nvidia.com/sdgm/kumo-tabular/get-started/quickstart), and [client repository](https://github.com/NVIDIA/structured-data-models-client).\n", + "\n", + "Each user is responsible for checking the content of datasets and the applicable licenses and determining if suitable for the intended use." + ] + }, + { + "cell_type": "markdown", + "id": "561d19ec", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## 1. Install and connect\n", + "\n", + "Use a Python 3.10 or newer kernel with network access to the NIM endpoint and an active Snowflake session for the dataset. The notebook also uses pandas, NumPy, and scikit-learn.\n", + "\n", + "Install the base client; the relational extra is not needed. See [Install and Connect](https://docs.nvidia.com/sdgm/kumo-tabular/get-started/sdk-getting-started)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "39f1ec88-eee7-40e9-9c9b-4b2d36e0031c", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "%pip install structured-data-models-client" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "80909c75-dfb5-4051-b037-4a6b92e92049", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "import os\n", + "\n", + "from sdm_client import KumoTabular, PredictiveQuery, Request, Schema, SDMClient\n", + "\n", + "# Override the endpoint through environment configuration when needed.\n", + "nim_endpoint = os.environ.get(\n", + " 'KUMO_TABULAR_API_ENDPOINT',\n", + " 'http://:8000',\n", + ")\n", + "\n", + "nim_api_key = os.environ.get('KUMO_TABULAR_API_KEY')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1ef18d21-c18a-44e6-b8f3-078ac5a2cc30", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "# Check the models served by the configured NIM endpoint.\n", + "with SDMClient(url=nim_endpoint, api_key=nim_api_key) as client:\n", + " print(client.models())" + ] + }, + { + "cell_type": "markdown", + "id": "e94f92c0", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## 2. Load the dataset\n", + "\n", + "**Dataset:** [Mammographic Mass Data Set (Kaggle)](https://www.kaggle.com/datasets/overratedgman/mammographic-mass-data-set), version 3\n", + "\n", + "**Task:** Binary classification of breast masses: benign (0) or malignant (1)\n", + "\n", + "Here we will primarily use four predictors: `age`, `mass` `shape`, `margin`, and `density`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bfdc049a", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "import pandas as pd\n", + "from sklearn.model_selection import train_test_split\n", + "from snowflake.snowpark.context import get_active_session\n", + "\n", + "# Read the Kaggle CSV uploaded to the stage\n", + "session = get_active_session()\n", + "with session.file.get_stream(\n", + " '@../Cleaned_data.csv'\n", + ") as f:\n", + " data = pd.read_csv(f)\n", + "\n", + "# Select predictors and readable class labels\n", + "X = data[['Age', 'Shape', 'Margin', 'Density']]\n", + "y = data['Severity'].map({0: 'benign', 1: 'malignant'})\n", + "classes = sorted(y.unique())\n", + "df = X.assign(target=pd.Categorical(y, categories=classes))\n", + "\n", + "# Split ICL examples (\"train\") and test sets\n", + "train, test = train_test_split(\n", + " data.index,\n", + " test_size=0.2,\n", + " stratify=y,\n", + " random_state=0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "9b2d96a3", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## 3. Prepare rows and run Kumo Tabular through the NIM\n", + "\n", + "Use the training split as labelled `context` and the held-out test split as `predict`. The target column must be present only in the context. A row ID lets PQL identify entities and keeps the held-out rows distinct from the context.\n", + "\n", + "The explicit schema preserves the original numerical and categorical feature types. The NIM accepts `task=\"classification\"` for both binary and multiclass tasks. Direct requests support up to 10,000 context rows; this dataset fits within that limit.\n", + "\n", + "See [Make Predictions](https://docs.nvidia.com/sdgm/kumo-tabular/user-guide/make-predictions/make-predictions)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "91fccd58", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "context = df.iloc[train].copy()\n", + "to_score = df.iloc[test].drop(columns='target').copy()\n", + "context.insert(0, 'row_id', train)\n", + "to_score.insert(0, 'row_id', test)\n", + "\n", + "schema = Schema.from_data(\n", + " {'masses': context},\n", + " primary_keys={'masses': 'row_id'},\n", + " relationships=[],\n", + ")\n", + "for column, stype in {\n", + " 'Age': 'numerical',\n", + " 'Density': 'numerical',\n", + " 'Shape': 'categorical',\n", + " 'Margin': 'categorical',\n", + " 'target': 'categorical',\n", + "}.items():\n", + " schema['masses'][column].stype = stype\n", + "schema.check_consistent()\n", + "\n", + "request = Request(\n", + " context=context,\n", + " predict=to_score,\n", + " target='target',\n", + " task='classification',\n", + " schema=schema,\n", + " entity_table='masses',\n", + ")\n", + "\n", + "with SDMClient(url=nim_endpoint, api_key=nim_api_key) as client:\n", + " result = client.predict(\n", + " request,\n", + " model=KumoTabular(),\n", + " outputs=['prediction', 'probabilities'],\n", + " )" + ] + }, + { + "cell_type": "markdown", + "id": "9e64dc5a", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## 4. Read the predictions\n", + "\n", + "`PredictionResult.to_pandas()` returns `row_index`, `prediction`, and `probability_` columns. `row_index` refers to the zero-based position within `to_score`, not its pandas index. Align on it before evaluating against the test labels.\n", + "\n", + "See [Understand Prediction Results](https://docs.nvidia.com/sdgm/kumo-tabular/user-guide/work-with-prediction-results/prediction-results)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c46ac604", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "answers = (\n", + " result.to_pandas().set_index('row_index').reindex(range(len(to_score)))\n", + ")\n", + "predictions = answers['prediction'].to_numpy()\n", + "probabilities = answers[\n", + " [f'probability_{label}' for label in classes]\n", + "].to_numpy()\n", + "answers.head()" + ] + }, + { + "cell_type": "markdown", + "id": "6dc8e858", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## 5. Evaluate predictions" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "039b95c2", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "from sklearn.dummy import DummyClassifier\n", + "from sklearn.metrics import accuracy_score, roc_auc_score\n", + "\n", + "baseline = DummyClassifier(strategy='most_frequent').fit(\n", + " X.iloc[train], y.iloc[train]\n", + ")\n", + "pd.DataFrame(\n", + " {\n", + " 'Model': ['Kumo Tabular (NIM)', 'Majority class'],\n", + " 'Accuracy': [\n", + " accuracy_score(y.iloc[test], predictions),\n", + " accuracy_score(y.iloc[test], baseline.predict(X.iloc[test])),\n", + " ],\n", + " 'ROC AUC': [\n", + " roc_auc_score(\n", + " y.iloc[test], probabilities[:, classes.index('malignant')]\n", + " ),\n", + " roc_auc_score(\n", + " y.iloc[test],\n", + " baseline.predict_proba(X.iloc[test])[\n", + " :, list(baseline.classes_).index('malignant')\n", + " ],\n", + " ),\n", + " ],\n", + " }\n", + ").set_index('Model').round(4)" + ] + }, + { + "cell_type": "markdown", + "id": "f1d2284c", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## 6. Run the same task with a predictive query (PQL)\n", + "\n", + "PQL names the prediction target and entity. Here `PREDICT masses.target FOR EACH masses.row_id` asks for each mass's benign/malignant label. `materialize()` builds the request, and `client.predict()` sends it to the NIM.\n", + "\n", + "Reuse the schema built from **training rows only** and supply the held-out feature rows with `predict=to_score`. This keeps test labels out of the prediction context. Pass `task=\"classification\"` explicitly. Both workflows use the same context, test split, and semantic types; predictions can vary between calls.\n", + "\n", + "For rows already in a schema's table, `materialize(indices=[...], task=\"classification\")` selects rows by primary-key value and excludes them from context. The `predict=` form below scores rows supplied separately.\n", + "\n", + "See [Make Predictions with PQL](https://docs.nvidia.com/sdgm/kumo-tabular/user-guide/make-predictions/make-predictions#ask-in-pql)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c7d38349", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "query = PredictiveQuery(\n", + " 'PREDICT masses.target FOR EACH masses.row_id',\n", + " schema,\n", + ")\n", + "pql_request = query.materialize(\n", + " predict=to_score,\n", + " task='classification',\n", + " model=KumoTabular(),\n", + ")\n", + "\n", + "with SDMClient(url=nim_endpoint, api_key=nim_api_key) as client:\n", + " pql_result = client.predict(\n", + " pql_request,\n", + " model=KumoTabular(),\n", + " outputs=['prediction', 'probabilities'],\n", + " )\n", + "\n", + "pql_answers = (\n", + " pql_result.to_pandas().set_index('row_index').reindex(range(len(to_score)))\n", + ")\n", + "pql_answers.head()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "21562091", + "metadata": { + "language": "python" + }, + "outputs": [], + "source": [ + "pd.DataFrame(\n", + " {\n", + " 'Model': ['Kumo Tabular (NIM, PQL)'],\n", + " 'Accuracy': [accuracy_score(y.iloc[test], pql_answers['prediction'])],\n", + " 'ROC AUC': [\n", + " roc_auc_score(y.iloc[test], pql_answers['probability_malignant'])\n", + " ],\n", + " }\n", + ").set_index('Model').round(4)" + ] + }, + { + "cell_type": "markdown", + "id": "425c9d32", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "## Next Steps" + ] + }, + { + "cell_type": "markdown", + "id": "053b0106", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "### Check out other available Structured Data Foundation Models:\n", + "- [Kumo-Relational](https://huggingface.co/nvidia/Kumo-Relational)\n", + "- [Kumo-Anomaly](https://huggingface.co/nvidia/Kumo-Anomaly)\n", + "- [Kumo-Forecast](https://huggingface.co/nvidia/Kumo-Forecast)" + ] + }, + { + "cell_type": "markdown", + "id": "27aae15a", + "metadata": { + "codeCollapsed": true + }, + "source": [ + "### Check out Kumo Engine\n", + "To run foundation models for structured data in production at scale, check out Kumo Engine\n", + "- [Documentation](https://docs.nvidia.com/sdgm/kumo-engine/get-started-with-kumo-engine/overview)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.15" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From 4c66160403d2560a300b45b6a0ef14591075dcc5 Mon Sep 17 00:00:00 2001 From: k-ruthvika Date: Fri, 13 Nov 2026 10:20:19 -0500 Subject: [PATCH 2/5] examples: clarify Snowflake setup and project requested columns Signed-off-by: k-ruthvika --- .../kumo_relational_quickstart.ipynb | 31 ++++++++++++++----- .../snowflake/kumo_tabular_quickstart.ipynb | 4 +-- 2 files changed, 25 insertions(+), 10 deletions(-) diff --git a/examples/snowflake/kumo_relational_quickstart.ipynb b/examples/snowflake/kumo_relational_quickstart.ipynb index eb0d772c8..64d7f5226 100644 --- a/examples/snowflake/kumo_relational_quickstart.ipynb +++ b/examples/snowflake/kumo_relational_quickstart.ipynb @@ -219,10 +219,14 @@ "The repository's samples and templates are provided under the\n", "[MIT License](https://github.com/microsoft/sql-server-samples/blob/master/license.txt).\n", "\n", - "Set `DATA_LOCATION` to the Snowflake database and schema that hold the\n", - "AdventureWorks tables (`RUTH.ADVENTUREWORKS` in this account). Each table is\n", - "read with the Snowpark session, and only the listed columns that exist in the\n", - "table are kept.\n", + "After downloading the CSVs, upload them into Snowflake as tables named\n", + "`Customer`, `Product`, `SalesOrderHeader`, and `SalesOrderDetail`. Preserve\n", + "these table names and the CSV schemas' column names with their exact\n", + "capitalization, using quoted identifiers when creating the tables.\n", + "\n", + "Set `DATA_LOCATION` to `.`, replacing the placeholders with\n", + "the database and schema containing those tables. Each table is read through\n", + "the Snowpark session, and only the listed columns are loaded.\n", "\n", "The sales-order header and detail tables remain separate. During preparation,\n", "only `CustomerID` and `OrderDate` are copied from each header to its detail\n", @@ -329,7 +333,11 @@ "\n", "def read_headerless_table(filename, columns):\n", " path = f'{DATA_LOCATION}.\"{Path(filename).stem}\"'\n", - " return session.table(path).to_pandas()\n", + " return (\n", + " session.table(path)\n", + " .select([f'\"{column}\"' for column in columns])\n", + " .to_pandas()\n", + " )\n", "\n", "\n", "customers_df = read_headerless_table('Customer.csv', CUSTOMER_COLUMNS)\n", @@ -488,8 +496,7 @@ "`rowguid` columns, and `CarrierTrackingNumber` identify records rather than\n", "categories, and several exceed the NIM's 10,000-value categorical cardinality\n", "limit in the full extracts, so this example marks them `\"ID\"`. Product names\n", - "are typed `\"text\"` to reflect their content, though Kumo Relational currently\n", - "ignores `text` features for prediction; see\n", + "are typed `\"categorical\"`, matching the schema configuration below; see\n", "[Data Types](https://docs.nvidia.com/sdgm/kumo-relational/user-guide/prepare-data/data-types).\n", "Order quantity is a measured numerical value." ] @@ -693,7 +700,15 @@ "source": [ "### Example 1B: Forecast from a historical anchor time\n", "\n", - "By default, predictions use the maximum timestamp in the temporal schema. Set a historical `anchor_time` to simulate what the prediction would have looked like at that point in time. Kumo Relational uses only information available before the anchor time." + "By default, predictions use the maximum timestamp in the temporal schema.\n", + "Set a historical `anchor_time` to make a prediction relative to an earlier date.\n", + "Temporal sampling uses the configured `OrderDate` columns for sales-order\n", + "headers and details to restrict their records to the anchor time.\n", + "\n", + "The `customers` and `products` tables have no time column in this schema, so\n", + "their current attributes remain available at every anchor. Setting an anchor\n", + "does not recover earlier attribute values. Use historical snapshots of those\n", + "tables for a point-in-time evaluation." ] }, { diff --git a/examples/snowflake/kumo_tabular_quickstart.ipynb b/examples/snowflake/kumo_tabular_quickstart.ipynb index 56bc4ba10..42ed234a4 100644 --- a/examples/snowflake/kumo_tabular_quickstart.ipynb +++ b/examples/snowflake/kumo_tabular_quickstart.ipynb @@ -52,7 +52,7 @@ }, "outputs": [], "source": [ - "%pip install structured-data-models-client" + "%pip install structured-data-models-client scikit-learn" ] }, { @@ -104,7 +104,7 @@ "\n", "**Task:** Binary classification of breast masses: benign (0) or malignant (1)\n", "\n", - "Here we will primarily use four predictors: `age`, `mass` `shape`, `margin`, and `density`" + "Use four predictors: `Age`, `Shape`, `Margin`, and `Density`." ] }, { From d7e07936223b409ccc09939ddbc86e9bb35fcb90 Mon Sep 17 00:00:00 2001 From: k-ruthvika Date: Tue, 17 Nov 2026 10:32:23 -0500 Subject: [PATCH 3/5] examples: sample Snowflake tables in place and clarify notebook setup Signed-off-by: k-ruthvika --- CHANGELOG.md | 5 + docs/get-started/quickstart.md | 15 + .../kumo_relational_quickstart.ipynb | 438 +++++++++--------- .../snowflake/kumo_tabular_quickstart.ipynb | 23 +- .../tests/test_snowflake_notebooks.py | 111 +++++ 5 files changed, 366 insertions(+), 226 deletions(-) create mode 100644 packages/structured-data-models-client/tests/test_snowflake_notebooks.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 14af03209..fd1054d5b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,11 @@ these packages. All five distributions share one version and one entry. ### Added +- Snowflake Container Runtime quickstarts for Kumo Relational and Kumo Tabular + in `examples/snowflake/`. The relational example samples uploaded + AdventureWorks tables through `SnowflakeSource`; the tabular example reads + `Cleaned_data.csv` from a stage. Both call a deployed NIM. + - `SDMClient.in_process()` runs `kumo-tabular` and `kumo-relational` in this Python process with structured-data-models, through the same calls as a NIM: predictions, fitted sessions, rankings, forecasts and explanations. It loads each model diff --git a/docs/get-started/quickstart.md b/docs/get-started/quickstart.md index 102af90d4..bbd52c5e2 100644 --- a/docs/get-started/quickstart.md +++ b/docs/get-started/quickstart.md @@ -164,6 +164,21 @@ The relational notebook installs the tabular notebook installs `structured-data-models-client[databricks,databricks-serving]`. +## Snowflake notebooks + +The Snowflake quickstarts run on Container Runtime with an active Snowpark +session and a deployed NIM endpoint: + +- [Kumo Relational](../../examples/snowflake/kumo_relational_quickstart.ipynb) + samples uploaded AdventureWorks tables in place through `SnowflakeSource`. +- [Kumo Tabular](../../examples/snowflake/kumo_tabular_quickstart.ipynb) + reads `Cleaned_data.csv` from a Snowflake stage and predicts through direct + requests and PQL. + +Download and upload the sample data yourself, following the notebook's source +links and schema instructions. Configure external access for package +installation and any NIM endpoint outside Snowflake. + ## A model in this process The model can also run in your own Python process, with no NIM at all. The diff --git a/examples/snowflake/kumo_relational_quickstart.ipynb b/examples/snowflake/kumo_relational_quickstart.ipynb index 64d7f5226..a5c59aef0 100644 --- a/examples/snowflake/kumo_relational_quickstart.ipynb +++ b/examples/snowflake/kumo_relational_quickstart.ipynb @@ -69,7 +69,17 @@ "source": [ "## Step 1. Install structured-data-models-client\n", "\n", - "Install the SDK with the relational extra. See [Deploy, Install, and Connect](https://docs.nvidia.com/sdgm/kumo-relational/get-started/sdk-getting-started) for the supported Python versions and platforms." + "Use a Snowflake notebook on **Container Runtime** with an active Snowpark\n", + "session. Configure an external access integration for PyPI package installation\n", + "and attach it to the notebook service. If the NIM is outside Snowflake, allow\n", + "its host through an external access integration too. See Snowflake's\n", + "[external access setup](https://docs.snowflake.com/en/user-guide/ui-snowsight/notebooks-external-access).\n", + "The notebook itself does not need a GPU; inference runs on the NIM service.\n", + "\n", + "Install the relational and Snowflake extras. See\n", + "[Install and Connect](https://docs.nvidia.com/sdgm/client/kumo-relational/get-started/sdk-getting-started)\n", + "for supported Python versions and platforms. Restart the notebook kernel after\n", + "installation if these packages were already imported, then continue below." ] }, { @@ -81,7 +91,7 @@ }, "outputs": [], "source": [ - "%pip install \"structured-data-models-client[relational]\"" + "%pip install \"structured-data-models-client[relational,snowflake]\"" ] }, { @@ -94,7 +104,6 @@ "outputs": [], "source": [ "import os\n", - "from pathlib import Path\n", "\n", "import pandas as pd\n", "from sdm_client import (\n", @@ -103,7 +112,8 @@ " Relationship,\n", " Schema,\n", " SDMClient,\n", - ")" + ")\n", + "from sdm_connectors import SnowflakeSource" ] }, { @@ -125,13 +135,16 @@ "source": [ "## Step 2. Configure the Kumo Relational NIM endpoint\n", "\n", - "Deploy Kumo Relational as an NVIDIA NIM before running this notebook. See\n", - "[Deploy, Install, and Connect](https://docs.nvidia.com/sdgm/kumo-relational/get-started/sdk-getting-started)\n", - "for deployment instructions.\n", + "Deploy the NIM first using the\n", + "[Snowflake deployment guide](https://docs.nvidia.com/sdgm/client/kumo-relational/deployment/snowflake).\n", + "Set `KUMO_RELATIONAL_API_ENDPOINT` to its reachable address. The\n", + "`http://:8000` placeholder below is a private SPCS service address,\n", + "not a container running in the notebook.\n", "\n", - "Set `KUMO_RELATIONAL_API_ENDPOINT` to the reachable NIM endpoint. The default below uses a\n", - "local container on port `8000`. Set `KUMO_RELATIONAL_API_KEY` only when an\n", - "authenticating gateway protects the endpoint." + "For endpoints outside that private service network, use an `https://` URL with\n", + "TLS termination. Plain HTTP transmits table rows unencrypted. Set\n", + "`KUMO_RELATIONAL_API_KEY` only when an authenticating gateway protects the\n", + "endpoint; keep credentials in environment configuration." ] }, { @@ -201,36 +214,127 @@ "codeCollapsed": true }, "source": [ - "## Step 4. Data Preparation\n", - "\n", - "This notebook uses the **AdventureWorks OLTP sample database**, originally\n", - "published by Microsoft as a SQL Server sample. The four source extracts are:\n", - "\n", - "- `Customer.csv`\n", - "- `Product.csv`\n", - "- `SalesOrderHeader.csv`\n", - "- `SalesOrderDetail.csv`\n", - "\n", - "Download the four `*.csv` extracts from the AdventureWorks OLTP install scripts in the\n", - "[Microsoft SQL Server samples repository](https://github.com/microsoft/sql-server-samples/tree/master/samples/databases/adventure-works/oltp-install-script).\n", - "\n", - "> Each user is responsible for checking the content of datasets and the\n", - "> applicable licenses and determining if suitable for the intended use.\n", - "The repository's samples and templates are provided under the\n", - "[MIT License](https://github.com/microsoft/sql-server-samples/blob/master/license.txt).\n", - "\n", - "After downloading the CSVs, upload them into Snowflake as tables named\n", - "`Customer`, `Product`, `SalesOrderHeader`, and `SalesOrderDetail`. Preserve\n", - "these table names and the CSV schemas' column names with their exact\n", - "capitalization, using quoted identifiers when creating the tables.\n", - "\n", - "Set `DATA_LOCATION` to `.`, replacing the placeholders with\n", - "the database and schema containing those tables. Each table is read through\n", - "the Snowpark session, and only the listed columns are loaded.\n", - "\n", - "The sales-order header and detail tables remain separate. During preparation,\n", - "only `CustomerID` and `OrderDate` are copied from each header to its detail\n", - "rows using `SalesOrderID`." + "## Step 4. Prepare the Snowflake tables\n", + "\n", + "Download `Customer.csv`, `Product.csv`, `SalesOrderHeader.csv`, and\n", + "`SalesOrderDetail.csv` from the pinned\n", + "[Microsoft AdventureWorks OLTP sample files](https://github.com/microsoft/sql-server-samples/tree/b47eadc85277f72843647223cde53aa1b85ffcb8/samples/databases/adventure-works/oltp-install-script).\n", + "These files are **tab-separated and have no header row**, despite their `.csv`\n", + "extension. The table definitions below provide the column names in file order.\n", + "\n", + "Upload the four files yourself to an internal Snowflake stage. In a SQL\n", + "worksheet, replace ``, ``, and `` with your destination\n", + "names, then create and load these tables in a dedicated schema. The quoted\n", + "identifiers preserve the capitalization used by the notebook. Your role needs\n", + "access to the database, schema, warehouse, and stage, permission to create and\n", + "load these tables, and permission to create the context view used below.\n", + "\n", + "Each user is responsible for checking the content of datasets and the applicable licenses and determining if suitable for the intended use.\n", + "Microsoft's samples are provided under the\n", + "[MIT License](https://github.com/microsoft/sql-server-samples/blob/b47eadc85277f72843647223cde53aa1b85ffcb8/license.txt).\n", + "\n", + "```sql\n", + "CREATE TABLE IF NOT EXISTS ..\"Customer\" (\n", + " \"CustomerID\" INTEGER,\n", + " \"PersonID\" INTEGER,\n", + " \"StoreID\" INTEGER,\n", + " \"TerritoryID\" INTEGER,\n", + " \"AccountNumber\" VARCHAR,\n", + " \"rowguid\" VARCHAR,\n", + " \"ModifiedDate\" TIMESTAMP_NTZ\n", + ");\n", + "\n", + "CREATE TABLE IF NOT EXISTS ..\"Product\" (\n", + " \"ProductID\" INTEGER,\n", + " \"Name\" VARCHAR,\n", + " \"ProductNumber\" VARCHAR,\n", + " \"MakeFlag\" INTEGER,\n", + " \"FinishedGoodsFlag\" INTEGER,\n", + " \"Color\" VARCHAR,\n", + " \"SafetyStockLevel\" INTEGER,\n", + " \"ReorderPoint\" INTEGER,\n", + " \"StandardCost\" FLOAT,\n", + " \"ListPrice\" FLOAT,\n", + " \"Size\" VARCHAR,\n", + " \"SizeUnitMeasureCode\" VARCHAR,\n", + " \"WeightUnitMeasureCode\" VARCHAR,\n", + " \"Weight\" FLOAT,\n", + " \"DaysToManufacture\" INTEGER,\n", + " \"ProductLine\" VARCHAR,\n", + " \"Class\" VARCHAR,\n", + " \"Style\" VARCHAR,\n", + " \"ProductSubcategoryID\" INTEGER,\n", + " \"ProductModelID\" INTEGER,\n", + " \"SellStartDate\" TIMESTAMP_NTZ,\n", + " \"SellEndDate\" TIMESTAMP_NTZ,\n", + " \"DiscontinuedDate\" TIMESTAMP_NTZ,\n", + " \"rowguid\" VARCHAR,\n", + " \"ModifiedDate\" TIMESTAMP_NTZ\n", + ");\n", + "\n", + "CREATE TABLE IF NOT EXISTS ..\"SalesOrderHeader\" (\n", + " \"SalesOrderID\" INTEGER,\n", + " \"RevisionNumber\" INTEGER,\n", + " \"OrderDate\" TIMESTAMP_NTZ,\n", + " \"DueDate\" TIMESTAMP_NTZ,\n", + " \"ShipDate\" TIMESTAMP_NTZ,\n", + " \"Status\" INTEGER,\n", + " \"OnlineOrderFlag\" INTEGER,\n", + " \"SalesOrderNumber\" VARCHAR,\n", + " \"PurchaseOrderNumber\" VARCHAR,\n", + " \"AccountNumber\" VARCHAR,\n", + " \"CustomerID\" INTEGER,\n", + " \"SalesPersonID\" INTEGER,\n", + " \"TerritoryID\" INTEGER,\n", + " \"BillToAddressID\" INTEGER,\n", + " \"ShipToAddressID\" INTEGER,\n", + " \"ShipMethodID\" INTEGER,\n", + " \"CreditCardID\" INTEGER,\n", + " \"CreditCardApprovalCode\" VARCHAR,\n", + " \"CurrencyRateID\" INTEGER,\n", + " \"SubTotal\" FLOAT,\n", + " \"TaxAmt\" FLOAT,\n", + " \"Freight\" FLOAT,\n", + " \"TotalDue\" FLOAT,\n", + " \"Comment\" VARCHAR,\n", + " \"rowguid\" VARCHAR,\n", + " \"ModifiedDate\" TIMESTAMP_NTZ\n", + ");\n", + "\n", + "CREATE TABLE IF NOT EXISTS ..\"SalesOrderDetail\" (\n", + " \"SalesOrderID\" INTEGER,\n", + " \"SalesOrderDetailID\" INTEGER,\n", + " \"CarrierTrackingNumber\" VARCHAR,\n", + " \"OrderQty\" INTEGER,\n", + " \"ProductID\" INTEGER,\n", + " \"SpecialOfferID\" INTEGER,\n", + " \"UnitPrice\" FLOAT,\n", + " \"UnitPriceDiscount\" FLOAT,\n", + " \"LineTotal\" FLOAT,\n", + " \"rowguid\" VARCHAR,\n", + " \"ModifiedDate\" TIMESTAMP_NTZ\n", + ");\n", + "\n", + "COPY INTO ..\"Customer\"\n", + " FROM @../Customer.csv\n", + " FILE_FORMAT = (TYPE = CSV FIELD_DELIMITER = '\\t' SKIP_HEADER = 0\n", + " FIELD_OPTIONALLY_ENCLOSED_BY = NONE NULL_IF = (''));\n", + "\n", + "COPY INTO ..\"Product\"\n", + " FROM @../Product.csv\n", + " FILE_FORMAT = (TYPE = CSV FIELD_DELIMITER = '\\t' SKIP_HEADER = 0\n", + " FIELD_OPTIONALLY_ENCLOSED_BY = NONE NULL_IF = (''));\n", + "\n", + "COPY INTO ..\"SalesOrderHeader\"\n", + " FROM @../SalesOrderHeader.csv\n", + " FILE_FORMAT = (TYPE = CSV FIELD_DELIMITER = '\\t' SKIP_HEADER = 0\n", + " FIELD_OPTIONALLY_ENCLOSED_BY = NONE NULL_IF = (''));\n", + "\n", + "COPY INTO ..\"SalesOrderDetail\"\n", + " FROM @../SalesOrderDetail.csv\n", + " FILE_FORMAT = (TYPE = CSV FIELD_DELIMITER = '\\t' SKIP_HEADER = 0\n", + " FIELD_OPTIONALLY_ENCLOSED_BY = NONE NULL_IF = (''));\n", + "```" ] }, { @@ -242,112 +346,13 @@ }, "outputs": [], "source": [ + "from snowflake.snowpark import functions as F\n", "from snowflake.snowpark.context import get_active_session\n", "\n", "session = get_active_session()\n", - "\n", - "DATA_LOCATION = '.'\n", - "print('AdventureWorks data:', DATA_LOCATION)\n", - "\n", - "CUSTOMER_COLUMNS = [\n", - " 'CustomerID',\n", - " 'PersonID',\n", - " 'StoreID',\n", - " 'TerritoryID',\n", - " 'AccountNumber',\n", - " 'rowguid',\n", - " 'ModifiedDate',\n", - "]\n", - "\n", - "PRODUCT_COLUMNS = [\n", - " 'ProductID',\n", - " 'Name',\n", - " 'ProductNumber',\n", - " 'MakeFlag',\n", - " 'FinishedGoodsFlag',\n", - " 'Color',\n", - " 'SafetyStockLevel',\n", - " 'ReorderPoint',\n", - " 'StandardCost',\n", - " 'ListPrice',\n", - " 'Size',\n", - " 'SizeUnitMeasureCode',\n", - " 'WeightUnitMeasureCode',\n", - " 'Weight',\n", - " 'DaysToManufacture',\n", - " 'ProductLine',\n", - " 'Class',\n", - " 'Style',\n", - " 'ProductSubcategoryID',\n", - " 'ProductModelID',\n", - " 'SellStartDate',\n", - " 'SellEndDate',\n", - " 'DiscontinuedDate',\n", - " 'rowguid',\n", - " 'ModifiedDate',\n", - "]\n", - "\n", - "SALES_HEADER_COLUMNS = [\n", - " 'SalesOrderID',\n", - " 'RevisionNumber',\n", - " 'OrderDate',\n", - " 'DueDate',\n", - " 'ShipDate',\n", - " 'Status',\n", - " 'OnlineOrderFlag',\n", - " 'SalesOrderNumber',\n", - " 'PurchaseOrderNumber',\n", - " 'AccountNumber',\n", - " 'CustomerID',\n", - " 'SalesPersonID',\n", - " 'TerritoryID',\n", - " 'BillToAddressID',\n", - " 'ShipToAddressID',\n", - " 'ShipMethodID',\n", - " 'CreditCardID',\n", - " 'CreditCardApprovalCode',\n", - " 'CurrencyRateID',\n", - " 'SubTotal',\n", - " 'TaxAmt',\n", - " 'Freight',\n", - " 'TotalDue',\n", - " 'Comment',\n", - " 'rowguid',\n", - " 'ModifiedDate',\n", - "]\n", - "\n", - "SALES_DETAIL_COLUMNS = [\n", - " 'SalesOrderID',\n", - " 'SalesOrderDetailID',\n", - " 'CarrierTrackingNumber',\n", - " 'OrderQty',\n", - " 'ProductID',\n", - " 'SpecialOfferID',\n", - " 'UnitPrice',\n", - " 'UnitPriceDiscount',\n", - " 'LineTotal',\n", - " 'rowguid',\n", - " 'ModifiedDate',\n", - "]\n", - "\n", - "\n", - "def read_headerless_table(filename, columns):\n", - " path = f'{DATA_LOCATION}.\"{Path(filename).stem}\"'\n", - " return (\n", - " session.table(path)\n", - " .select([f'\"{column}\"' for column in columns])\n", - " .to_pandas()\n", - " )\n", - "\n", - "\n", - "customers_df = read_headerless_table('Customer.csv', CUSTOMER_COLUMNS)\n", - "products_df = read_headerless_table('Product.csv', PRODUCT_COLUMNS)\n", - "sales_order_headers_df = read_headerless_table(\n", - " 'SalesOrderHeader.csv', SALES_HEADER_COLUMNS\n", - ")\n", - "sales_order_details_df = read_headerless_table(\n", - " 'SalesOrderDetail.csv', SALES_DETAIL_COLUMNS\n", - ")" + "DATABASE = ''\n", + "SCHEMA = ''\n", + "DATA_LOCATION = f'\"{DATABASE}\".\"{SCHEMA}\"'" ] }, { @@ -357,7 +362,10 @@ "codeCollapsed": true }, "source": [ - "Parse the timestamp fields, then enrich each sales-order detail with only its parent order's `CustomerID` and `OrderDate`. The header and detail dataframes remain separate." + "Create a temporary view that joins each order line to its parent order's\n", + "`CustomerID` and `OrderDate`. The uploaded source tables stay unchanged.\n", + "`SnowflakeSource` uses this same session, so the view remains visible when the\n", + "client samples context. Rerun this cell after restarting the session." ] }, { @@ -369,32 +377,13 @@ }, "outputs": [], "source": [ - "timestamp_columns = {\n", - " 'customers': (customers_df, ['ModifiedDate']),\n", - " 'products': (\n", - " products_df,\n", - " ['SellStartDate', 'SellEndDate', 'DiscontinuedDate', 'ModifiedDate'],\n", - " ),\n", - " 'sales_order_headers': (\n", - " sales_order_headers_df,\n", - " ['OrderDate', 'DueDate', 'ShipDate', 'ModifiedDate'],\n", - " ),\n", - " 'sales_order_details': (sales_order_details_df, ['ModifiedDate']),\n", - "}\n", - "\n", - "for _, (table_df, columns) in timestamp_columns.items():\n", - " for column in columns:\n", - " if (\n", - " column in table_df\n", - " ): # skip columns not present in the Snowflake table\n", - " table_df[column] = pd.to_datetime(table_df[column], errors='coerce')\n", - "\n", - "sales_order_details_df = sales_order_details_df.merge(\n", - " sales_order_headers_df[['SalesOrderID', 'CustomerID', 'OrderDate']],\n", - " on='SalesOrderID',\n", - " how='left',\n", - " validate='many_to_one',\n", - ")" + "session.sql(f\"\"\"CREATE OR REPLACE TEMPORARY VIEW\n", + " {DATA_LOCATION}.\"SDM_SalesOrderDetail\" AS\n", + "SELECT d.*, h.\"CustomerID\", h.\"OrderDate\"\n", + "FROM {DATA_LOCATION}.\"SalesOrderDetail\" AS d\n", + "JOIN {DATA_LOCATION}.\"SalesOrderHeader\" AS h\n", + " ON d.\"SalesOrderID\" = h.\"SalesOrderID\"\n", + "\"\"\").collect()" ] }, { @@ -404,13 +393,13 @@ "codeCollapsed": true }, "source": [ - "## Step 5. Build a schema from DataFrames\n", + "## Step 5. Build a schema from Snowflake\n", "\n", - "`Schema.from_data()` accepts a dictionary of pandas DataFrames and infers\n", - "each table's column types and semantic roles automatically. The AdventureWorks\n", - "relationships are supplied explicitly so the example is reproducible.\n", - "Kumo Relational still infers column data types and semantic metadata, which\n", - "you should review before prediction." + "`Schema.from_source()` describes the tables where they are stored. It reads\n", + "metadata and a small preview to infer types, and checks the configured keys in\n", + "Snowflake. Prediction requests sample related rows with SQL rather than copying\n", + "all four source tables into pandas. Review the inferred semantic types before\n", + "prediction." ] }, { @@ -422,12 +411,14 @@ }, "outputs": [], "source": [ - "schema = Schema.from_data(\n", + "source = SnowflakeSource(session, database=DATABASE, schema=SCHEMA)\n", + "schema = Schema.from_source(\n", + " source,\n", " {\n", - " 'customers': customers_df,\n", - " 'products': products_df,\n", - " 'sales_order_headers': sales_order_headers_df,\n", - " 'sales_order_details': sales_order_details_df,\n", + " 'customers': 'Customer',\n", + " 'products': 'Product',\n", + " 'sales_order_headers': 'SalesOrderHeader',\n", + " 'sales_order_details': 'SDM_SalesOrderDetail',\n", " },\n", " primary_keys={\n", " 'customers': 'CustomerID',\n", @@ -436,8 +427,7 @@ " 'sales_order_details': 'SalesOrderDetailID',\n", " },\n", " time_columns={\n", - " # These two tables contain datetime columns but do not represent\n", - " # event timelines, so no time column is assigned to them.\n", + " # Customer and product attributes have no event timeline here.\n", " 'customers': None,\n", " 'products': None,\n", " 'sales_order_headers': 'OrderDate',\n", @@ -483,22 +473,15 @@ "codeCollapsed": true }, "source": [ - "Primary-key and relationship columns are automatically assigned the `ID`\n", - "semantic type. Review the remaining columns before prediction:\n", - "\n", - "- Override a column's semantic type with\n", - " `schema[\"table_name\"][\"column_name\"].stype = \"\"`.\n", - "- Available semantic types: `\"numerical\"`, `\"categorical\"`,\n", - " `\"multicategorical\"`, `\"ID\"`, `\"text\"`, `\"timestamp\"`.\n", - "\n", - "For this known AdventureWorks schema, review identifier-like strings before\n", - "prediction. Account numbers, product numbers, order numbers, approval codes,\n", - "`rowguid` columns, and `CarrierTrackingNumber` identify records rather than\n", - "categories, and several exceed the NIM's 10,000-value categorical cardinality\n", - "limit in the full extracts, so this example marks them `\"ID\"`. Product names\n", - "are typed `\"categorical\"`, matching the schema configuration below; see\n", - "[Data Types](https://docs.nvidia.com/sdgm/kumo-relational/user-guide/prepare-data/data-types).\n", - "Order quantity is a measured numerical value." + "Keys and relationship columns use the `ID` semantic type. The next cell\n", + "omits identifier-like strings, free-text comments, and fields that can reveal\n", + "information recorded after an anchor, including modification times, ship dates,\n", + "and product sell dates. Product `Name` remains categorical and `OrderQty` is\n", + "numerical. See\n", + "[Data Types](https://docs.nvidia.com/sdgm/client/kumo-relational/user-guide/prepare-data/data-types).\n", + "\n", + "This removes known leakage-prone fields; it does not reconstruct earlier values\n", + "of the remaining untimed customer and product attributes." ] }, { @@ -510,30 +493,34 @@ }, "outputs": [], "source": [ - "# Correct semantic types that are specific to this known schema. These values\n", - "# identify business records rather than categories or free-text documents.\n", - "id_columns = {\n", - " 'customers': ['AccountNumber', 'rowguid'],\n", - " 'products': ['ProductNumber', 'rowguid'],\n", + "leave_out = {\n", + " 'customers': ['rowguid', 'ModifiedDate', 'AccountNumber'],\n", + " 'products': [\n", + " 'rowguid',\n", + " 'ModifiedDate',\n", + " 'ProductNumber',\n", + " 'SellStartDate',\n", + " 'SellEndDate',\n", + " 'DiscontinuedDate',\n", + " ],\n", " 'sales_order_headers': [\n", + " 'rowguid',\n", + " 'ModifiedDate',\n", " 'SalesOrderNumber',\n", " 'PurchaseOrderNumber',\n", " 'AccountNumber',\n", " 'CreditCardApprovalCode',\n", - " 'rowguid',\n", + " 'Comment',\n", + " 'RevisionNumber',\n", + " 'Status',\n", + " 'ShipDate',\n", " ],\n", - " 'sales_order_details': ['CarrierTrackingNumber', 'rowguid'],\n", + " 'sales_order_details': ['rowguid', 'ModifiedDate', 'CarrierTrackingNumber'],\n", "}\n", - "\n", - "for table_name, column_names in id_columns.items():\n", + "for table_name, column_names in leave_out.items():\n", " for column_name in column_names:\n", - " if (\n", - " column_name in schema[table_name].columns\n", - " ): # skip columns not in the Snowflake table\n", - " schema[table_name][column_name].stype = 'ID'\n", + " del schema[table_name][column_name]\n", "\n", - "# Product names carry natural-language meaning; order quantity is measured.\n", - "# This NIM does not read the 'text' stype, so Name is typed categorical.\n", "schema['products']['Name'].stype = 'categorical'\n", "schema['sales_order_details']['OrderQty'].stype = 'numerical'" ] @@ -720,10 +707,13 @@ }, "outputs": [], "source": [ - "# Pin the anchor instead of letting it be derived. It is taken from the\n", - "# data rather than written as a literal: AdventureWorks is re-dated\n", - "# upstream, so a hardcoded date stops matching the file you just read.\n", - "anchor_time = sales_order_headers_df['OrderDate'].max() - pd.Timedelta(days=60)\n", + "# Query only the latest timestamp rather than loading the order table.\n", + "latest_order = (\n", + " session.table(f'{DATA_LOCATION}.\"SalesOrderHeader\"')\n", + " .select(F.max(F.col('\"OrderDate\"')))\n", + " .collect()[0][0]\n", + ")\n", + "anchor_time = pd.Timestamp(latest_order) - pd.Timedelta(days=60)\n", "\n", "request = query.materialize(indices=[870], anchor_time=anchor_time)\n", "result = client.predict(request, model=KumoRelational())\n", @@ -848,12 +838,14 @@ }, "outputs": [], "source": [ + "# Collect only the 15 candidate IDs; aggregate purchase counts in Snowflake.\n", "candidate_product_ids = (\n", - " sales_order_details_df['ProductID']\n", - " .value_counts()\n", - " .head(15)\n", - " .index.to_series()\n", - " .reset_index(drop=True)\n", + " session.table(f'{DATA_LOCATION}.\"SalesOrderDetail\"')\n", + " .group_by(F.col('\"ProductID\"'))\n", + " .agg(F.count('*').alias('purchase_count'))\n", + " .sort(F.col('purchase_count').desc(), F.col('\"ProductID\"'))\n", + " .limit(15)\n", + " .to_pandas()['ProductID']\n", ")\n", "\n", "query_rank = PredictiveQuery(\n", diff --git a/examples/snowflake/kumo_tabular_quickstart.ipynb b/examples/snowflake/kumo_tabular_quickstart.ipynb index 42ed234a4..71ffd1394 100644 --- a/examples/snowflake/kumo_tabular_quickstart.ipynb +++ b/examples/snowflake/kumo_tabular_quickstart.ipynb @@ -38,9 +38,20 @@ "source": [ "## 1. Install and connect\n", "\n", - "Use a Python 3.10 or newer kernel with network access to the NIM endpoint and an active Snowflake session for the dataset. The notebook also uses pandas, NumPy, and scikit-learn.\n", + "Use a Snowflake notebook on **Container Runtime** with an active Snowpark\n", + "session. Configure an external access integration for PyPI package installation\n", + "and attach it to the notebook service. If the NIM is outside Snowflake, allow\n", + "its host through an external access integration too. See Snowflake's\n", + "[external access setup](https://docs.snowflake.com/en/user-guide/ui-snowsight/notebooks-external-access).\n", + "The notebook itself does not need a GPU; inference runs on the NIM service.\n", "\n", - "Install the base client; the relational extra is not needed. See [Install and Connect](https://docs.nvidia.com/sdgm/kumo-tabular/get-started/sdk-getting-started)." + "Use a supported Python kernel with pandas, NumPy, and scikit-learn.\n", + "\n", + "Install the base client; the relational extra is not needed. See [Install and Connect](https://docs.nvidia.com/sdgm/kumo-tabular/get-started/sdk-getting-started).\n", + "\n", + "The `http://:8000` placeholder is the NIM's private SPCS service\n", + "address. For endpoints outside that private service network, use `https://`\n", + "with TLS termination. Plain HTTP transmits table rows unencrypted." ] }, { @@ -104,7 +115,13 @@ "\n", "**Task:** Binary classification of breast masses: benign (0) or malignant (1)\n", "\n", - "Use four predictors: `Age`, `Shape`, `Margin`, and `Density`." + "Use four predictors: `Age`, `Shape`, `Margin`, and `Density`.\n", + "\n", + "Download `Cleaned_data.csv` from the Kaggle dataset page and upload it yourself\n", + "to a Snowflake stage with that exact filename. Replace the stage path in the\n", + "next cell with its location. Your notebook role needs permission to read it.\n", + "\n", + "Each user is responsible for checking the content of datasets and the applicable licenses and determining if suitable for the intended use." ] }, { diff --git a/packages/structured-data-models-client/tests/test_snowflake_notebooks.py b/packages/structured-data-models-client/tests/test_snowflake_notebooks.py new file mode 100644 index 000000000..ca35bbcaa --- /dev/null +++ b/packages/structured-data-models-client/tests/test_snowflake_notebooks.py @@ -0,0 +1,111 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. +# All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Check the Snowflake example's table layout and source-backed schema offline.""" + +import ast +import datetime +import json +import re +from pathlib import Path + +import pytest +from sdm_connectors import DuckDBSource + +from sdm_client import Relationship, Schema, SourceTable + +NOTEBOOK = ( + Path(__file__).parents[3] + / 'examples/snowflake/kumo_relational_quickstart.ipynb' +) + + +def test_uploaded_tables_join_and_omit_future_information(): + duckdb = pytest.importorskip('duckdb') + pytest.importorskip('sdm_relational_engine') + cells = json.loads(NOTEBOOK.read_text())['cells'] + sources = [''.join(cell['source']) for cell in cells] + setup = next( + source for source in sources if 'CREATE TABLE IF NOT EXISTS' in source + ) + connection = duckdb.connect() + try: + for statement in re.findall( + r'CREATE TABLE IF NOT EXISTS .*?;', setup, re.DOTALL + ): + statement = statement.replace('..', '') + connection.execute(statement.replace('TIMESTAMP_NTZ', 'TIMESTAMP')) + + for table in ( + 'Customer', + 'Product', + 'SalesOrderHeader', + 'SalesOrderDetail', + ): + columns = connection.execute(f'DESCRIBE "{table}"').fetchall() + row = {} + for name, kind, *_ in columns: + if kind == 'TIMESTAMP': + row[name] = datetime.datetime(2024, 1, 1) + elif kind in ('BIGINT', 'INTEGER', 'FLOAT', 'DOUBLE'): + row[name] = 1 + else: + row[name] = 'example' + placeholders = ', '.join('?' for _ in row) + connection.execute( + f'INSERT INTO "{table}" VALUES ({placeholders})', + list(row.values()), + ) + + view = next( + source for source in sources if source.startswith('session.sql(') + ) + sql_call = ast.parse(view).body[0].value.func.value + sql = eval( + compile(ast.Expression(sql_call.args[0]), '', 'eval'), + {'DATA_LOCATION': 'main'}, + ) + # DuckDB cursors do not share temporary views across connections. + connection.execute(sql.replace('TEMPORARY ', '')) + joined = connection.execute( + 'SELECT "SalesOrderDetailID", "CustomerID", "OrderDate" ' + 'FROM "SDM_SalesOrderDetail"' + ).fetchall() + assert joined == [(1, 1, datetime.datetime(2024, 1, 1))] + + source = DuckDBSource(connection) + namespace = { + 'session': object(), + 'DATABASE': 'DB', + 'SCHEMA': 'PUBLIC', + 'SnowflakeSource': lambda session, *, database, schema: source, + 'Schema': Schema, + 'Relationship': Relationship, + } + for marker in ('source = SnowflakeSource(', 'leave_out = {'): + code = next(item for item in sources if item.startswith(marker)) + exec(compile(code, '', 'exec'), namespace) + schema = namespace['schema'] + schema.check_consistent() + assert schema.source is source + assert all( + isinstance(table, SourceTable) for table in schema.tables.values() + ) + assert set(schema.tables) == { + 'customers', + 'products', + 'sales_order_headers', + 'sales_order_details', + } + assert schema['sales_order_details'].time_column == 'OrderDate' + assert schema['products']['Name'].stype == 'categorical' + assert schema['sales_order_details']['OrderQty'].stype == 'numerical' + for table in schema.tables.values(): + assert 'ModifiedDate' not in table.columns + assert 'ShipDate' not in schema['sales_order_headers'].columns + assert not {'SellStartDate', 'SellEndDate', 'DiscontinuedDate'} & set( + schema['products'].columns + ) + finally: + connection.close() From e1a3ff574bcebda7c646b24b4c8a9cd3b02ec1ae Mon Sep 17 00:00:00 2001 From: k-ruthvika Date: Wed, 18 Nov 2026 16:44:28 -0500 Subject: [PATCH 4/5] examples: refresh Snowflake Relational notebook and fix namespace casing Signed-off-by: k-ruthvika --- .../kumo_relational_quickstart.ipynb | 64 ++++++------------- .../snowflake/kumo_tabular_quickstart.ipynb | 4 +- .../tests/test_snowflake_notebooks.py | 53 ++++++++++----- 3 files changed, 60 insertions(+), 61 deletions(-) diff --git a/examples/snowflake/kumo_relational_quickstart.ipynb b/examples/snowflake/kumo_relational_quickstart.ipynb index a5c59aef0..0b4cbe4c2 100644 --- a/examples/snowflake/kumo_relational_quickstart.ipynb +++ b/examples/snowflake/kumo_relational_quickstart.ipynb @@ -138,7 +138,7 @@ "Deploy the NIM first using the\n", "[Snowflake deployment guide](https://docs.nvidia.com/sdgm/client/kumo-relational/deployment/snowflake).\n", "Set `KUMO_RELATIONAL_API_ENDPOINT` to its reachable address. The\n", - "`http://:8000` placeholder below is a private SPCS service address,\n", + "`http://:8000` placeholder below is a private Snowpark Container Services (SPCS) service address,\n", "not a container running in the notebook.\n", "\n", "For endpoints outside that private service network, use an `https://` URL with\n", @@ -225,9 +225,12 @@ "Upload the four files yourself to an internal Snowflake stage. In a SQL\n", "worksheet, replace ``, ``, and `` with your destination\n", "names, then create and load these tables in a dedicated schema. The quoted\n", - "identifiers preserve the capitalization used by the notebook. Your role needs\n", + "table and column identifiers preserve the capitalization used by the notebook.\n", + "Use unquoted database and schema names in both SQL and the Python configuration\n", + "below. The Python cell normalizes them to uppercase, matching the SQL setup\n", + "and the names passed to `SnowflakeSource`. Your role needs\n", "access to the database, schema, warehouse, and stage, permission to create and\n", - "load these tables, and permission to create the context view used below.\n", + "load these tables, and permission to create the temporary context table used below.\n", "\n", "Each user is responsible for checking the content of datasets and the applicable licenses and determining if suitable for the intended use.\n", "Microsoft's samples are provided under the\n", @@ -350,9 +353,10 @@ "from snowflake.snowpark.context import get_active_session\n", "\n", "session = get_active_session()\n", - "DATABASE = ''\n", - "SCHEMA = ''\n", - "DATA_LOCATION = f'\"{DATABASE}\".\"{SCHEMA}\"'" + "# Match Snowflake's handling of the unquoted names in the SQL setup.\n", + "DATABASE = ''.upper()\n", + "SCHEMA = ''.upper()\n", + "DATA_LOCATION = f'{DATABASE}.{SCHEMA}'" ] }, { @@ -362,9 +366,9 @@ "codeCollapsed": true }, "source": [ - "Create a temporary view that joins each order line to its parent order's\n", + "Create a temporary table that joins each order line to its parent order's\n", "`CustomerID` and `OrderDate`. The uploaded source tables stay unchanged.\n", - "`SnowflakeSource` uses this same session, so the view remains visible when the\n", + "`SnowflakeSource` uses this same session, so the temporary table remains visible when the\n", "client samples context. Rerun this cell after restarting the session." ] }, @@ -377,7 +381,10 @@ }, "outputs": [], "source": [ - "session.sql(f\"\"\"CREATE OR REPLACE TEMPORARY VIEW\n", + "session.sql(\n", + " f'DROP VIEW IF EXISTS {DATA_LOCATION}.\"SDM_SalesOrderDetail\"'\n", + ").collect()\n", + "session.sql(f\"\"\"CREATE OR REPLACE TEMPORARY TABLE\n", " {DATA_LOCATION}.\"SDM_SalesOrderDetail\" AS\n", "SELECT d.*, h.\"CustomerID\", h.\"OrderDate\"\n", "FROM {DATA_LOCATION}.\"SalesOrderDetail\" AS d\n", @@ -474,14 +481,13 @@ }, "source": [ "Keys and relationship columns use the `ID` semantic type. The next cell\n", - "omits identifier-like strings, free-text comments, and fields that can reveal\n", - "information recorded after an anchor, including modification times, ship dates,\n", - "and product sell dates. Product `Name` remains categorical and `OrderQty` is\n", - "numerical. See\n", + "sets product `Name` to categorical and `OrderQty` to numerical. See\n", "[Data Types](https://docs.nvidia.com/sdgm/client/kumo-relational/user-guide/prepare-data/data-types).\n", "\n", - "This removes known leakage-prone fields; it does not reconstruct earlier values\n", - "of the remaining untimed customer and product attributes." + "This example retains the other inferred columns. Before a point-in-time\n", + "evaluation, exclude fields that reveal information recorded after the anchor,\n", + "such as modification times and ship dates. An anchor does not reconstruct\n", + "earlier values of untimed customer and product attributes." ] }, { @@ -493,34 +499,6 @@ }, "outputs": [], "source": [ - "leave_out = {\n", - " 'customers': ['rowguid', 'ModifiedDate', 'AccountNumber'],\n", - " 'products': [\n", - " 'rowguid',\n", - " 'ModifiedDate',\n", - " 'ProductNumber',\n", - " 'SellStartDate',\n", - " 'SellEndDate',\n", - " 'DiscontinuedDate',\n", - " ],\n", - " 'sales_order_headers': [\n", - " 'rowguid',\n", - " 'ModifiedDate',\n", - " 'SalesOrderNumber',\n", - " 'PurchaseOrderNumber',\n", - " 'AccountNumber',\n", - " 'CreditCardApprovalCode',\n", - " 'Comment',\n", - " 'RevisionNumber',\n", - " 'Status',\n", - " 'ShipDate',\n", - " ],\n", - " 'sales_order_details': ['rowguid', 'ModifiedDate', 'CarrierTrackingNumber'],\n", - "}\n", - "for table_name, column_names in leave_out.items():\n", - " for column_name in column_names:\n", - " del schema[table_name][column_name]\n", - "\n", "schema['products']['Name'].stype = 'categorical'\n", "schema['sales_order_details']['OrderQty'].stype = 'numerical'" ] diff --git a/examples/snowflake/kumo_tabular_quickstart.ipynb b/examples/snowflake/kumo_tabular_quickstart.ipynb index 71ffd1394..63f8d449a 100644 --- a/examples/snowflake/kumo_tabular_quickstart.ipynb +++ b/examples/snowflake/kumo_tabular_quickstart.ipynb @@ -49,8 +49,8 @@ "\n", "Install the base client; the relational extra is not needed. See [Install and Connect](https://docs.nvidia.com/sdgm/kumo-tabular/get-started/sdk-getting-started).\n", "\n", - "The `http://:8000` placeholder is the NIM's private SPCS service\n", - "address. For endpoints outside that private service network, use `https://`\n", + "The `http://:8000` placeholder is the NIM's private Snowpark Container Services (SPCS)\n", + "service address. For endpoints outside that private service network, use `https://`\n", "with TLS termination. Plain HTTP transmits table rows unencrypted." ] }, diff --git a/packages/structured-data-models-client/tests/test_snowflake_notebooks.py b/packages/structured-data-models-client/tests/test_snowflake_notebooks.py index ca35bbcaa..563346506 100644 --- a/packages/structured-data-models-client/tests/test_snowflake_notebooks.py +++ b/packages/structured-data-models-client/tests/test_snowflake_notebooks.py @@ -21,7 +21,7 @@ ) -def test_uploaded_tables_join_and_omit_future_information(): +def test_uploaded_tables_join_and_build_source_schema(): duckdb = pytest.importorskip('duckdb') pytest.importorskip('sdm_relational_engine') cells = json.loads(NOTEBOOK.read_text())['cells'] @@ -58,16 +58,17 @@ def test_uploaded_tables_join_and_omit_future_information(): list(row.values()), ) - view = next( + context_setup = next( source for source in sources if source.startswith('session.sql(') ) - sql_call = ast.parse(view).body[0].value.func.value - sql = eval( - compile(ast.Expression(sql_call.args[0]), '', 'eval'), - {'DATA_LOCATION': 'main'}, - ) - # DuckDB cursors do not share temporary views across connections. - connection.execute(sql.replace('TEMPORARY ', '')) + for statement in ast.parse(context_setup).body: + sql_call = statement.value.func.value + sql = eval( + compile(ast.Expression(sql_call.args[0]), '', 'eval'), + {'DATA_LOCATION': 'main'}, + ) + # DuckDB cursors do not share temporary tables across connections. + connection.execute(sql.replace('TEMPORARY ', '')) joined = connection.execute( 'SELECT "SalesOrderDetailID", "CustomerID", "OrderDate" ' 'FROM "SDM_SalesOrderDetail"' @@ -83,7 +84,7 @@ def test_uploaded_tables_join_and_omit_future_information(): 'Schema': Schema, 'Relationship': Relationship, } - for marker in ('source = SnowflakeSource(', 'leave_out = {'): + for marker in ('source = SnowflakeSource(', "schema['products']"): code = next(item for item in sources if item.startswith(marker)) exec(compile(code, '', 'exec'), namespace) schema = namespace['schema'] @@ -101,11 +102,31 @@ def test_uploaded_tables_join_and_omit_future_information(): assert schema['sales_order_details'].time_column == 'OrderDate' assert schema['products']['Name'].stype == 'categorical' assert schema['sales_order_details']['OrderQty'].stype == 'numerical' - for table in schema.tables.values(): - assert 'ModifiedDate' not in table.columns - assert 'ShipDate' not in schema['sales_order_headers'].columns - assert not {'SellStartDate', 'SellEndDate', 'DiscontinuedDate'} & set( - schema['products'].columns - ) finally: connection.close() + + +def test_lowercase_namespace_matches_unquoted_snowflake_setup(): + cells = json.loads(NOTEBOOK.read_text())['cells'] + configuration = next( + ''.join(cell['source']) + for cell in cells + if "DATABASE = ''" in ''.join(cell['source']) + ) + configuration = configuration.replace('', 'example_db').replace( + '', 'example_schema' + ) + # Exercise the configuration without requiring Snowpark or a live session. + assignments = ast.Module( + body=[ + node + for node in ast.parse(configuration).body + if isinstance(node, ast.Assign) + ], + type_ignores=[], + ) + namespace = {'get_active_session': lambda: object()} + exec(compile(assignments, '', 'exec'), namespace) + assert namespace['DATABASE'] == 'EXAMPLE_DB' + assert namespace['SCHEMA'] == 'EXAMPLE_SCHEMA' + assert namespace['DATA_LOCATION'] == 'EXAMPLE_DB.EXAMPLE_SCHEMA' From 1286e872b55a4707483869933a8dbd0ef1b18aab Mon Sep 17 00:00:00 2001 From: k-ruthvika Date: Wed, 18 Nov 2026 17:13:02 -0500 Subject: [PATCH 5/5] examples: use generic Kumo Tabular quickstart title Signed-off-by: k-ruthvika --- examples/snowflake/kumo_tabular_quickstart.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/snowflake/kumo_tabular_quickstart.ipynb b/examples/snowflake/kumo_tabular_quickstart.ipynb index 63f8d449a..b4693d774 100644 --- a/examples/snowflake/kumo_tabular_quickstart.ipynb +++ b/examples/snowflake/kumo_tabular_quickstart.ipynb @@ -14,7 +14,7 @@ "source": [ "![NVIDIA Structured Data Models logo](attachment:logo.png)\n", "\n", - "# Kumo Tabular on Breast Mass Classification (Kaggle)\n", + "# Kumo Tabular Quickstart with structured-data-models-client\n", "\n", "**Kumo Tabular** is a structured-data foundation model for predictive AI on a single table. It uses labelled context rows to predict outcomes for new rows without training a task-specific model.\n", "\n",