From 3960dd9286baee8ee95747a0e2ec72680f68359a Mon Sep 17 00:00:00 2001 From: aliu65803 Date: Tue, 17 Nov 2026 21:43:39 -0600 Subject: [PATCH 1/4] Add synthetic telco churn example notebooks Add a direct KumoTabular notebook on a single engineered feature table and a Kumo Engine notebook on seven related tables, and index both under a new Telecommunications section. Co-Authored-By: Claude Opus 5.5 --- examples/kumo-example-notebooks/README.md | 2 + .../kumo-engine/README.md | 8 + ...al_kumo_engine_synthetic_telco_churn.ipynb | 1390 +++++++++++++++++ .../structured-data-models/README.md | 8 + ...l_structured_data_models_telco_churn.ipynb | 944 +++++++++++ 5 files changed, 2352 insertions(+) create mode 100644 examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb create mode 100644 examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb diff --git a/examples/kumo-example-notebooks/README.md b/examples/kumo-example-notebooks/README.md index b6fea2a36..38e8994d6 100644 --- a/examples/kumo-example-notebooks/README.md +++ b/examples/kumo-example-notebooks/README.md @@ -14,6 +14,7 @@ Notebooks and Python scripts for training, prediction and model reuse. - [Media and entertainment](kumo-engine/README.md#media-and-entertainment) - [Retail](kumo-engine/README.md#retail) - [Science](kumo-engine/README.md#science) + - [Telecommunications](kumo-engine/README.md#telecommunications) - [Structured Data Models](structured-data-models/README.md) - [Relational](structured-data-models/README.md#relational) - [Cross-industry](structured-data-models/README.md#cross-industry) @@ -29,6 +30,7 @@ Notebooks and Python scripts for training, prediction and model reuse. - [Hospitality](structured-data-models/README.md#hospitality) - [Real estate](structured-data-models/README.md#real-estate) - [Retail](structured-data-models/README.md#retail-1) + - [Telecommunications](structured-data-models/README.md#telecommunications) - [Time series](structured-data-models/README.md#time-series) - [Cross-industry](structured-data-models/README.md#cross-industry-2) - [Retail](structured-data-models/README.md#retail-2) diff --git a/examples/kumo-example-notebooks/kumo-engine/README.md b/examples/kumo-example-notebooks/kumo-engine/README.md index b93db0615..fbcfc1e25 100644 --- a/examples/kumo-example-notebooks/kumo-engine/README.md +++ b/examples/kumo-example-notebooks/kumo-engine/README.md @@ -157,6 +157,14 @@ The synthetic industry notebooks below generate their own related tables and use | --- | --- | --- | | [Forecast monthly sunspot counts from the public SILSO series](science/sdk-code-examples/local_kumo_engine_sunspot_forecasting.py) | Local | Model workflow | +## Telecommunications + +### Notebooks + +| Example | Deployment | Scope | +| --- | --- | --- | +| [Predict telco customer churn in the next 30 days](telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb) | Local | Synthetic temporal binary classification | + Commented previews preserve demonstrations of features unavailable in the supported package API. Their commented demonstrations are not runnable. Example-specific companion directories hold shared data preparation, helpers and assets; use them through their owning examples. diff --git a/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb b/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb new file mode 100644 index 000000000..397266a6d --- /dev/null +++ b/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb @@ -0,0 +1,1390 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "cell-00", + "metadata": {}, + "source": [ + "# Predict telco customer churn with Kumo Engine\n", + "\n", + "Which mobile and broadband customers will cancel in the next 30 days? Train a Kumo Engine graph model directly on synthetic relational telco tables (customers, plans, subscriptions, weekly usage, bills, support tickets and cancellations), with no manual feature engineering, then rescore the active base when a new month of data lands.\n", + "\n", + "Use a Linux notebook kernel with a supported NVIDIA GPU and local SSD scratch. Complete [Linux deployment setup](https://docs.nvidia.com/sdgm/kumo-engine/get-started-with-kumo-engine/installation/linux-machine-deployment) and check [GPU sizing](https://docs.nvidia.com/sdgm/kumo-engine/get-started-with-kumo-engine/gpu-sizing). This example follows the [binary classification workflow](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/code-examples/linux-deployment/binary-classification). All data is synthetic, so metrics are **demonstration only**." + ] + }, + { + "cell_type": "markdown", + "id": "cell-01", + "metadata": {}, + "source": [ + "## Install\n", + "\n", + "Install the Engine SDK and the data libraries used by this notebook." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "cell-02", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:26.418626Z", + "iopub.status.busy": "2026-11-17T23:34:26.418403Z", + "iopub.status.idle": "2026-11-17T23:34:27.412968Z", + "shell.execute_reply": "2026-11-17T23:34:27.411831Z" + } + }, + "outputs": [], + "source": [ + "%pip install -q kumo-engine pandas numpy pyarrow" + ] + }, + { + "cell_type": "markdown", + "id": "cell-03", + "metadata": {}, + "source": [ + "## Import the local Engine APIs" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "cell-04", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:27.415541Z", + "iopub.status.busy": "2026-11-17T23:34:27.415279Z", + "iopub.status.idle": "2026-11-17T23:34:34.463367Z", + "shell.execute_reply": "2026-11-17T23:34:34.462294Z" + } + }, + "outputs": [], + "source": [ + "import json\n", + "import os\n", + "import shutil\n", + "from pathlib import Path\n", + "from tempfile import mkdtemp\n", + "\n", + "import numpy as np\n", + "import pandas as pd\n", + "from IPython.display import display\n", + "\n", + "import kumo.engine as kumo\n", + "from kumo.engine.trainer.output_config import ModelOutputConfig, PredictionOutputConfig, TrainingTableOutputConfig\n", + "from kumoapi.graph import Edge\n", + "from kumoapi.jobs import MetadataField, PredictionArtifactType\n", + "from kumoapi.model_plan import RunMode, TrainingTableGenerationPlan\n", + "from kumoapi.typing import Dtype, Stype" + ] + }, + { + "cell_type": "markdown", + "id": "cell-05", + "metadata": {}, + "source": [ + "## Set the example parameters\n", + "\n", + "Set `KUMO_SCRATCH` to a writable local SSD directory. The model trains on data before `TRAIN_END`; the month from `TRAIN_END` to `AS_OF` arrives later and is used only for batch scoring.\n", + "Leave `SMOKE_MODE=False` for the FAST tutorial plan; smoke mode trains for one epoch to check the pipeline only." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "cell-06", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:34.466378Z", + "iopub.status.busy": "2026-11-17T23:34:34.465946Z", + "iopub.status.idle": "2026-11-17T23:34:34.472963Z", + "shell.execute_reply": "2026-11-17T23:34:34.472034Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Run: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-9x83dxwe\n" + ] + } + ], + "source": [ + "SCRATCH_PARENT = Path(os.environ[\"KUMO_SCRATCH\"]).expanduser().resolve()\n", + "OUTPUT_PARENT = Path.cwd() / \"kumo_example_runs\"\n", + "RANDOM_SEED = 42\n", + "N_CUSTOMERS = 4000\n", + "SIM_START = pd.Timestamp(\"2024-01-01\")\n", + "TRAIN_END = pd.Timestamp(\"2025-06-01\")\n", + "AS_OF = pd.Timestamp(\"2025-07-01\")\n", + "SMOKE_MODE = False\n", + "DECISION_THRESHOLD = 0.5\n", + "\n", + "OUTPUT_PARENT.mkdir(parents=True, exist_ok=True)\n", + "run_dir = Path(mkdtemp(prefix=\"telco_churn-\", dir=OUTPUT_PARENT))\n", + "scratch_dir = Path(mkdtemp(prefix=\"telco_churn-\", dir=SCRATCH_PARENT))\n", + "source_dir, output_dir = run_dir / \"source\", run_dir / \"output\"\n", + "source_dir.mkdir()\n", + "output_dir.mkdir()\n", + "kumo.configure(scratch_dir=scratch_dir)\n", + "destination = kumo.FileConnector(root_dir=str(output_dir))\n", + "print(\"Run:\", run_dir)" + ] + }, + { + "cell_type": "markdown", + "id": "cell-07", + "metadata": {}, + "source": [ + "## Generate synthetic example data\n", + "\n", + "Run the next two cells to simulate 18 months of a consumer telco base. Churn risk rises with network problems, bill shock when a contract ends, the April price rise for value-segment customers, late payments and declining usage; customers with two products churn less. Rows on or after `TRAIN_END` are held back as the \"new month\" for scoring.\n", + "You can skim the generation code; the Kumo workflow starts at **Build the graph**." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "cell-08", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:34.475387Z", + "iopub.status.busy": "2026-11-17T23:34:34.475174Z", + "iopub.status.idle": "2026-11-17T23:34:34.746869Z", + "shell.execute_reply": "2026-11-17T23:34:34.745854Z" + } + }, + "outputs": [], + "source": [ + "rng = np.random.default_rng(RANDOM_SEED)\n", + "REGIONS = [\"north\", \"south\", \"east\", \"west\", \"central\"]\n", + "region_dropped_calls = {\"north\": 0.9, \"south\": 0.4, \"east\": 0.5, \"west\": 0.7, \"central\": 0.35}\n", + "plans = pd.DataFrame({\n", + " \"plan_id\": np.arange(1, 7, dtype=np.int64),\n", + " \"product\": [\"sim_only\", \"sim_only\", \"handset\", \"broadband\", \"broadband\", \"broadband_tv\"],\n", + " \"contract_months\": [1, 12, 24, 18, 24, 24],\n", + " \"monthly_price\": [10.0, 14.0, 38.0, 30.0, 45.0, 70.0],\n", + "})\n", + "\n", + "\n", + "def simulate_telco(n_customers, start, end, signup_from, recent_share):\n", + " start, end = pd.Timestamp(start), pd.Timestamp(end)\n", + " ids = np.arange(1, n_customers + 1, dtype=np.int64)\n", + " recent = rng.random(n_customers) < recent_share\n", + " signup = np.where(recent,\n", + " start + pd.to_timedelta(rng.integers(0, (end - start).days - 30, n_customers), unit=\"D\"),\n", + " pd.Timestamp(signup_from) + pd.to_timedelta(rng.integers(0, (start - pd.Timestamp(signup_from)).days, n_customers), unit=\"D\"))\n", + " customers = pd.DataFrame({\"customer_id\": ids, \"signup_date\": pd.to_datetime(signup),\n", + " \"region\": rng.choice(REGIONS, n_customers),\n", + " \"age_band\": rng.choice([\"18-24\", \"25-34\", \"35-54\", \"55+\"], n_customers),\n", + " \"acquisition_channel\": rng.choice([\"online\", \"retail_store\", \"telesales\"], n_customers),\n", + " \"segment\": rng.choice([\"value\", \"standard\", \"premium\"], n_customers, p=[0.35, 0.45, 0.2])})\n", + "\n", + " plan_id = rng.choice(plans.plan_id, n_customers, p=[0.15, 0.2, 0.25, 0.15, 0.1, 0.15])\n", + " contract_days = plans.set_index(\"plan_id\").contract_months[plan_id].to_numpy() * 30\n", + " contract_start = np.where(recent, customers.signup_date,\n", + " start - pd.to_timedelta(rng.integers(0, contract_days), unit=\"D\"))\n", + " contract_start = pd.to_datetime(contract_start)\n", + " contract_end = (contract_start + pd.to_timedelta(contract_days, unit=\"D\")).to_numpy()\n", + " second_line = rng.random(n_customers) < 0.3\n", + " subscriptions = pd.DataFrame({\"customer_id\": np.r_[ids, ids[second_line]],\n", + " \"plan_id\": np.r_[plan_id, np.where(plan_id[second_line] <= 3, 4, 2)],\n", + " \"started_at\": np.r_[contract_start, contract_start[second_line]],\n", + " \"contract_end_date\": np.r_[contract_end, contract_end[second_line]]})\n", + " subscriptions.insert(0, \"subscription_id\", np.arange(1, len(subscriptions) + 1, dtype=np.int64))\n", + " monthly_price = subscriptions.merge(plans, on=\"plan_id\").groupby(\"customer_id\").monthly_price.sum()[ids].to_numpy()\n", + "\n", + " base_usage = rng.gamma(2.0, 2.5, n_customers)\n", + " price_sensitive = (customers.segment.to_numpy() == \"value\") & (rng.random(n_customers) < 0.8)\n", + " financial_stress = rng.beta(1.5, 10, n_customers)\n", + " dropped_rate = customers.region.map(region_dropped_calls).to_numpy()\n", + " rolling = plan_id == 1\n", + " dissatisfaction = np.zeros(n_customers)\n", + " signup_dates = customers.signup_date.to_numpy()\n", + " churned_at = np.full(n_customers, np.datetime64(\"NaT\", \"ns\"))\n", + " day = np.timedelta64(1, \"D\")\n", + " last_late = np.full(n_customers, -99)\n", + " usage, bills, tickets = [], [], []\n", + "\n", + " for week, t in enumerate(pd.date_range(start, end - np.timedelta64(7, \"D\"), freq=\"7D\").to_numpy()):\n", + " active = (signup_dates <= t) & np.isnat(churned_at)\n", + " idx = np.flatnonzero(active)\n", + " outage = {r: 5.0 if rng.random() < 0.04 else 1.0 for r in REGIONS}\n", + " dropped = rng.poisson(dropped_rate[idx] * customers.region.iloc[idx].map(outage).to_numpy())\n", + " out_of_contract = contract_end[idx] < t\n", + " price_rise = 1.064 ** ((t >= pd.Timestamp(\"2024-04-01\")) + (t >= pd.Timestamp(\"2025-04-01\")))\n", + " recent_rise = (t >= pd.Timestamp(\"2025-04-01\")) & (t < pd.Timestamp(\"2025-05-15\"))\n", + " bill_shock = (out_of_contract & (contract_end[idx] > t - 60 * day)) | (recent_rise & price_sensitive[idx])\n", + "\n", + " network_ticket = rng.random(len(idx)) < 0.01 + 0.06 * dropped\n", + " billing_ticket = rng.random(len(idx)) < 0.01 + 0.12 * bill_shock\n", + " dissatisfaction[idx] = np.maximum(0, 0.8 * dissatisfaction[idx] + 1.2 * network_ticket\n", + " + 1.0 * billing_ticket + 0.05 * dropped + rng.normal(0, 0.05, len(idx)))\n", + " engagement = np.exp(-0.5 * dissatisfaction[idx])\n", + " usage.append(pd.DataFrame({\"customer_id\": ids[idx], \"week_start\": t,\n", + " \"data_gb\": (base_usage[idx] * engagement * rng.lognormal(0, 0.25, len(idx))).round(2),\n", + " \"voice_minutes\": (60 * engagement * rng.lognormal(0, 0.4, len(idx))).round(0),\n", + " \"dropped_calls\": dropped}))\n", + " for flag, category in ((network_ticket, \"network\"), (billing_ticket, \"billing\")):\n", + " tickets.append(pd.DataFrame({\"customer_id\": ids[idx[flag]], \"category\": category,\n", + " \"opened_at\": t + rng.integers(0, 7, flag.sum()) * day}))\n", + " if week % 4 == 0:\n", + " paid_late = rng.random(len(idx)) < 0.02 + 0.6 * financial_stress[idx]\n", + " last_late[idx[paid_late]] = week\n", + " amount = monthly_price[idx] * price_rise * np.where(out_of_contract & ~rolling[idx], 1.3, 1.0)\n", + " bills.append(pd.DataFrame({\"customer_id\": ids[idx], \"billed_at\": t,\n", + " \"amount\": amount.round(2), \"paid_late\": paid_late.astype(np.int64)}))\n", + "\n", + " logit = (-8.6 + 1.6 * dissatisfaction[idx] + 2.2 * out_of_contract + 1.2 * rolling[idx]\n", + " + 3.0 * (recent_rise & price_sensitive[idx]) + 2.0 * (week - last_late[idx] <= 4)\n", + " - 1.5 * np.isin(ids[idx], subscriptions.customer_id[subscriptions.customer_id.duplicated()]))\n", + " leaves = rng.random(len(idx)) < 1 / (1 + np.exp(-logit))\n", + " churned_at[idx[leaves]] = t + rng.integers(0, 7, leaves.sum()) * day\n", + "\n", + " usage, bills, tickets = (pd.concat(frames, ignore_index=True) for frames in (usage, bills, tickets))\n", + " cancellations = pd.DataFrame({\"customer_id\": ids, \"cancelled_at\": churned_at}).dropna()\n", + " for frame, key in ((usage, \"usage_id\"), (bills, \"bill_id\"), (tickets, \"ticket_id\"), (cancellations, \"cancellation_id\")):\n", + " frame.insert(0, key, np.arange(1, len(frame) + 1, dtype=np.int64))\n", + " return {\"customers\": customers, \"plans\": plans, \"subscriptions\": subscriptions, \"usage\": usage,\n", + " \"bills\": bills, \"support_tickets\": tickets, \"cancellations\": cancellations}" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "cell-09", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:34.749104Z", + "iopub.status.busy": "2026-11-17T23:34:34.748872Z", + "iopub.status.idle": "2026-11-17T23:34:35.202403Z", + "shell.execute_reply": "2026-11-17T23:34:35.201481Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "customers history= 4000 new= 0\n", + "plans history= 6 new= 0\n", + "subscriptions history= 5194 new= 0\n", + "usage history= 208877 new= 10481\n", + "bills history= 53589 new= 2616\n", + "support_tickets history= 15732 new= 780\n", + "cancellations history= 1350 new= 69\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
usage_idcustomer_idweek_startdata_gbvoice_minutesdropped_calls
0112024-01-0110.2746.01
1222024-01-014.5969.02
2332024-01-017.2291.00
\n", + "
" + ], + "text/plain": [ + " usage_id customer_id week_start data_gb voice_minutes dropped_calls\n", + "0 1 1 2024-01-01 10.27 46.0 1\n", + "1 2 2 2024-01-01 4.59 69.0 2\n", + "2 3 3 2024-01-01 7.22 91.0 0" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "tables = simulate_telco(N_CUSTOMERS, SIM_START, AS_OF, signup_from=\"2019-01-01\", recent_share=0.3)\n", + "time_columns = {\"customers\": \"signup_date\", \"subscriptions\": \"started_at\", \"usage\": \"week_start\",\n", + " \"bills\": \"billed_at\", \"support_tickets\": \"opened_at\", \"cancellations\": \"cancelled_at\"}\n", + "new_rows = {}\n", + "for name, frame in tables.items():\n", + " (source_dir / name).mkdir()\n", + " is_new = frame[time_columns[name]] >= TRAIN_END if name in time_columns else np.zeros(len(frame), bool)\n", + " frame[~is_new].to_parquet(source_dir / name / \"history.parquet\", index=False)\n", + " new_rows[name] = frame[is_new]\n", + " print(f\"{name:16s} history={(~is_new).sum():7d} new={is_new.sum():6d}\")\n", + "display(tables[\"usage\"].head(3))" + ] + }, + { + "cell_type": "markdown", + "id": "cell-10", + "metadata": {}, + "source": [ + "## Build the graph\n", + "\n", + "Each table becomes a graph node type and each foreign key an edge. Event tables carry a time column, so Kumo only uses history before each prediction time. See [tables and relationships](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/set-up-a-graph/graphs)." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "cell-11", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:35.204702Z", + "iopub.status.busy": "2026-11-17T23:34:35.204444Z", + "iopub.status.idle": "2026-11-17T23:34:35.577474Z", + "shell.execute_reply": "2026-11-17T23:34:35.576326Z" + } + }, + "outputs": [], + "source": [ + "files = kumo.FileConnector(root_dir=str(source_dir))\n", + "\n", + "\n", + "def table(name, primary_key, time_column=None, categorical=(), foreign_keys=(), dates=()):\n", + " t = kumo.Table.from_source_table(files[name], primary_key=primary_key, time_column=time_column)\n", + " for column in ([time_column] if time_column else []) + list(dates):\n", + " t = t.with_column(column, dtype=Dtype.date)\n", + " for column in categorical:\n", + " t = t.with_column(column, stype=Stype.categorical)\n", + " for column in foreign_keys:\n", + " t = t.with_column(column, stype=Stype.ID)\n", + " return t\n", + "\n", + "\n", + "graph = kumo.Graph(\n", + " tables={\n", + " \"customers\": table(\"customers\", \"customer_id\", \"signup_date\", [\"region\", \"age_band\", \"acquisition_channel\", \"segment\"]),\n", + " \"plans\": table(\"plans\", \"plan_id\", categorical=[\"product\"]),\n", + " \"subscriptions\": table(\"subscriptions\", \"subscription_id\", \"started_at\", foreign_keys=[\"customer_id\", \"plan_id\"],\n", + " dates=[\"contract_end_date\"]),\n", + " \"usage\": table(\"usage\", \"usage_id\", \"week_start\", foreign_keys=[\"customer_id\"]),\n", + " \"bills\": table(\"bills\", \"bill_id\", \"billed_at\", foreign_keys=[\"customer_id\"]),\n", + " \"support_tickets\": table(\"support_tickets\", \"ticket_id\", \"opened_at\", [\"category\"], [\"customer_id\"]),\n", + " \"cancellations\": table(\"cancellations\", \"cancellation_id\", \"cancelled_at\", foreign_keys=[\"customer_id\"]),\n", + " },\n", + " edges=[Edge(src, \"customer_id\", \"customers\") for src in (\"subscriptions\", \"usage\", \"bills\", \"support_tickets\", \"cancellations\")]\n", + " + [Edge(\"subscriptions\", \"plan_id\", \"plans\")],\n", + ")\n", + "graph.validate()" + ] + }, + { + "cell_type": "markdown", + "id": "cell-12", + "metadata": {}, + "source": [ + "## Inspect the graph\n", + "\n", + "[Graph visualization](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/set-up-a-graph/graphs) is a commented preview because the published package does not expose it yet; the relationship table below comes from the actual graph." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "cell-13", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:35.579877Z", + "iopub.status.busy": "2026-11-17T23:34:35.579646Z", + "iopub.status.idle": "2026-11-17T23:34:35.586747Z", + "shell.execute_reply": "2026-11-17T23:34:35.585921Z" + } + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
sourceforeign_keydestination
0subscriptionscustomer_idcustomers
1usagecustomer_idcustomers
2billscustomer_idcustomers
3support_ticketscustomer_idcustomers
4cancellationscustomer_idcustomers
5subscriptionsplan_idplans
\n", + "
" + ], + "text/plain": [ + " source foreign_key destination\n", + "0 subscriptions customer_id customers\n", + "1 usage customer_id customers\n", + "2 bills customer_id customers\n", + "3 support_tickets customer_id customers\n", + "4 cancellations customer_id customers\n", + "5 subscriptions plan_id plans" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Preview only: graph.visualize(show_cols=True)\n", + "display(pd.DataFrame([{\"source\": e.src_table, \"foreign_key\": e.fkey, \"destination\": e.dst_table} for e in graph.edges]))" + ] + }, + { + "cell_type": "markdown", + "id": "cell-14", + "metadata": {}, + "source": [ + "## Materialize the graph\n", + "\n", + "Snapshot the historical source files for training." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "cell-15", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:35.588776Z", + "iopub.status.busy": "2026-11-17T23:34:35.588556Z", + "iopub.status.idle": "2026-11-17T23:34:37.339016Z", + "shell.execute_reply": "2026-11-17T23:34:37.337982Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Training snapshot: 6054b25aa3392517642866d7b6bd7d38defcd6da755f7ce8b2896f0f28097e29@f91dac4d3aefe6e0\n" + ] + } + ], + "source": [ + "snapshot = graph.snapshot()\n", + "print(\"Training snapshot:\", snapshot.id)" + ] + }, + { + "cell_type": "markdown", + "id": "cell-16", + "metadata": {}, + "source": [ + "## Define the prediction task\n", + "\n", + "Churn is a cancellation in the 30 days after each prediction time. The `WHERE` clause restricts predictions to active customers, meaning those with usage in the previous 14 days, so customers who have already left are not scored. See [predictive queries](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/predictive-queries)." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "cell-17", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:37.341270Z", + "iopub.status.busy": "2026-11-17T23:34:37.341034Z", + "iopub.status.idle": "2026-11-17T23:34:37.368381Z", + "shell.execute_reply": "2026-11-17T23:34:37.367493Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Task: binary_classification\n" + ] + } + ], + "source": [ + "QUERY = (\"PREDICT COUNT(cancellations.*, 0, 30, days) > 0 \"\n", + " \"FOR EACH customers.customer_id WHERE COUNT(usage.*, -14, 0, days) > 0\")\n", + "query = kumo.PredictiveQuery(graph, QUERY)\n", + "query.validate()\n", + "print(\"Task:\", query.get_task_type())" + ] + }, + { + "cell_type": "markdown", + "id": "cell-18", + "metadata": {}, + "source": [ + "## Generate training rows and inspect the splits\n", + "\n", + "Create monthly prediction times over the last year of history and split them chronologically (70/15/15), so validation and test are later than training, as they would be in production. In the [training table](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/training-tables), `SPLIT` 0 is training, 1 validation and 2 test; `TARGET1` is the churn label." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "cell-19", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:37.370623Z", + "iopub.status.busy": "2026-11-17T23:34:37.370381Z", + "iopub.status.idle": "2026-11-17T23:34:37.693035Z", + "shell.execute_reply": "2026-11-17T23:34:37.692068Z" + } + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
rowschurn_ratefirst_anchorlast_anchor
SPLIT
0256950.0234682024-06-052025-01-31
157800.0482702025-03-022025-04-01
228100.0544482025-05-012025-05-01
\n", + "
" + ], + "text/plain": [ + " rows churn_rate first_anchor last_anchor\n", + "SPLIT \n", + "0 25695 0.023468 2024-06-05 2025-01-31\n", + "1 5780 0.048270 2025-03-02 2025-04-01\n", + "2 2810 0.054448 2025-05-01 2025-05-01" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "generation_plan = TrainingTableGenerationPlan(\n", + " split=\"ApproxDateOffsetSplit([0.7, 0.15, 0.15])\",\n", + " train_start_offset=360, train_end_offset=0, timeframe_step=30,\n", + ")\n", + "training_table = query.generate_training_table(snapshot, plan=generation_plan)\n", + "training_table.export(TrainingTableOutputConfig(connector=destination, table_name=\"training_rows/\"))\n", + "training_rows = pd.read_parquet(output_dir / \"training_rows\")\n", + "display(training_rows.groupby(\"SPLIT\").agg(rows=(\"ENTITY1\", \"size\"), churn_rate=(\"TARGET1\", \"mean\"),\n", + " first_anchor=(\"TIMESTAMP\", \"min\"), last_anchor=(\"TIMESTAMP\", \"max\")))" + ] + }, + { + "cell_type": "markdown", + "id": "cell-20", + "metadata": {}, + "source": [ + "## Configure the model\n", + "\n", + "Start from the [FAST plan](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/training). Churn is a minority outcome, so select the model on AUPRC and also report AUROC. Disable refitting to keep the test split unseen, and enable training explanations." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "cell-21", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:37.695524Z", + "iopub.status.busy": "2026-11-17T23:34:37.695252Z", + "iopub.status.idle": "2026-11-17T23:34:37.706183Z", + "shell.execute_reply": "2026-11-17T23:34:37.705215Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tuning metric: auprc | Max epochs: 8\n" + ] + } + ], + "source": [ + "plan = query.suggest_model_plan(snapshot, run_mode=RunMode.FAST)\n", + "plan.training_job.refit_trainval = False\n", + "plan.training_job.refit_full = False\n", + "plan.training_job.disable_explain = False\n", + "plan.training_job.metrics = [\"auprc\", \"auroc\"]\n", + "plan.training_job.tune_metric = \"auprc\"\n", + "if SMOKE_MODE:\n", + " plan.training_job.num_experiments = 1\n", + " plan.optimization.max_epochs = 1\n", + "(output_dir / \"model_plan.json\").write_text(json.dumps(plan.as_json(), indent=2) + \"\\n\")\n", + "print(\"Tuning metric:\", plan.training_job.tune_metric, \"| Max epochs:\", plan.optimization.max_epochs)" + ] + }, + { + "cell_type": "markdown", + "id": "cell-22", + "metadata": {}, + "source": [ + "## Train and retain the model\n", + "\n", + "Fit on the training rows and export the model bundle for reuse." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "cell-23", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:34:37.708177Z", + "iopub.status.busy": "2026-11-17T23:34:37.707954Z", + "iopub.status.idle": "2026-11-17T23:37:00.621180Z", + "shell.execute_reply": "2026-11-17T23:37:00.620235Z" + } + }, + "outputs": [], + "source": [ + "trainer = kumo.Trainer(plan)\n", + "fit_result = trainer.fit(training_table, snapshot=snapshot,\n", + " export_config=ModelOutputConfig(connector=destination, path=\"model\"))" + ] + }, + { + "cell_type": "markdown", + "id": "cell-24", + "metadata": {}, + "source": [ + "## Evaluate\n", + "\n", + "**Synthetic-data metrics: demonstration only.** AUROC measures how well the model ranks customers who cancel within 30 days above those who stay; AUPRC is more informative when churners are rare, and should be compared with the split's churn rate shown above." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "cell-25", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:37:00.624093Z", + "iopub.status.busy": "2026-11-17T23:37:00.623853Z", + "iopub.status.idle": "2026-11-17T23:37:00.632875Z", + "shell.execute_reply": "2026-11-17T23:37:00.632100Z" + } + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
value
metric
val_loss0.203904
val_auprc0.126241
val_auroc0.740524
test_loss0.227209
test_auprc0.143237
test_auroc0.747191
train_loss0.077621
train_auprc0.308035
train_auroc0.891191
\n", + "
" + ], + "text/plain": [ + " value\n", + "metric \n", + "val_loss 0.203904\n", + "val_auprc 0.126241\n", + "val_auroc 0.740524\n", + "test_loss 0.227209\n", + "test_auprc 0.143237\n", + "test_auroc 0.747191\n", + "train_loss 0.077621\n", + "train_auprc 0.308035\n", + "train_auroc 0.891191" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "(output_dir / \"metrics.json\").write_text(json.dumps(fit_result.metrics, indent=2, default=float) + \"\\n\")\n", + "display(pd.Series(fit_result.metrics, name=\"value\").rename_axis(\"metric\").to_frame())" + ] + }, + { + "cell_type": "markdown", + "id": "cell-26", + "metadata": {}, + "source": [ + "## Inspect training explanations\n", + "\n", + "[Training explanations](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/model-explainability) rank the table columns the selected model is most sensitive to on test examples. Sensitivity is not a causal effect." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "cell-27", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:37:00.635071Z", + "iopub.status.busy": "2026-11-17T23:37:00.634866Z", + "iopub.status.idle": "2026-11-17T23:37:00.663694Z", + "shell.execute_reply": "2026-11-17T23:37:00.662788Z" + } + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
tablecolumnscore
0subscriptionscontract_end_date0.377510
1subscriptionsstarted_at0.320016
2customerssignup_date0.142261
3support_ticketsopened_at0.035750
4customersregion0.022616
5customersage_band0.021190
6customerssegment0.015604
7customersacquisition_channel0.014025
8billsbilled_at0.012711
9support_ticketscategory0.011421
\n", + "
" + ], + "text/plain": [ + " table column score\n", + "0 subscriptions contract_end_date 0.377510\n", + "1 subscriptions started_at 0.320016\n", + "2 customers signup_date 0.142261\n", + "3 support_tickets opened_at 0.035750\n", + "4 customers region 0.022616\n", + "5 customers age_band 0.021190\n", + "6 customers segment 0.015604\n", + "7 customers acquisition_channel 0.014025\n", + "8 bills billed_at 0.012711\n", + "9 support_tickets category 0.011421" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "score_files = list(Path(fit_result.bundle_dir).parent.rglob(\"explain_test/scores.csv\"))\n", + "if score_files:\n", + " sensitivity = pd.read_csv(score_files[0]).sort_values(\"score\", ascending=False)\n", + " sensitivity.to_csv(output_dir / \"column_sensitivity.csv\", index=False)\n", + " display(sensitivity[[\"table\", \"column\", \"score\"]].head(10))\n", + "else:\n", + " print(\"No explanation scores found; check the explanation step in the fit logs.\")" + ] + }, + { + "cell_type": "markdown", + "id": "cell-28", + "metadata": {}, + "source": [ + "## Batch predictions\n", + "\n", + "A new month of data arrives. Append the usage, bills, support tickets and cancellations recorded from `TRAIN_END` to `AS_OF` as new Parquet parts, then snapshot again. The training snapshot is unchanged." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "cell-29", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:37:00.665688Z", + "iopub.status.busy": "2026-11-17T23:37:00.665458Z", + "iopub.status.idle": "2026-11-17T23:37:02.460636Z", + "shell.execute_reply": "2026-11-17T23:37:02.459577Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Scoring snapshot: 6054b25aa3392517642866d7b6bd7d38defcd6da755f7ce8b2896f0f28097e29@055d0798cd4f514b\n" + ] + } + ], + "source": [ + "for name, frame in new_rows.items():\n", + " if len(frame):\n", + " frame.to_parquet(source_dir / name / \"new.parquet\", index=False)\n", + "scoring_snapshot = graph.snapshot()\n", + "print(\"Scoring snapshot:\", scoring_snapshot.id)" + ] + }, + { + "cell_type": "markdown", + "id": "cell-30", + "metadata": {}, + "source": [ + "### Prepare the scoring population\n", + "\n", + "Score every customer active at `AS_OF`: signed up, not cancelled and with usage in the last 14 days. A [custom prediction table](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/making-predictions) lists those customer IDs with the scoring time and no labels." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "cell-31", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:37:02.463442Z", + "iopub.status.busy": "2026-11-17T23:37:02.463190Z", + "iopub.status.idle": "2026-11-17T23:37:02.731778Z", + "shell.execute_reply": "2026-11-17T23:37:02.730959Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Customers to score: 2581\n" + ] + } + ], + "source": [ + "recent_usage = tables[\"usage\"].loc[tables[\"usage\"].week_start >= AS_OF - np.timedelta64(14, \"D\"), \"customer_id\"]\n", + "active_ids = np.setdiff1d(recent_usage.unique(), tables[\"cancellations\"].customer_id)\n", + "scoring_population = pd.DataFrame({\"ENTITY1\": active_ids, \"TIMESTAMP\": AS_OF})\n", + "scoring_population.to_parquet(source_dir / \"scoring_population.parquet\", index=False)\n", + "prediction_table = kumo.PredictionTable.from_source_table(\n", + " files[\"scoring_population\"], pq=query, snapshot=scoring_snapshot, plan=generation_plan)\n", + "print(\"Customers to score:\", len(scoring_population))" + ] + }, + { + "cell_type": "markdown", + "id": "cell-32", + "metadata": {}, + "source": [ + "### Score the active base\n", + "\n", + "Run the trained model on the new snapshot and save predictions with their scoring time." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "cell-33", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:37:02.734557Z", + "iopub.status.busy": "2026-11-17T23:37:02.734304Z", + "iopub.status.idle": "2026-11-17T23:37:05.123720Z", + "shell.execute_reply": "2026-11-17T23:37:05.122734Z" + } + }, + "outputs": [], + "source": [ + "prediction_result = trainer.predict(\n", + " prediction_table, snapshot=scoring_snapshot,\n", + " binary_classification_threshold=DECISION_THRESHOLD,\n", + " output_config=PredictionOutputConfig(connector=destination,\n", + " artifact_type=PredictionArtifactType.PREDICTIONS, table_name=\"predictions.parquet\",\n", + " extra_fields=[MetadataField.ANCHOR_TIMESTAMP]),\n", + ")\n", + "scores = prediction_result.frame" + ] + }, + { + "cell_type": "markdown", + "id": "cell-34", + "metadata": {}, + "source": [ + "### Inspect the results\n", + "\n", + "There is one row per scored customer: `ENTITY` is the customer ID and `True_PROB` the probability of cancelling within 30 days. Sort by `True_PROB` to build a retention contact list; `TARGET_PRED` applies the untuned `DECISION_THRESHOLD`." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "cell-35", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:37:05.126204Z", + "iopub.status.busy": "2026-11-17T23:37:05.125952Z", + "iopub.status.idle": "2026-11-17T23:37:05.137212Z", + "shell.execute_reply": "2026-11-17T23:37:05.136749Z" + } + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
ENTITYANCHOR_TIMESTAMPTrue_PROB
89019112025-07-010.500259
131524832025-07-010.424149
25511392025-07-010.417413
186715942025-07-010.392592
189020642025-07-010.357828
151925862025-07-010.355676
88518142025-07-010.355617
127316102025-07-010.316929
16267252025-07-010.311057
244412462025-07-010.299319
\n", + "
" + ], + "text/plain": [ + " ENTITY ANCHOR_TIMESTAMP True_PROB\n", + "890 1911 2025-07-01 0.500259\n", + "1315 2483 2025-07-01 0.424149\n", + "255 1139 2025-07-01 0.417413\n", + "1867 1594 2025-07-01 0.392592\n", + "1890 2064 2025-07-01 0.357828\n", + "1519 2586 2025-07-01 0.355676\n", + "885 1814 2025-07-01 0.355617\n", + "1273 1610 2025-07-01 0.316929\n", + "1626 725 2025-07-01 0.311057\n", + "2444 1246 2025-07-01 0.299319" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Saved predictions: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-9x83dxwe/output/predictions.parquet\n" + ] + } + ], + "source": [ + "assert len(scores) == len(scoring_population) and set(scores.ENTITY) == set(active_ids)\n", + "display(scores.sort_values(\"True_PROB\", ascending=False)[[\"ENTITY\", \"ANCHOR_TIMESTAMP\", \"True_PROB\"]].head(10))\n", + "print(\"Saved predictions:\", output_dir / \"predictions.parquet\")" + ] + }, + { + "cell_type": "markdown", + "id": "cell-36", + "metadata": {}, + "source": [ + "## Retain and reuse\n", + "\n", + "Reload the saved model bundle without retraining. Keep the source tables and graph definition with the bundle: a later run rebuilds the graph and snapshot, then predicts as above. See [publish and retrieve models](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/publish-and-retrieve-models)." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "cell-37", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:37:05.139459Z", + "iopub.status.busy": "2026-11-17T23:37:05.139253Z", + "iopub.status.idle": "2026-11-17T23:37:05.204680Z", + "shell.execute_reply": "2026-11-17T23:37:05.203850Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Reloaded model from: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-9x83dxwe/output/model\n" + ] + } + ], + "source": [ + "reloaded_trainer = kumo.Trainer.load(output_dir / \"model\")\n", + "print(\"Reloaded model from:\", output_dir / \"model\")" + ] + }, + { + "cell_type": "markdown", + "id": "cell-38", + "metadata": {}, + "source": [ + "## Cleanup\n", + "\n", + "Generated data, the model and predictions stay in `run_dir`. Set `REMOVE_RUN_SCRATCH=True` to delete only this run's scratch directory." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "cell-39", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:37:05.206906Z", + "iopub.status.busy": "2026-11-17T23:37:05.206688Z", + "iopub.status.idle": "2026-11-17T23:37:05.209895Z", + "shell.execute_reply": "2026-11-17T23:37:05.209159Z" + } + }, + "outputs": [], + "source": [ + "REMOVE_RUN_SCRATCH = False\n", + "if REMOVE_RUN_SCRATCH:\n", + " shutil.rmtree(scratch_dir)\n", + " print(\"Removed run scratch:\", scratch_dir)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.16" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/kumo-example-notebooks/structured-data-models/README.md b/examples/kumo-example-notebooks/structured-data-models/README.md index 9de5abf4b..af72e21d1 100644 --- a/examples/kumo-example-notebooks/structured-data-models/README.md +++ b/examples/kumo-example-notebooks/structured-data-models/README.md @@ -158,6 +158,14 @@ Client examples require an existing endpoint; they do not provision one. | [Classify Otto Group products](tabular/retail/sdk-code-examples/local_structured_data_models_otto_group_product_classification_challenge.py) | Local | Model workflow | | [Predict Mercari listing prices from product metadata and text](tabular/retail/sdk-code-examples/local_structured_data_models_mercari_price_suggestion_challenge.py) | Local | Model workflow | +### Telecommunications + +#### Notebooks + +| Example | Deployment | Scope | +| --- | --- | --- | +| [Predict telco customer churn with direct KumoTabular](tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb) | Local | Synthetic binary classification | + ## Time series ### Cross-industry diff --git a/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb b/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb new file mode 100644 index 000000000..1777317da --- /dev/null +++ b/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb @@ -0,0 +1,944 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "cell-00", + "metadata": {}, + "source": [ + "# Predict telco customer churn with KumoTabular\n", + "\n", + "Which mobile and broadband customers are likely to cancel in the next 30 days? Score a synthetic table of customer features with pretrained KumoTabular, without training a model, and save a ranked list for a retention team.\n", + "\n", + "Runs locally on CPU or an NVIDIA GPU with Python 3.11 or newer. The first run downloads public [Kumo-Tabular weights](https://huggingface.co/nvidia/Kumo-Tabular). See the [KumoTabular API](https://nvidia.github.io/structured-data-models/api/generated/sdm.models.KumoTabular.html). All data is synthetic, so metrics are **demonstration only**." + ] + }, + { + "cell_type": "markdown", + "id": "cell-01", + "metadata": {}, + "source": [ + "## Install\n", + "\n", + "Install the Structured Data Models package and the data libraries used by this notebook." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "cell-02", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:39.105078Z", + "iopub.status.busy": "2026-11-17T23:27:39.104881Z", + "iopub.status.idle": "2026-11-17T23:27:40.004880Z", + "shell.execute_reply": "2026-11-17T23:27:40.003782Z" + } + }, + "outputs": [], + "source": [ + "%pip install -q structured-data-models pandas pyarrow scikit-learn" + ] + }, + { + "cell_type": "markdown", + "id": "cell-03", + "metadata": {}, + "source": [ + "## Import libraries" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "cell-04", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:40.007263Z", + "iopub.status.busy": "2026-11-17T23:27:40.007034Z", + "iopub.status.idle": "2026-11-17T23:27:43.769854Z", + "shell.execute_reply": "2026-11-17T23:27:43.768805Z" + } + }, + "outputs": [], + "source": [ + "import tempfile\n", + "from pathlib import Path\n", + "\n", + "import numpy as np\n", + "import pandas as pd\n", + "import torch\n", + "from IPython.display import display\n", + "from sklearn.metrics import average_precision_score, roc_auc_score\n", + "\n", + "import sdm\n", + "from sdm.explain import GradientExplainer\n", + "from sdm.models import KumoTabular" + ] + }, + { + "cell_type": "markdown", + "id": "cell-05", + "metadata": {}, + "source": [ + "## Set the example parameters\n", + "\n", + "Each cohort is a monthly customer snapshot. The context and held-out snapshots are far enough apart that their 30-day churn windows do not overlap, and the scoring snapshot has no known outcomes.\n", + "`TOP_SHARE` is the fraction of customers a retention team can contact. `MODEL_SIZE` can be raised to `\"medium\"` or `\"large\"` on a GPU; keep `NUM_ESTIMATORS = 1` for the explanation step." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "cell-06", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:43.772142Z", + "iopub.status.busy": "2026-11-17T23:27:43.771767Z", + "iopub.status.idle": "2026-11-17T23:27:43.800533Z", + "shell.execute_reply": "2026-11-17T23:27:43.799547Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Device: cuda | Outputs: /tmp/telco-churn-tabular-w63p8wsq\n" + ] + } + ], + "source": [ + "SEED = 7\n", + "N_CONTEXT, N_HELDOUT, N_SCORE = 3000, 1500, 1000\n", + "CONTEXT_TIME, HELDOUT_TIME, SCORE_TIME = \"2025-03-01\", \"2025-04-15\", \"2025-06-01\"\n", + "HORIZON_DAYS = 30\n", + "TOP_SHARE = 0.10\n", + "\n", + "MODEL_SIZE = \"small\"\n", + "DEVICE = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", + "NUM_ESTIMATORS = 1\n", + "\n", + "run_dir = Path(tempfile.mkdtemp(prefix=\"telco-churn-tabular-\"))\n", + "torch.manual_seed(SEED)\n", + "print(\"Device:\", DEVICE, \"| Outputs:\", run_dir)" + ] + }, + { + "cell_type": "markdown", + "id": "cell-07", + "metadata": {}, + "source": [ + "## Generate synthetic example data\n", + "\n", + "Run the next cell to create one row per customer per snapshot: plan and contract status, billing, usage trend, network quality and support contacts, plus whether the customer cancelled within 30 days. You can skim the generation code; the Kumo workflow starts at **Prepare context, held-out and scoring rows**." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "cell-08", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:43.802707Z", + "iopub.status.busy": "2026-11-17T23:27:43.802443Z", + "iopub.status.idle": "2026-11-17T23:27:43.925552Z", + "shell.execute_reply": "2026-11-17T23:27:43.924654Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " rows churn_rate\n", + "cohort \n", + "context 3000 0.148000\n", + "heldout 1500 0.146667\n", + "score 1000 NaN\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
customer_idcohortsnapshot_dateplanregiontenure_monthsnum_linesmonthly_chargecontract_months_leftprice_change_pct_12mdata_usage_gb_30dusage_change_pct_90ddropped_calls_30dnetwork_tickets_90dbilling_tickets_90dlate_payments_6mchurned_30d
0cust_000000context2025-03-01handset_24meast4138.69140.012.1-5.831000.0
1cust_000001context2025-03-01broadband_tveast652121.03216.412.0-1.810000.0
2cust_000002context2025-03-01broadbandwest36263.03-1125.07.216.111000.0
\n", + "
" + ], + "text/plain": [ + " customer_id cohort snapshot_date plan region tenure_months \\\n", + "0 cust_000000 context 2025-03-01 handset_24m east 4 \n", + "1 cust_000001 context 2025-03-01 broadband_tv east 65 \n", + "2 cust_000002 context 2025-03-01 broadband west 36 \n", + "\n", + " num_lines monthly_charge contract_months_left price_change_pct_12m \\\n", + "0 1 38.69 14 0.0 \n", + "1 2 121.03 21 6.4 \n", + "2 2 63.03 -11 25.0 \n", + "\n", + " data_usage_gb_30d usage_change_pct_90d dropped_calls_30d \\\n", + "0 12.1 -5.8 3 \n", + "1 12.0 -1.8 1 \n", + "2 7.2 16.1 1 \n", + "\n", + " network_tickets_90d billing_tickets_90d late_payments_6m churned_30d \n", + "0 1 0 0 0.0 \n", + "1 0 0 0 0.0 \n", + "2 1 0 0 0.0 " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "rng = np.random.default_rng(SEED)\n", + "cohort_sizes = {\"context\": N_CONTEXT, \"heldout\": N_HELDOUT, \"score\": N_SCORE}\n", + "snapshot_times = {\"context\": CONTEXT_TIME, \"heldout\": HELDOUT_TIME, \"score\": SCORE_TIME}\n", + "n = sum(cohort_sizes.values())\n", + "cohort = np.repeat(list(cohort_sizes), list(cohort_sizes.values()))\n", + "\n", + "plan_prices = {\"sim_only_rolling\": 10, \"sim_only_12m\": 14, \"handset_24m\": 38, \"broadband\": 32, \"broadband_tv\": 68}\n", + "plan = rng.choice(list(plan_prices), n, p=[0.18, 0.2, 0.27, 0.2, 0.15])\n", + "region = rng.choice([\"north\", \"south\", \"east\", \"west\", \"central\"], n)\n", + "region_quality = pd.Series({\"north\": 1.6, \"south\": 0.8, \"east\": 1.0, \"west\": 1.2, \"central\": 0.7})[region].to_numpy()\n", + "num_lines = 1 + rng.poisson(0.5, n)\n", + "months_left = np.where(plan == \"sim_only_rolling\", 0, rng.integers(-18, 24, n))\n", + "out_of_contract = months_left <= 0\n", + "price_change = np.where(out_of_contract & (plan != \"sim_only_rolling\"), rng.choice([0, 15, 25], n), 0) + rng.choice([0, 3.9, 6.4], n)\n", + "dissatisfaction = rng.gamma(1.0, 0.6, n)\n", + "stress = rng.beta(1.5, 8, n)\n", + "\n", + "customers = pd.DataFrame({\n", + " \"customer_id\": [f\"cust_{i:06d}\" for i in range(n)],\n", + " \"cohort\": cohort,\n", + " \"snapshot_date\": pd.to_datetime([snapshot_times[c] for c in cohort]),\n", + " \"plan\": plan,\n", + " \"region\": region,\n", + " \"tenure_months\": rng.integers(1, 120, n),\n", + " \"num_lines\": num_lines,\n", + " \"monthly_charge\": (np.array([plan_prices[p] for p in plan]) * (1 + 0.6 * (num_lines - 1)) * (1 + price_change / 100) * rng.lognormal(0, 0.08, n)).round(2),\n", + " \"contract_months_left\": months_left,\n", + " \"price_change_pct_12m\": price_change.round(1),\n", + " \"data_usage_gb_30d\": (rng.gamma(2.0, 9.0, n) * np.exp(-0.3 * dissatisfaction)).round(1),\n", + " \"usage_change_pct_90d\": (rng.normal(3, 10, n) - 14 * dissatisfaction).round(1),\n", + " \"dropped_calls_30d\": rng.poisson(1.5 * region_quality * (1 + dissatisfaction)),\n", + " \"network_tickets_90d\": rng.poisson(0.25 * region_quality + 0.6 * dissatisfaction),\n", + " \"billing_tickets_90d\": rng.poisson(0.15 + 0.03 * price_change),\n", + " \"late_payments_6m\": rng.poisson(4 * stress),\n", + "})\n", + "\n", + "logit = (-3.3 + 1.1 * out_of_contract + 0.6 * (plan == \"sim_only_rolling\") - 0.6 * (plan == \"broadband_tv\")\n", + " + 0.55 * np.minimum(customers.network_tickets_90d, 3) + 0.08 * customers.dropped_calls_30d\n", + " + 0.05 * customers.price_change_pct_12m * out_of_contract + 0.35 * customers.billing_tickets_90d\n", + " + 0.3 * customers.late_payments_6m - 0.03 * customers.usage_change_pct_90d\n", + " - 0.3 * (customers.num_lines - 1) - 0.25 * np.log1p(customers.tenure_months) + rng.normal(0, 0.5, n))\n", + "churned = rng.binomial(1, 1 / (1 + np.exp(-logit)))\n", + "customers[\"churned_30d\"] = np.where(cohort == \"score\", np.nan, churned)\n", + "\n", + "customers.to_csv(run_dir / \"customer_snapshots.csv\", index=False)\n", + "print(customers.groupby(\"cohort\", sort=False).churned_30d.agg(rows=\"size\", churn_rate=\"mean\"))\n", + "display(customers.head(3))" + ] + }, + { + "cell_type": "markdown", + "id": "cell-09", + "metadata": {}, + "source": [ + "## Prepare context, held-out and scoring rows\n", + "\n", + "KumoTabular is pretrained, so \"fitting\" only supplies labeled examples as in-context data. Use the March snapshot as context, hold out the April snapshot to measure ranking quality, and score the June snapshot without labels. Only the feature columns reach the model; IDs, dates and cohort markers are excluded." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "cell-10", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:43.927668Z", + "iopub.status.busy": "2026-11-17T23:27:43.927402Z", + "iopub.status.idle": "2026-11-17T23:27:43.940966Z", + "shell.execute_reply": "2026-11-17T23:27:43.940125Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Context: 3000 | Held-out: 1500 | Scoring: 1000\n" + ] + } + ], + "source": [ + "feature_columns = [\n", + " \"plan\", \"region\", \"tenure_months\", \"num_lines\", \"monthly_charge\", \"contract_months_left\",\n", + " \"price_change_pct_12m\", \"data_usage_gb_30d\", \"usage_change_pct_90d\", \"dropped_calls_30d\",\n", + " \"network_tickets_90d\", \"billing_tickets_90d\", \"late_payments_6m\",\n", + "]\n", + "target = \"churned_30d\"\n", + "\n", + "context = customers[customers.cohort == \"context\"]\n", + "heldout = customers[customers.cohort == \"heldout\"]\n", + "scoring = customers[customers.cohort == \"score\"].reset_index(drop=True)\n", + "\n", + "context_df = context[feature_columns + [target]].astype({target: int}).reset_index(drop=True)\n", + "y_heldout = heldout[target].astype(int).to_numpy()\n", + "print(f\"Context: {len(context_df)} | Held-out: {len(heldout)} | Scoring: {len(scoring)}\")" + ] + }, + { + "cell_type": "markdown", + "id": "cell-11", + "metadata": {}, + "source": [ + "Convert the frames to SDM tensors. Infer column types from the context, and mark the 0/1 churn label as categorical so the model treats this as classification." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "cell-12", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:43.942880Z", + "iopub.status.busy": "2026-11-17T23:27:43.942641Z", + "iopub.status.idle": "2026-11-17T23:27:44.864338Z", + "shell.execute_reply": "2026-11-17T23:27:44.863279Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "{'plan': Stype.categorical,\n", + " 'region': Stype.categorical,\n", + " 'tenure_months': Stype.numerical,\n", + " 'num_lines': Stype.numerical,\n", + " 'monthly_charge': Stype.numerical,\n", + " 'contract_months_left': Stype.numerical,\n", + " 'price_change_pct_12m': Stype.numerical,\n", + " 'data_usage_gb_30d': Stype.numerical,\n", + " 'usage_change_pct_90d': Stype.numerical,\n", + " 'dropped_calls_30d': Stype.numerical,\n", + " 'network_tickets_90d': Stype.numerical,\n", + " 'billing_tickets_90d': Stype.numerical,\n", + " 'late_payments_6m': Stype.numerical}" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "context_table = sdm.TableTensor.from_pandas(\n", + " context_df, stypes=sdm.infer_stypes(context_df, overrides={target: \"categorical\"}), device=DEVICE\n", + ")\n", + "x_context = context_table.drop_columns(target)\n", + "y_context = context_table[:, target]\n", + "\n", + "feature_stypes = sdm.infer_stypes(context_df[feature_columns])\n", + "x_heldout = sdm.TableTensor.from_pandas(heldout[feature_columns], stypes=feature_stypes, device=DEVICE)\n", + "x_score = sdm.TableTensor.from_pandas(scoring[feature_columns], stypes=feature_stypes, device=DEVICE)\n", + "feature_stypes" + ] + }, + { + "cell_type": "markdown", + "id": "cell-13", + "metadata": {}, + "source": [ + "## Load the pretrained classifier and fit the context\n", + "\n", + "Use the small checkpoint so the example runs quickly on CPU. The weights use the [OpenMDW license](https://huggingface.co/nvidia/Kumo-Tabular). `fit` stores the context rows; it does not update model weights." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "cell-14", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:44.866689Z", + "iopub.status.busy": "2026-11-17T23:27:44.866216Z", + "iopub.status.idle": "2026-11-17T23:27:46.524770Z", + "shell.execute_reply": "2026-11-17T23:27:46.523720Z" + } + }, + "outputs": [], + "source": [ + "model = KumoTabular(task=\"classification\", size=MODEL_SIZE, device=DEVICE)\n", + "model.fit(x=x_context, y=y_context, num_estimators=NUM_ESTIMATORS,\n", + " generator=torch.Generator(device=DEVICE).manual_seed(SEED))" + ] + }, + { + "cell_type": "markdown", + "id": "cell-15", + "metadata": {}, + "source": [ + "## Evaluate on held-out customers\n", + "\n", + "Predictions return one probability column per class; read the churn class by its name, `\"1\"`.\n", + "AUROC measures how well the model ranks churners above non-churners. For a retention campaign, the more direct measure is how many churners fall in the top `TOP_SHARE` of scores compared with contacting customers at random. **Synthetic-data metrics: demonstration only.**" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "cell-16", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:46.527286Z", + "iopub.status.busy": "2026-11-17T23:27:46.527049Z", + "iopub.status.idle": "2026-11-17T23:27:46.658019Z", + "shell.execute_reply": "2026-11-17T23:27:46.656912Z" + } + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
held-out
AUROC0.798
Average precision0.456
Overall churn rate0.147
Churn rate in top 10%0.487
Share of churners in top 10%0.332
\n", + "
" + ], + "text/plain": [ + " held-out\n", + "AUROC 0.798\n", + "Average precision 0.456\n", + "Overall churn rate 0.147\n", + "Churn rate in top 10% 0.487\n", + "Share of churners in top 10% 0.332" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "def churn_probability(prediction):\n", + " class_names = list(prediction.columns[sdm.Stype.numerical])\n", + " return prediction.numerical.detach().cpu().numpy()[:, class_names.index(\"1\")]\n", + "\n", + "\n", + "p_heldout = churn_probability(model.predict(x_heldout))\n", + "\n", + "top = p_heldout >= np.quantile(p_heldout, 1 - TOP_SHARE)\n", + "metrics = pd.Series({\n", + " \"AUROC\": roc_auc_score(y_heldout, p_heldout),\n", + " \"Average precision\": average_precision_score(y_heldout, p_heldout),\n", + " \"Overall churn rate\": y_heldout.mean(),\n", + " f\"Churn rate in top {TOP_SHARE:.0%}\": y_heldout[top].mean(),\n", + " f\"Share of churners in top {TOP_SHARE:.0%}\": y_heldout[top].sum() / y_heldout.sum(),\n", + "}, name=\"held-out\").round(3)\n", + "display(metrics.to_frame())" + ] + }, + { + "cell_type": "markdown", + "id": "cell-17", + "metadata": {}, + "source": [ + "## Batch predictions\n", + "\n", + "Score the June snapshot with the same context. Each scoring row receives one churn probability." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "cell-18", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:46.660178Z", + "iopub.status.busy": "2026-11-17T23:27:46.659908Z", + "iopub.status.idle": "2026-11-17T23:27:46.713269Z", + "shell.execute_reply": "2026-11-17T23:27:46.712212Z" + } + }, + "outputs": [], + "source": [ + "p_score = churn_probability(model.predict(x_score))" + ] + }, + { + "cell_type": "markdown", + "id": "cell-19", + "metadata": {}, + "source": [ + "### Inspect and save the scores\n", + "\n", + "Attach the probabilities to customer IDs in input order and flag the top `TOP_SHARE` as a retention priority. The flag is an illustrative campaign cut-off, not a calibrated decision threshold." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "cell-20", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:46.715620Z", + "iopub.status.busy": "2026-11-17T23:27:46.715377Z", + "iopub.status.idle": "2026-11-17T23:27:46.735131Z", + "shell.execute_reply": "2026-11-17T23:27:46.734231Z" + } + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
customer_idsnapshot_datechurn_probabilityretention_priority
510cust_0050102025-06-010.866048True
51cust_0045512025-06-010.810825True
36cust_0045362025-06-010.803690True
489cust_0049892025-06-010.801787True
493cust_0049932025-06-010.774026True
\n", + "
" + ], + "text/plain": [ + " customer_id snapshot_date churn_probability retention_priority\n", + "510 cust_005010 2025-06-01 0.866048 True\n", + "51 cust_004551 2025-06-01 0.810825 True\n", + "36 cust_004536 2025-06-01 0.803690 True\n", + "489 cust_004989 2025-06-01 0.801787 True\n", + "493 cust_004993 2025-06-01 0.774026 True" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Saved 1000 predictions to /tmp/telco-churn-tabular-w63p8wsq/predictions.parquet\n" + ] + } + ], + "source": [ + "predictions = pd.DataFrame({\n", + " \"customer_id\": scoring.customer_id,\n", + " \"snapshot_date\": scoring.snapshot_date,\n", + " \"churn_probability\": p_score,\n", + " \"retention_priority\": p_score >= np.quantile(p_score, 1 - TOP_SHARE),\n", + "})\n", + "assert len(predictions) == len(scoring)\n", + "predictions.to_parquet(run_dir / \"predictions.parquet\", index=False)\n", + "display(predictions.sort_values(\"churn_probability\", ascending=False).head(5))\n", + "print(f\"Saved {len(predictions)} predictions to\", run_dir / \"predictions.parquet\")" + ] + }, + { + "cell_type": "markdown", + "id": "cell-21", + "metadata": {}, + "source": [ + "## Explain the highest-risk customers\n", + "\n", + "[GradientExplainer](https://nvidia.github.io/structured-data-models/api/generated/sdm.explain.GradientExplainer.html) measures how sensitive each churn probability is to the model's preprocessed inputs. Use it to see which signals drive the riskiest customers' scores; it shows local sensitivity, not causal effect." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "cell-22", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:46.737056Z", + "iopub.status.busy": "2026-11-17T23:27:46.736833Z", + "iopub.status.idle": "2026-11-17T23:27:46.907152Z", + "shell.execute_reply": "2026-11-17T23:27:46.906206Z" + } + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
mean |gradient|
late_payments_6m1.454464
billing_tickets_90d1.099764
network_tickets_90d0.401423
num_lines0.299152
tenure_months0.271113
dropped_calls_30d0.076381
data_usage_gb_30d0.047753
usage_change_pct_90d0.047037
\n", + "
" + ], + "text/plain": [ + " mean |gradient|\n", + "late_payments_6m 1.454464\n", + "billing_tickets_90d 1.099764\n", + "network_tickets_90d 0.401423\n", + "num_lines 0.299152\n", + "tenure_months 0.271113\n", + "dropped_calls_30d 0.076381\n", + "data_usage_gb_30d 0.047753\n", + "usage_change_pct_90d 0.047037" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "riskiest = np.argsort(-p_score)[:5]\n", + "attributions, _ = GradientExplainer().explain(model, x_score[riskiest])\n", + "churn_class = list(model.predict(x_score[riskiest]).columns[sdm.Stype.numerical]).index(\"1\")\n", + "gradients = attributions.numerical.detach().cpu().numpy()[churn_class]\n", + "drivers = pd.DataFrame(np.abs(gradients), columns=list(attributions.columns[sdm.Stype.numerical]),\n", + " index=scoring.customer_id.iloc[riskiest])\n", + "display(drivers.mean().sort_values(ascending=False).head(8).rename(\"mean |gradient|\").to_frame())" + ] + }, + { + "cell_type": "markdown", + "id": "cell-23", + "metadata": {}, + "source": [ + "## Retain and reuse\n", + "\n", + "KumoTabular has no trained weights to save; the reusable artifact is the context. Save the labeled context, then rebuild a fresh model from the file and confirm it reproduces the June scores." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "cell-24", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:46.909288Z", + "iopub.status.busy": "2026-11-17T23:27:46.909064Z", + "iopub.status.idle": "2026-11-17T23:27:47.322607Z", + "shell.execute_reply": "2026-11-17T23:27:47.321626Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Rebuilt model reproduces the saved predictions.\n" + ] + } + ], + "source": [ + "context_df.to_parquet(run_dir / \"context.parquet\", index=False)\n", + "\n", + "restored_context = pd.read_parquet(run_dir / \"context.parquet\")\n", + "restored_table = sdm.TableTensor.from_pandas(\n", + " restored_context, stypes=sdm.infer_stypes(restored_context, overrides={target: \"categorical\"}), device=DEVICE\n", + ")\n", + "restored_model = KumoTabular(task=\"classification\", size=MODEL_SIZE, device=DEVICE)\n", + "restored_model.fit(x=restored_table.drop_columns(target), y=restored_table[:, target],\n", + " num_estimators=NUM_ESTIMATORS, generator=torch.Generator(device=DEVICE).manual_seed(SEED))\n", + "assert np.allclose(churn_probability(restored_model.predict(x_score)), p_score, atol=1e-5)\n", + "print(\"Rebuilt model reproduces the saved predictions.\")" + ] + }, + { + "cell_type": "markdown", + "id": "cell-25", + "metadata": {}, + "source": [ + "## Cleanup\n", + "\n", + "Release the context held by both models. Generated data, context and predictions remain in `run_dir`." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "cell-26", + "metadata": { + "execution": { + "iopub.execute_input": "2026-11-17T23:27:47.325076Z", + "iopub.status.busy": "2026-11-17T23:27:47.324834Z", + "iopub.status.idle": "2026-11-17T23:27:47.329483Z", + "shell.execute_reply": "2026-11-17T23:27:47.328631Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Outputs retained in /tmp/telco-churn-tabular-w63p8wsq\n" + ] + } + ], + "source": [ + "model.clear()\n", + "restored_model.clear()\n", + "print(\"Outputs retained in\", run_dir)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.16" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From 7240a742d5555c81d3292bde3565891d6769f24e Mon Sep 17 00:00:00 2001 From: aliu65803 Date: Wed, 18 Nov 2026 10:40:25 -0600 Subject: [PATCH 2/4] Require a GPU for the telco churn notebooks Co-Authored-By: Claude Opus 5.5 --- .../local_kumo_engine_synthetic_telco_churn.ipynb | 2 +- .../local_structured_data_models_telco_churn.ipynb | 10 ++++++---- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb b/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb index 397266a6d..33d4f9192 100644 --- a/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb +++ b/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb @@ -9,7 +9,7 @@ "\n", "Which mobile and broadband customers will cancel in the next 30 days? Train a Kumo Engine graph model directly on synthetic relational telco tables (customers, plans, subscriptions, weekly usage, bills, support tickets and cancellations), with no manual feature engineering, then rescore the active base when a new month of data lands.\n", "\n", - "Use a Linux notebook kernel with a supported NVIDIA GPU and local SSD scratch. Complete [Linux deployment setup](https://docs.nvidia.com/sdgm/kumo-engine/get-started-with-kumo-engine/installation/linux-machine-deployment) and check [GPU sizing](https://docs.nvidia.com/sdgm/kumo-engine/get-started-with-kumo-engine/gpu-sizing). This example follows the [binary classification workflow](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/code-examples/linux-deployment/binary-classification). All data is synthetic, so metrics are **demonstration only**." + "Use a Linux notebook kernel with a supported GPU and local SSD scratch. Complete [Linux deployment setup](https://docs.nvidia.com/sdgm/kumo-engine/get-started-with-kumo-engine/installation/linux-machine-deployment) and check [GPU sizing](https://docs.nvidia.com/sdgm/kumo-engine/get-started-with-kumo-engine/gpu-sizing). This example follows the [binary classification workflow](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/code-examples/linux-deployment/binary-classification). All data is synthetic, so metrics are **demonstration only**." ] }, { diff --git a/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb b/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb index 1777317da..d68f2a5fc 100644 --- a/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb +++ b/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb @@ -9,7 +9,7 @@ "\n", "Which mobile and broadband customers are likely to cancel in the next 30 days? Score a synthetic table of customer features with pretrained KumoTabular, without training a model, and save a ranked list for a retention team.\n", "\n", - "Runs locally on CPU or an NVIDIA GPU with Python 3.11 or newer. The first run downloads public [Kumo-Tabular weights](https://huggingface.co/nvidia/Kumo-Tabular). See the [KumoTabular API](https://nvidia.github.io/structured-data-models/api/generated/sdm.models.KumoTabular.html). All data is synthetic, so metrics are **demonstration only**." + "Requires a GPU and Python 3.11 or newer. The first run downloads public [Kumo-Tabular weights](https://huggingface.co/nvidia/Kumo-Tabular). See the [KumoTabular API](https://nvidia.github.io/structured-data-models/api/generated/sdm.models.KumoTabular.html). All data is synthetic, so metrics are **demonstration only**." ] }, { @@ -83,7 +83,7 @@ "## Set the example parameters\n", "\n", "Each cohort is a monthly customer snapshot. The context and held-out snapshots are far enough apart that their 30-day churn windows do not overlap, and the scoring snapshot has no known outcomes.\n", - "`TOP_SHARE` is the fraction of customers a retention team can contact. `MODEL_SIZE` can be raised to `\"medium\"` or `\"large\"` on a GPU; keep `NUM_ESTIMATORS = 1` for the explanation step." + "`TOP_SHARE` is the fraction of customers a retention team can contact. `MODEL_SIZE` can be raised to `\"medium\"` or `\"large\"`; keep `NUM_ESTIMATORS = 1` for the explanation step." ] }, { @@ -115,7 +115,9 @@ "TOP_SHARE = 0.10\n", "\n", "MODEL_SIZE = \"small\"\n", - "DEVICE = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", + "if not torch.cuda.is_available():\n", + " raise RuntimeError(\"This notebook requires a GPU.\")\n", + "DEVICE = \"cuda\"\n", "NUM_ESTIMATORS = 1\n", "\n", "run_dir = Path(tempfile.mkdtemp(prefix=\"telco-churn-tabular-\"))\n", @@ -450,7 +452,7 @@ "source": [ "## Load the pretrained classifier and fit the context\n", "\n", - "Use the small checkpoint so the example runs quickly on CPU. The weights use the [OpenMDW license](https://huggingface.co/nvidia/Kumo-Tabular). `fit` stores the context rows; it does not update model weights." + "Use the small checkpoint so the example runs quickly. The weights use the [OpenMDW license](https://huggingface.co/nvidia/Kumo-Tabular). `fit` stores the context rows; it does not update model weights." ] }, { From 182781efda6396b5e9d6f13a21c059f9ba206ad4 Mon Sep 17 00:00:00 2001 From: aliu65803 Date: Wed, 18 Nov 2026 11:09:01 -0600 Subject: [PATCH 3/4] Explain when to use each telco churn model and trim notebooks Co-Authored-By: Claude Opus 5.5 --- ...al_kumo_engine_synthetic_telco_churn.ipynb | 412 +++++------------- ...l_structured_data_models_telco_churn.ipynb | 190 ++------ 2 files changed, 160 insertions(+), 442 deletions(-) diff --git a/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb b/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb index 33d4f9192..e782616ae 100644 --- a/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb +++ b/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb @@ -7,7 +7,11 @@ "source": [ "# Predict telco customer churn with Kumo Engine\n", "\n", - "Which mobile and broadband customers will cancel in the next 30 days? Train a Kumo Engine graph model directly on synthetic relational telco tables (customers, plans, subscriptions, weekly usage, bills, support tickets and cancellations), with no manual feature engineering, then rescore the active base when a new month of data lands.\n", + "Which mobile and broadband customers will cancel in the next 30 days?\n", + "\n", + "Churn signals are spread across many tables: contracts, weekly usage, bills and support tickets. Kumo Engine trains a graph model directly on those related tables, so you do not hand-build features such as \"dropped calls in the last 30 days\" or \"months until contract end\". It reads each customer's history up to every prediction time, so training never sees the future. Use it for a churn model trained on your own data and rescored as new data lands. For a pretrained baseline on a single feature table, see the [KumoTabular churn notebook](../../../structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb).\n", + "\n", + "The workflow: describe the tables as a graph, state the prediction as a predictive query, train, then score the active base after a new month of data arrives.\n", "\n", "Use a Linux notebook kernel with a supported GPU and local SSD scratch. Complete [Linux deployment setup](https://docs.nvidia.com/sdgm/kumo-engine/get-started-with-kumo-engine/installation/linux-machine-deployment) and check [GPU sizing](https://docs.nvidia.com/sdgm/kumo-engine/get-started-with-kumo-engine/gpu-sizing). This example follows the [binary classification workflow](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/code-examples/linux-deployment/binary-classification). All data is synthetic, so metrics are **demonstration only**." ] @@ -26,14 +30,7 @@ "cell_type": "code", "execution_count": 1, "id": "cell-02", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:26.418626Z", - "iopub.status.busy": "2026-11-17T23:34:26.418403Z", - "iopub.status.idle": "2026-11-17T23:34:27.412968Z", - "shell.execute_reply": "2026-11-17T23:34:27.411831Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "%pip install -q kumo-engine pandas numpy pyarrow" @@ -51,17 +48,9 @@ "cell_type": "code", "execution_count": 2, "id": "cell-04", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:27.415541Z", - "iopub.status.busy": "2026-11-17T23:34:27.415279Z", - "iopub.status.idle": "2026-11-17T23:34:34.463367Z", - "shell.execute_reply": "2026-11-17T23:34:34.462294Z" - } - }, + "metadata": {}, "outputs": [], "source": [ - "import json\n", "import os\n", "import shutil\n", "from pathlib import Path\n", @@ -86,28 +75,20 @@ "source": [ "## Set the example parameters\n", "\n", - "Set `KUMO_SCRATCH` to a writable local SSD directory. The model trains on data before `TRAIN_END`; the month from `TRAIN_END` to `AS_OF` arrives later and is used only for batch scoring.\n", - "Leave `SMOKE_MODE=False` for the FAST tutorial plan; smoke mode trains for one epoch to check the pipeline only." + "Set `KUMO_SCRATCH` to a writable local SSD directory. The model trains on data before `TRAIN_END`; the month from `TRAIN_END` to `AS_OF` arrives later and is used only for batch scoring." ] }, { "cell_type": "code", "execution_count": 3, "id": "cell-06", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:34.466378Z", - "iopub.status.busy": "2026-11-17T23:34:34.465946Z", - "iopub.status.idle": "2026-11-17T23:34:34.472963Z", - "shell.execute_reply": "2026-11-17T23:34:34.472034Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Run: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-9x83dxwe\n" + "Run: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-934wo6cn\n" ] } ], @@ -119,7 +100,6 @@ "SIM_START = pd.Timestamp(\"2024-01-01\")\n", "TRAIN_END = pd.Timestamp(\"2025-06-01\")\n", "AS_OF = pd.Timestamp(\"2025-07-01\")\n", - "SMOKE_MODE = False\n", "DECISION_THRESHOLD = 0.5\n", "\n", "OUTPUT_PARENT.mkdir(parents=True, exist_ok=True)\n", @@ -148,14 +128,7 @@ "cell_type": "code", "execution_count": 4, "id": "cell-08", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:34.475387Z", - "iopub.status.busy": "2026-11-17T23:34:34.475174Z", - "iopub.status.idle": "2026-11-17T23:34:34.746869Z", - "shell.execute_reply": "2026-11-17T23:34:34.745854Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "rng = np.random.default_rng(RANDOM_SEED)\n", @@ -255,14 +228,7 @@ "cell_type": "code", "execution_count": 5, "id": "cell-09", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:34.749104Z", - "iopub.status.busy": "2026-11-17T23:34:34.748872Z", - "iopub.status.idle": "2026-11-17T23:34:35.202403Z", - "shell.execute_reply": "2026-11-17T23:34:35.201481Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -377,14 +343,7 @@ "cell_type": "code", "execution_count": 6, "id": "cell-11", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:35.204702Z", - "iopub.status.busy": "2026-11-17T23:34:35.204444Z", - "iopub.status.idle": "2026-11-17T23:34:35.577474Z", - "shell.execute_reply": "2026-11-17T23:34:35.576326Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "files = kumo.FileConnector(root_dir=str(source_dir))\n", @@ -432,14 +391,7 @@ "cell_type": "code", "execution_count": 7, "id": "cell-13", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:35.579877Z", - "iopub.status.busy": "2026-11-17T23:34:35.579646Z", - "iopub.status.idle": "2026-11-17T23:34:35.586747Z", - "shell.execute_reply": "2026-11-17T23:34:35.585921Z" - } - }, + "metadata": {}, "outputs": [ { "data": { @@ -541,20 +493,13 @@ "cell_type": "code", "execution_count": 8, "id": "cell-15", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:35.588776Z", - "iopub.status.busy": "2026-11-17T23:34:35.588556Z", - "iopub.status.idle": "2026-11-17T23:34:37.339016Z", - "shell.execute_reply": "2026-11-17T23:34:37.337982Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Training snapshot: 6054b25aa3392517642866d7b6bd7d38defcd6da755f7ce8b2896f0f28097e29@f91dac4d3aefe6e0\n" + "Training snapshot: c977bde5a24d94ca002fb95e37c8d85ad68278328a2115feedf8963f94d7468c@c2f04935bc3dd407\n" ] } ], @@ -577,14 +522,7 @@ "cell_type": "code", "execution_count": 9, "id": "cell-17", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:37.341270Z", - "iopub.status.busy": "2026-11-17T23:34:37.341034Z", - "iopub.status.idle": "2026-11-17T23:34:37.368381Z", - "shell.execute_reply": "2026-11-17T23:34:37.367493Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -616,14 +554,7 @@ "cell_type": "code", "execution_count": 10, "id": "cell-19", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:37.370623Z", - "iopub.status.busy": "2026-11-17T23:34:37.370381Z", - "iopub.status.idle": "2026-11-17T23:34:37.693035Z", - "shell.execute_reply": "2026-11-17T23:34:37.692068Z" - } - }, + "metadata": {}, "outputs": [ { "data": { @@ -723,14 +654,7 @@ "cell_type": "code", "execution_count": 11, "id": "cell-21", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:37.695524Z", - "iopub.status.busy": "2026-11-17T23:34:37.695252Z", - "iopub.status.idle": "2026-11-17T23:34:37.706183Z", - "shell.execute_reply": "2026-11-17T23:34:37.705215Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -747,10 +671,6 @@ "plan.training_job.disable_explain = False\n", "plan.training_job.metrics = [\"auprc\", \"auroc\"]\n", "plan.training_job.tune_metric = \"auprc\"\n", - "if SMOKE_MODE:\n", - " plan.training_job.num_experiments = 1\n", - " plan.optimization.max_epochs = 1\n", - "(output_dir / \"model_plan.json\").write_text(json.dumps(plan.as_json(), indent=2) + \"\\n\")\n", "print(\"Tuning metric:\", plan.training_job.tune_metric, \"| Max epochs:\", plan.optimization.max_epochs)" ] }, @@ -768,14 +688,7 @@ "cell_type": "code", "execution_count": 12, "id": "cell-23", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:34:37.708177Z", - "iopub.status.busy": "2026-11-17T23:34:37.707954Z", - "iopub.status.idle": "2026-11-17T23:37:00.621180Z", - "shell.execute_reply": "2026-11-17T23:37:00.620235Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "trainer = kumo.Trainer(plan)\n", @@ -790,21 +703,14 @@ "source": [ "## Evaluate\n", "\n", - "**Synthetic-data metrics: demonstration only.** AUROC measures how well the model ranks customers who cancel within 30 days above those who stay; AUPRC is more informative when churners are rare, and should be compared with the split's churn rate shown above." + "**Synthetic-data metrics: demonstration only.** AUROC measures how well the model ranks customers who cancel within 30 days above those who stay. AUPRC is more informative when churners are rare: a random ranking scores about the test split's churn rate shown above (5.4%), so a higher AUPRC means the model surfaces churners better than random." ] }, { "cell_type": "code", "execution_count": 13, "id": "cell-25", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:37:00.624093Z", - "iopub.status.busy": "2026-11-17T23:37:00.623853Z", - "iopub.status.idle": "2026-11-17T23:37:00.632875Z", - "shell.execute_reply": "2026-11-17T23:37:00.632100Z" - } - }, + "metadata": {}, "outputs": [ { "data": { @@ -836,57 +742,32 @@ " \n", " \n", " \n", - " val_loss\n", - " 0.203904\n", - " \n", - " \n", - " val_auprc\n", - " 0.126241\n", - " \n", - " \n", " val_auroc\n", - " 0.740524\n", + " 0.737\n", " \n", " \n", - " test_loss\n", - " 0.227209\n", - " \n", - " \n", - " test_auprc\n", - " 0.143237\n", + " val_auprc\n", + " 0.131\n", " \n", " \n", " test_auroc\n", - " 0.747191\n", - " \n", - " \n", - " train_loss\n", - " 0.077621\n", - " \n", - " \n", - " train_auprc\n", - " 0.308035\n", + " 0.791\n", " \n", " \n", - " train_auroc\n", - " 0.891191\n", + " test_auprc\n", + " 0.197\n", " \n", " \n", "\n", "" ], "text/plain": [ - " value\n", - "metric \n", - "val_loss 0.203904\n", - "val_auprc 0.126241\n", - "val_auroc 0.740524\n", - "test_loss 0.227209\n", - "test_auprc 0.143237\n", - "test_auroc 0.747191\n", - "train_loss 0.077621\n", - "train_auprc 0.308035\n", - "train_auroc 0.891191" + " value\n", + "metric \n", + "val_auroc 0.737\n", + "val_auprc 0.131\n", + "test_auroc 0.791\n", + "test_auprc 0.197" ] }, "metadata": {}, @@ -894,8 +775,8 @@ } ], "source": [ - "(output_dir / \"metrics.json\").write_text(json.dumps(fit_result.metrics, indent=2, default=float) + \"\\n\")\n", - "display(pd.Series(fit_result.metrics, name=\"value\").rename_axis(\"metric\").to_frame())" + "metrics = pd.Series(fit_result.metrics)[[\"val_auroc\", \"val_auprc\", \"test_auroc\", \"test_auprc\"]]\n", + "display(metrics.round(3).rename_axis(\"metric\").to_frame(\"value\"))" ] }, { @@ -905,21 +786,14 @@ "source": [ "## Inspect training explanations\n", "\n", - "[Training explanations](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/model-explainability) rank the table columns the selected model is most sensitive to on test examples. Sensitivity is not a causal effect." + "[Training explanations](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/model-explainability) rank the table columns the model is most sensitive to on test examples. Here contract end dates and tenure rank highest: contract expiry is the moment to target with retention offers. Sensitivity is not a causal effect." ] }, { "cell_type": "code", "execution_count": 14, "id": "cell-27", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:37:00.635071Z", - "iopub.status.busy": "2026-11-17T23:37:00.634866Z", - "iopub.status.idle": "2026-11-17T23:37:00.663694Z", - "shell.execute_reply": "2026-11-17T23:37:00.662788Z" - } - }, + "metadata": {}, "outputs": [ { "data": { @@ -952,61 +826,49 @@ " 0\n", " subscriptions\n", " contract_end_date\n", - " 0.377510\n", + " 0.352105\n", " \n", " \n", " 1\n", " subscriptions\n", " started_at\n", - " 0.320016\n", + " 0.282906\n", " \n", " \n", " 2\n", " customers\n", " signup_date\n", - " 0.142261\n", + " 0.202447\n", " \n", " \n", " 3\n", " support_tickets\n", " opened_at\n", - " 0.035750\n", + " 0.088562\n", " \n", " \n", " 4\n", " customers\n", " region\n", - " 0.022616\n", + " 0.014972\n", " \n", " \n", " 5\n", " customers\n", - " age_band\n", - " 0.021190\n", + " acquisition_channel\n", + " 0.014872\n", " \n", " \n", " 6\n", " customers\n", " segment\n", - " 0.015604\n", + " 0.011432\n", " \n", " \n", " 7\n", " customers\n", - " acquisition_channel\n", - " 0.014025\n", - " \n", - " \n", - " 8\n", - " bills\n", - " billed_at\n", - " 0.012711\n", - " \n", - " \n", - " 9\n", - " support_tickets\n", - " category\n", - " 0.011421\n", + " age_band\n", + " 0.010234\n", " \n", " \n", "\n", @@ -1014,16 +876,14 @@ ], "text/plain": [ " table column score\n", - "0 subscriptions contract_end_date 0.377510\n", - "1 subscriptions started_at 0.320016\n", - "2 customers signup_date 0.142261\n", - "3 support_tickets opened_at 0.035750\n", - "4 customers region 0.022616\n", - "5 customers age_band 0.021190\n", - "6 customers segment 0.015604\n", - "7 customers acquisition_channel 0.014025\n", - "8 bills billed_at 0.012711\n", - "9 support_tickets category 0.011421" + "0 subscriptions contract_end_date 0.352105\n", + "1 subscriptions started_at 0.282906\n", + "2 customers signup_date 0.202447\n", + "3 support_tickets opened_at 0.088562\n", + "4 customers region 0.014972\n", + "5 customers acquisition_channel 0.014872\n", + "6 customers segment 0.011432\n", + "7 customers age_band 0.010234" ] }, "metadata": {}, @@ -1031,13 +891,9 @@ } ], "source": [ - "score_files = list(Path(fit_result.bundle_dir).parent.rglob(\"explain_test/scores.csv\"))\n", - "if score_files:\n", - " sensitivity = pd.read_csv(score_files[0]).sort_values(\"score\", ascending=False)\n", - " sensitivity.to_csv(output_dir / \"column_sensitivity.csv\", index=False)\n", - " display(sensitivity[[\"table\", \"column\", \"score\"]].head(10))\n", - "else:\n", - " print(\"No explanation scores found; check the explanation step in the fit logs.\")" + "scores_csv = next(Path(fit_result.bundle_dir).parent.rglob(\"explain_test/scores.csv\"))\n", + "sensitivity = pd.read_csv(scores_csv).sort_values(\"score\", ascending=False)\n", + "display(sensitivity[[\"table\", \"column\", \"score\"]].head(8))" ] }, { @@ -1054,20 +910,13 @@ "cell_type": "code", "execution_count": 15, "id": "cell-29", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:37:00.665688Z", - "iopub.status.busy": "2026-11-17T23:37:00.665458Z", - "iopub.status.idle": "2026-11-17T23:37:02.460636Z", - "shell.execute_reply": "2026-11-17T23:37:02.459577Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Scoring snapshot: 6054b25aa3392517642866d7b6bd7d38defcd6da755f7ce8b2896f0f28097e29@055d0798cd4f514b\n" + "Scoring snapshot: c977bde5a24d94ca002fb95e37c8d85ad68278328a2115feedf8963f94d7468c@3592239bcd4aa59c\n" ] } ], @@ -1093,14 +942,7 @@ "cell_type": "code", "execution_count": 16, "id": "cell-31", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:37:02.463442Z", - "iopub.status.busy": "2026-11-17T23:37:02.463190Z", - "iopub.status.idle": "2026-11-17T23:37:02.731778Z", - "shell.execute_reply": "2026-11-17T23:37:02.730959Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -1134,14 +976,7 @@ "cell_type": "code", "execution_count": 17, "id": "cell-33", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:37:02.734557Z", - "iopub.status.busy": "2026-11-17T23:37:02.734304Z", - "iopub.status.idle": "2026-11-17T23:37:05.123720Z", - "shell.execute_reply": "2026-11-17T23:37:05.122734Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "prediction_result = trainer.predict(\n", @@ -1168,14 +1003,7 @@ "cell_type": "code", "execution_count": 18, "id": "cell-35", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:37:05.126204Z", - "iopub.status.busy": "2026-11-17T23:37:05.125952Z", - "iopub.status.idle": "2026-11-17T23:37:05.137212Z", - "shell.execute_reply": "2026-11-17T23:37:05.136749Z" - } - }, + "metadata": {}, "outputs": [ { "data": { @@ -1205,64 +1033,64 @@ " \n", " \n", " \n", - " 890\n", - " 1911\n", + " 1897\n", + " 2218\n", " 2025-07-01\n", - " 0.500259\n", + " 0.103908\n", " \n", " \n", - " 1315\n", - " 2483\n", + " 890\n", + " 1911\n", " 2025-07-01\n", - " 0.424149\n", + " 0.101261\n", " \n", " \n", - " 255\n", - " 1139\n", + " 885\n", + " 1814\n", " 2025-07-01\n", - " 0.417413\n", + " 0.094809\n", " \n", " \n", - " 1867\n", - " 1594\n", + " 1890\n", + " 2064\n", " 2025-07-01\n", - " 0.392592\n", + " 0.093637\n", " \n", " \n", - " 1890\n", - " 2064\n", + " 1315\n", + " 2483\n", " 2025-07-01\n", - " 0.357828\n", + " 0.090609\n", " \n", " \n", - " 1519\n", - " 2586\n", + " 1298\n", + " 2130\n", " 2025-07-01\n", - " 0.355676\n", + " 0.090488\n", " \n", " \n", - " 885\n", - " 1814\n", + " 1626\n", + " 725\n", " 2025-07-01\n", - " 0.355617\n", + " 0.090171\n", " \n", " \n", - " 1273\n", - " 1610\n", + " 255\n", + " 1139\n", " 2025-07-01\n", - " 0.316929\n", + " 0.090048\n", " \n", " \n", - " 1626\n", - " 725\n", + " 1519\n", + " 2586\n", " 2025-07-01\n", - " 0.311057\n", + " 0.089504\n", " \n", " \n", - " 2444\n", - " 1246\n", + " 648\n", + " 1033\n", " 2025-07-01\n", - " 0.299319\n", + " 0.087281\n", " \n", " \n", "\n", @@ -1270,16 +1098,16 @@ ], "text/plain": [ " ENTITY ANCHOR_TIMESTAMP True_PROB\n", - "890 1911 2025-07-01 0.500259\n", - "1315 2483 2025-07-01 0.424149\n", - "255 1139 2025-07-01 0.417413\n", - "1867 1594 2025-07-01 0.392592\n", - "1890 2064 2025-07-01 0.357828\n", - "1519 2586 2025-07-01 0.355676\n", - "885 1814 2025-07-01 0.355617\n", - "1273 1610 2025-07-01 0.316929\n", - "1626 725 2025-07-01 0.311057\n", - "2444 1246 2025-07-01 0.299319" + "1897 2218 2025-07-01 0.103908\n", + "890 1911 2025-07-01 0.101261\n", + "885 1814 2025-07-01 0.094809\n", + "1890 2064 2025-07-01 0.093637\n", + "1315 2483 2025-07-01 0.090609\n", + "1298 2130 2025-07-01 0.090488\n", + "1626 725 2025-07-01 0.090171\n", + "255 1139 2025-07-01 0.090048\n", + "1519 2586 2025-07-01 0.089504\n", + "648 1033 2025-07-01 0.087281" ] }, "metadata": {}, @@ -1289,7 +1117,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Saved predictions: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-9x83dxwe/output/predictions.parquet\n" + "Saved predictions: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-934wo6cn/output/predictions.parquet\n" ] } ], @@ -1313,20 +1141,13 @@ "cell_type": "code", "execution_count": 19, "id": "cell-37", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:37:05.139459Z", - "iopub.status.busy": "2026-11-17T23:37:05.139253Z", - "iopub.status.idle": "2026-11-17T23:37:05.204680Z", - "shell.execute_reply": "2026-11-17T23:37:05.203850Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Reloaded model from: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-9x83dxwe/output/model\n" + "Reloaded model from: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-934wo6cn/output/model\n" ] } ], @@ -1349,14 +1170,7 @@ "cell_type": "code", "execution_count": 20, "id": "cell-39", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:37:05.206906Z", - "iopub.status.busy": "2026-11-17T23:37:05.206688Z", - "iopub.status.idle": "2026-11-17T23:37:05.209895Z", - "shell.execute_reply": "2026-11-17T23:37:05.209159Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "REMOVE_RUN_SCRATCH = False\n", @@ -1364,6 +1178,16 @@ " shutil.rmtree(scratch_dir)\n", " print(\"Removed run scratch:\", scratch_dir)" ] + }, + { + "cell_type": "markdown", + "id": "bac88a17", + "metadata": {}, + "source": [ + "## Use your own data\n", + "\n", + "Replace the generation cells with your own tables, one folder of Parquet files per table under `source_dir`, and update the `graph` definition to match your columns and keys. The query, training and scoring steps stay the same. Retrain periodically and rescore the active base as each month of data lands." + ] } ], "metadata": { diff --git a/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb b/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb index d68f2a5fc..a096fae57 100644 --- a/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb +++ b/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb @@ -7,9 +7,13 @@ "source": [ "# Predict telco customer churn with KumoTabular\n", "\n", - "Which mobile and broadband customers are likely to cancel in the next 30 days? Score a synthetic table of customer features with pretrained KumoTabular, without training a model, and save a ranked list for a retention team.\n", + "Which mobile and broadband customers are likely to cancel in the next 30 days, and who should a retention team contact first?\n", "\n", - "Requires a GPU and Python 3.11 or newer. The first run downloads public [Kumo-Tabular weights](https://huggingface.co/nvidia/Kumo-Tabular). See the [KumoTabular API](https://nvidia.github.io/structured-data-models/api/generated/sdm.models.KumoTabular.html). All data is synthetic, so metrics are **demonstration only**." + "[KumoTabular](https://nvidia.github.io/structured-data-models/api/generated/sdm.models.KumoTabular.html) is a pretrained tabular foundation model. Instead of building, training and tuning a churn model, you give it labeled past customers as context and it scores new customers directly. Use it when you already have one row of features per customer and want a strong churn ranking quickly, for example as a proof of concept. To learn from raw related tables such as usage, bills and tickets without engineering features, see the [Kumo Engine churn notebook](../../../../kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb).\n", + "\n", + "The workflow: supply labeled context rows, check ranking quality on held-out customers, score a new batch, explain the riskiest scores and save the context for reuse.\n", + "\n", + "Requires a GPU and Python 3.11 or newer. The first run downloads the public [Kumo-Tabular weights](https://huggingface.co/nvidia/Kumo-Tabular). All data is synthetic, so metrics are **demonstration only**." ] }, { @@ -26,14 +30,7 @@ "cell_type": "code", "execution_count": 1, "id": "cell-02", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:39.105078Z", - "iopub.status.busy": "2026-11-17T23:27:39.104881Z", - "iopub.status.idle": "2026-11-17T23:27:40.004880Z", - "shell.execute_reply": "2026-11-17T23:27:40.003782Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "%pip install -q structured-data-models pandas pyarrow scikit-learn" @@ -51,14 +48,7 @@ "cell_type": "code", "execution_count": 2, "id": "cell-04", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:40.007263Z", - "iopub.status.busy": "2026-11-17T23:27:40.007034Z", - "iopub.status.idle": "2026-11-17T23:27:43.769854Z", - "shell.execute_reply": "2026-11-17T23:27:43.768805Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "import tempfile\n", @@ -82,28 +72,20 @@ "source": [ "## Set the example parameters\n", "\n", - "Each cohort is a monthly customer snapshot. The context and held-out snapshots are far enough apart that their 30-day churn windows do not overlap, and the scoring snapshot has no known outcomes.\n", - "`TOP_SHARE` is the fraction of customers a retention team can contact. `MODEL_SIZE` can be raised to `\"medium\"` or `\"large\"`; keep `NUM_ESTIMATORS = 1` for the explanation step." + "Each cohort is a monthly customer snapshot: March is the labeled context, April is held out for evaluation and June is the unlabeled batch to score. `TOP_SHARE` is the fraction of customers a retention team can contact. `MODEL_SIZE` can be raised to `\"medium\"` or `\"large\"`; keep `NUM_ESTIMATORS = 1` for the explanation step." ] }, { "cell_type": "code", "execution_count": 3, "id": "cell-06", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:43.772142Z", - "iopub.status.busy": "2026-11-17T23:27:43.771767Z", - "iopub.status.idle": "2026-11-17T23:27:43.800533Z", - "shell.execute_reply": "2026-11-17T23:27:43.799547Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Device: cuda | Outputs: /tmp/telco-churn-tabular-w63p8wsq\n" + "Outputs: /tmp/telco-churn-tabular-tnz9fz39\n" ] } ], @@ -111,7 +93,6 @@ "SEED = 7\n", "N_CONTEXT, N_HELDOUT, N_SCORE = 3000, 1500, 1000\n", "CONTEXT_TIME, HELDOUT_TIME, SCORE_TIME = \"2025-03-01\", \"2025-04-15\", \"2025-06-01\"\n", - "HORIZON_DAYS = 30\n", "TOP_SHARE = 0.10\n", "\n", "MODEL_SIZE = \"small\"\n", @@ -122,7 +103,7 @@ "\n", "run_dir = Path(tempfile.mkdtemp(prefix=\"telco-churn-tabular-\"))\n", "torch.manual_seed(SEED)\n", - "print(\"Device:\", DEVICE, \"| Outputs:\", run_dir)" + "print(\"Outputs:\", run_dir)" ] }, { @@ -139,14 +120,7 @@ "cell_type": "code", "execution_count": 4, "id": "cell-08", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:43.802707Z", - "iopub.status.busy": "2026-11-17T23:27:43.802443Z", - "iopub.status.idle": "2026-11-17T23:27:43.925552Z", - "shell.execute_reply": "2026-11-17T23:27:43.924654Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -335,7 +309,6 @@ "churned = rng.binomial(1, 1 / (1 + np.exp(-logit)))\n", "customers[\"churned_30d\"] = np.where(cohort == \"score\", np.nan, churned)\n", "\n", - "customers.to_csv(run_dir / \"customer_snapshots.csv\", index=False)\n", "print(customers.groupby(\"cohort\", sort=False).churned_30d.agg(rows=\"size\", churn_rate=\"mean\"))\n", "display(customers.head(3))" ] @@ -347,21 +320,14 @@ "source": [ "## Prepare context, held-out and scoring rows\n", "\n", - "KumoTabular is pretrained, so \"fitting\" only supplies labeled examples as in-context data. Use the March snapshot as context, hold out the April snapshot to measure ranking quality, and score the June snapshot without labels. Only the feature columns reach the model; IDs, dates and cohort markers are excluded." + "There is no training step. The labeled March customers become the model's context, the April customers are held out to measure ranking quality, and the June customers are scored without labels. Only the feature columns reach the model; IDs, dates and cohort markers are excluded." ] }, { "cell_type": "code", "execution_count": 5, "id": "cell-10", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:43.927668Z", - "iopub.status.busy": "2026-11-17T23:27:43.927402Z", - "iopub.status.idle": "2026-11-17T23:27:43.940966Z", - "shell.execute_reply": "2026-11-17T23:27:43.940125Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -400,49 +366,17 @@ "cell_type": "code", "execution_count": 6, "id": "cell-12", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:43.942880Z", - "iopub.status.busy": "2026-11-17T23:27:43.942641Z", - "iopub.status.idle": "2026-11-17T23:27:44.864338Z", - "shell.execute_reply": "2026-11-17T23:27:44.863279Z" - } - }, - "outputs": [ - { - "data": { - "text/plain": [ - "{'plan': Stype.categorical,\n", - " 'region': Stype.categorical,\n", - " 'tenure_months': Stype.numerical,\n", - " 'num_lines': Stype.numerical,\n", - " 'monthly_charge': Stype.numerical,\n", - " 'contract_months_left': Stype.numerical,\n", - " 'price_change_pct_12m': Stype.numerical,\n", - " 'data_usage_gb_30d': Stype.numerical,\n", - " 'usage_change_pct_90d': Stype.numerical,\n", - " 'dropped_calls_30d': Stype.numerical,\n", - " 'network_tickets_90d': Stype.numerical,\n", - " 'billing_tickets_90d': Stype.numerical,\n", - " 'late_payments_6m': Stype.numerical}" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], + "metadata": {}, + "outputs": [], "source": [ "context_table = sdm.TableTensor.from_pandas(\n", " context_df, stypes=sdm.infer_stypes(context_df, overrides={target: \"categorical\"}), device=DEVICE\n", ")\n", "x_context = context_table.drop_columns(target)\n", "y_context = context_table[:, target]\n", - "\n", "feature_stypes = sdm.infer_stypes(context_df[feature_columns])\n", "x_heldout = sdm.TableTensor.from_pandas(heldout[feature_columns], stypes=feature_stypes, device=DEVICE)\n", - "x_score = sdm.TableTensor.from_pandas(scoring[feature_columns], stypes=feature_stypes, device=DEVICE)\n", - "feature_stypes" + "x_score = sdm.TableTensor.from_pandas(scoring[feature_columns], stypes=feature_stypes, device=DEVICE)" ] }, { @@ -450,23 +384,16 @@ "id": "cell-13", "metadata": {}, "source": [ - "## Load the pretrained classifier and fit the context\n", + "## Load the pretrained model and fit the context\n", "\n", - "Use the small checkpoint so the example runs quickly. The weights use the [OpenMDW license](https://huggingface.co/nvidia/Kumo-Tabular). `fit` stores the context rows; it does not update model weights." + "Load the small checkpoint ([OpenMDW license](https://huggingface.co/nvidia/Kumo-Tabular)). `fit` stores the context rows; it does not update model weights, so there is nothing to train or tune." ] }, { "cell_type": "code", "execution_count": 7, "id": "cell-14", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:44.866689Z", - "iopub.status.busy": "2026-11-17T23:27:44.866216Z", - "iopub.status.idle": "2026-11-17T23:27:46.524770Z", - "shell.execute_reply": "2026-11-17T23:27:46.523720Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "model = KumoTabular(task=\"classification\", size=MODEL_SIZE, device=DEVICE)\n", @@ -481,22 +408,14 @@ "source": [ "## Evaluate on held-out customers\n", "\n", - "Predictions return one probability column per class; read the churn class by its name, `\"1\"`.\n", - "AUROC measures how well the model ranks churners above non-churners. For a retention campaign, the more direct measure is how many churners fall in the top `TOP_SHARE` of scores compared with contacting customers at random. **Synthetic-data metrics: demonstration only.**" + "AUROC measures how well the model ranks churners above non-churners. For a retention campaign, the more direct measure is how many churners fall in the top `TOP_SHARE` of scores. Here the top 10% have a churn rate about three times the overall rate, so the same outreach budget reaches about a third of next month's churners. **Synthetic-data metrics: demonstration only.**" ] }, { "cell_type": "code", "execution_count": 8, "id": "cell-16", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:46.527286Z", - "iopub.status.busy": "2026-11-17T23:27:46.527049Z", - "iopub.status.idle": "2026-11-17T23:27:46.658019Z", - "shell.execute_reply": "2026-11-17T23:27:46.656912Z" - } - }, + "metadata": {}, "outputs": [ { "data": { @@ -593,14 +512,7 @@ "cell_type": "code", "execution_count": 9, "id": "cell-18", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:46.660178Z", - "iopub.status.busy": "2026-11-17T23:27:46.659908Z", - "iopub.status.idle": "2026-11-17T23:27:46.713269Z", - "shell.execute_reply": "2026-11-17T23:27:46.712212Z" - } - }, + "metadata": {}, "outputs": [], "source": [ "p_score = churn_probability(model.predict(x_score))" @@ -620,14 +532,7 @@ "cell_type": "code", "execution_count": 10, "id": "cell-20", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:46.715620Z", - "iopub.status.busy": "2026-11-17T23:27:46.715377Z", - "iopub.status.idle": "2026-11-17T23:27:46.735131Z", - "shell.execute_reply": "2026-11-17T23:27:46.734231Z" - } - }, + "metadata": {}, "outputs": [ { "data": { @@ -712,7 +617,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Saved 1000 predictions to /tmp/telco-churn-tabular-w63p8wsq/predictions.parquet\n" + "Saved 1000 predictions to /tmp/telco-churn-tabular-tnz9fz39/predictions.parquet\n" ] } ], @@ -736,21 +641,14 @@ "source": [ "## Explain the highest-risk customers\n", "\n", - "[GradientExplainer](https://nvidia.github.io/structured-data-models/api/generated/sdm.explain.GradientExplainer.html) measures how sensitive each churn probability is to the model's preprocessed inputs. Use it to see which signals drive the riskiest customers' scores; it shows local sensitivity, not causal effect." + "[GradientExplainer](https://nvidia.github.io/structured-data-models/api/generated/sdm.explain.GradientExplainer.html) measures how sensitive each churn probability is to each input. For the riskiest customers here, late payments and billing contacts dominate, which points the retention team to billing-focused offers. Sensitivity is not a causal effect." ] }, { "cell_type": "code", "execution_count": 11, "id": "cell-22", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:46.737056Z", - "iopub.status.busy": "2026-11-17T23:27:46.736833Z", - "iopub.status.idle": "2026-11-17T23:27:46.907152Z", - "shell.execute_reply": "2026-11-17T23:27:46.906206Z" - } - }, + "metadata": {}, "outputs": [ { "data": { @@ -853,14 +751,7 @@ "cell_type": "code", "execution_count": 12, "id": "cell-24", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:46.909288Z", - "iopub.status.busy": "2026-11-17T23:27:46.909064Z", - "iopub.status.idle": "2026-11-17T23:27:47.322607Z", - "shell.execute_reply": "2026-11-17T23:27:47.321626Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -891,27 +782,20 @@ "source": [ "## Cleanup\n", "\n", - "Release the context held by both models. Generated data, context and predictions remain in `run_dir`." + "Release the context held by both models. The saved context and predictions remain in `run_dir`." ] }, { "cell_type": "code", "execution_count": 13, "id": "cell-26", - "metadata": { - "execution": { - "iopub.execute_input": "2026-11-17T23:27:47.325076Z", - "iopub.status.busy": "2026-11-17T23:27:47.324834Z", - "iopub.status.idle": "2026-11-17T23:27:47.329483Z", - "shell.execute_reply": "2026-11-17T23:27:47.328631Z" - } - }, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Outputs retained in /tmp/telco-churn-tabular-w63p8wsq\n" + "Outputs retained in /tmp/telco-churn-tabular-tnz9fz39\n" ] } ], @@ -920,6 +804,16 @@ "restored_model.clear()\n", "print(\"Outputs retained in\", run_dir)" ] + }, + { + "cell_type": "markdown", + "id": "8be0daab", + "metadata": {}, + "source": [ + "## Use your own data\n", + "\n", + "Replace the generation cell with your own customer snapshots: one row per customer, your feature columns and a 0/1 churn label for past snapshots. Update `feature_columns` and `target`; the rest of the notebook runs as written. If your data lives in several related tables, use the [Kumo Engine churn notebook](../../../../kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb)." + ] } ], "metadata": { From 0221481c9e2351bcb790f1fbb048dabbc4a46974 Mon Sep 17 00:00:00 2001 From: aliu65803 Date: Wed, 18 Nov 2026 16:58:30 -0600 Subject: [PATCH 4/4] Address review on telco churn notebooks Seed Engine training, read explanations from the selected trial, flag a top-share retention list instead of relying on the 0.5 threshold, verify the reloaded model's predictions, show every Tabular feature attribution, select exactly the top share without ties, and list the data each notebook needs when using your own data. Co-Authored-By: Claude Opus 5.5 --- ...al_kumo_engine_synthetic_telco_churn.ipynb | 241 +++++++++--------- ...l_structured_data_models_telco_churn.ipynb | 76 +++--- 2 files changed, 166 insertions(+), 151 deletions(-) diff --git a/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb b/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb index e782616ae..133384b92 100644 --- a/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb +++ b/examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb @@ -51,6 +51,7 @@ "metadata": {}, "outputs": [], "source": [ + "import json\n", "import os\n", "import shutil\n", "from pathlib import Path\n", @@ -87,9 +88,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Run: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-934wo6cn\n" - ] + "text": "Run: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-mjw5dbid\n" } ], "source": [ @@ -101,6 +100,7 @@ "TRAIN_END = pd.Timestamp(\"2025-06-01\")\n", "AS_OF = pd.Timestamp(\"2025-07-01\")\n", "DECISION_THRESHOLD = 0.5\n", + "TOP_SHARE = 0.10\n", "\n", "OUTPUT_PARENT.mkdir(parents=True, exist_ok=True)\n", "run_dir = Path(mkdtemp(prefix=\"telco_churn-\", dir=OUTPUT_PARENT))\n", @@ -233,15 +233,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "customers history= 4000 new= 0\n", - "plans history= 6 new= 0\n", - "subscriptions history= 5194 new= 0\n", - "usage history= 208877 new= 10481\n", - "bills history= 53589 new= 2616\n", - "support_tickets history= 15732 new= 780\n", - "cancellations history= 1350 new= 69\n" - ] + "text": "customers history= 4000 new= 0\nplans history= 6 new= 0\nsubscriptions history= 5194 new= 0\nusage history= 208877 new= 10481\nbills history= 53589 new= 2616\nsupport_tickets history= 15732 new= 780\ncancellations history= 1350 new= 69\n" }, { "data": { @@ -498,9 +490,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Training snapshot: c977bde5a24d94ca002fb95e37c8d85ad68278328a2115feedf8963f94d7468c@c2f04935bc3dd407\n" - ] + "text": "Training snapshot: 25fdcc9a29e693e2252d3a296b24656df10a84aa6d06b55acbd911d07e1f0b03@550d58c736d15fb8\n" } ], "source": [ @@ -527,9 +517,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Task: binary_classification\n" - ] + "text": "Task: binary_classification\n" } ], "source": [ @@ -647,7 +635,7 @@ "source": [ "## Configure the model\n", "\n", - "Start from the [FAST plan](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/training). Churn is a minority outcome, so select the model on AUPRC and also report AUROC. Disable refitting to keep the test split unseen, and enable training explanations." + "Start from the [FAST plan](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/training). Churn is a minority outcome, so select the model on AUPRC and also report AUROC. Set the training seed to reduce run-to-run variation (GPU training is not bit-for-bit repeatable, so metrics can still shift slightly), disable refitting to keep the test split unseen, and enable training explanations." ] }, { @@ -659,13 +647,12 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Tuning metric: auprc | Max epochs: 8\n" - ] + "text": "Tuning metric: auprc | Max epochs: 8\n" } ], "source": [ "plan = query.suggest_model_plan(snapshot, run_mode=RunMode.FAST)\n", + "plan.training_job.manual_seed = RANDOM_SEED\n", "plan.training_job.refit_trainval = False\n", "plan.training_job.refit_full = False\n", "plan.training_job.disable_explain = False\n", @@ -743,19 +730,19 @@ " \n", " \n", " val_auroc\n", - " 0.737\n", + " 0.719\n", " \n", " \n", " val_auprc\n", - " 0.131\n", + " 0.118\n", " \n", " \n", " test_auroc\n", - " 0.791\n", + " 0.764\n", " \n", " \n", " test_auprc\n", - " 0.197\n", + " 0.159\n", " \n", " \n", "\n", @@ -764,10 +751,10 @@ "text/plain": [ " value\n", "metric \n", - "val_auroc 0.737\n", - "val_auprc 0.131\n", - "test_auroc 0.791\n", - "test_auprc 0.197" + "val_auroc 0.719\n", + "val_auprc 0.118\n", + "test_auroc 0.764\n", + "test_auprc 0.159" ] }, "metadata": {}, @@ -786,7 +773,7 @@ "source": [ "## Inspect training explanations\n", "\n", - "[Training explanations](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/model-explainability) rank the table columns the model is most sensitive to on test examples. Here contract end dates and tenure rank highest: contract expiry is the moment to target with retention offers. Sensitivity is not a causal effect." + "[Training explanations](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/model-explainability) rank the table columns the selected model is most sensitive to on test examples. Here contract end dates and tenure rank highest: contract expiry is the moment to target with retention offers. Sensitivity is not a causal effect." ] }, { @@ -826,64 +813,64 @@ " 0\n", " subscriptions\n", " contract_end_date\n", - " 0.352105\n", + " 0.284193\n", " \n", " \n", " 1\n", - " subscriptions\n", - " started_at\n", - " 0.282906\n", + " customers\n", + " signup_date\n", + " 0.252554\n", " \n", " \n", " 2\n", - " customers\n", - " signup_date\n", - " 0.202447\n", + " subscriptions\n", + " started_at\n", + " 0.212036\n", " \n", " \n", " 3\n", - " support_tickets\n", - " opened_at\n", - " 0.088562\n", + " customers\n", + " region\n", + " 0.052566\n", " \n", " \n", " 4\n", " customers\n", - " region\n", - " 0.014972\n", + " age_band\n", + " 0.028033\n", " \n", " \n", " 5\n", - " customers\n", - " acquisition_channel\n", - " 0.014872\n", + " plans\n", + " product\n", + " 0.026125\n", " \n", " \n", " 6\n", " customers\n", - " segment\n", - " 0.011432\n", + " acquisition_channel\n", + " 0.025252\n", " \n", " \n", " 7\n", " customers\n", - " age_band\n", - " 0.010234\n", + " segment\n", + " 0.022976\n", " \n", " \n", "\n", "" ], "text/plain": [ - " table column score\n", - "0 subscriptions contract_end_date 0.352105\n", - "1 subscriptions started_at 0.282906\n", - "2 customers signup_date 0.202447\n", - "3 support_tickets opened_at 0.088562\n", - "4 customers region 0.014972\n", - "5 customers acquisition_channel 0.014872\n", - "6 customers segment 0.011432\n", - "7 customers age_band 0.010234" + " table column score\n", + "0 subscriptions contract_end_date 0.284193\n", + "1 customers signup_date 0.252554\n", + "2 subscriptions started_at 0.212036\n", + "3 customers region 0.052566\n", + "4 customers age_band 0.028033\n", + "5 plans product 0.026125\n", + "6 customers acquisition_channel 0.025252\n", + "7 customers segment 0.022976" ] }, "metadata": {}, @@ -891,8 +878,9 @@ } ], "source": [ - "scores_csv = next(Path(fit_result.bundle_dir).parent.rglob(\"explain_test/scores.csv\"))\n", - "sensitivity = pd.read_csv(scores_csv).sort_values(\"score\", ascending=False)\n", + "selection_file = next(Path(fit_result.bundle_dir).parent.rglob(\"best_trainer.json\"))\n", + "selected_trial = selection_file.parent / Path(json.loads(selection_file.read_text())[\"best_trainer\"]).name\n", + "sensitivity = pd.read_csv(selected_trial / \"explain_test\" / \"scores.csv\").sort_values(\"score\", ascending=False)\n", "display(sensitivity[[\"table\", \"column\", \"score\"]].head(8))" ] }, @@ -915,9 +903,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Scoring snapshot: c977bde5a24d94ca002fb95e37c8d85ad68278328a2115feedf8963f94d7468c@3592239bcd4aa59c\n" - ] + "text": "Scoring snapshot: 25fdcc9a29e693e2252d3a296b24656df10a84aa6d06b55acbd911d07e1f0b03@9f8ff73dee08cfdb\n" } ], "source": [ @@ -947,9 +933,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Customers to score: 2581\n" - ] + "text": "Customers to score: 2581\n" } ], "source": [ @@ -996,7 +980,7 @@ "source": [ "### Inspect the results\n", "\n", - "There is one row per scored customer: `ENTITY` is the customer ID and `True_PROB` the probability of cancelling within 30 days. Sort by `True_PROB` to build a retention contact list; `TARGET_PRED` applies the untuned `DECISION_THRESHOLD`." + "There is one row per scored customer: `ENTITY` is the customer ID and `True_PROB` the probability of cancelling within 30 days. Churn is rare, so few customers cross `DECISION_THRESHOLD` and `TARGET_PRED` is not a useful contact list. Rank by `True_PROB` instead: flag the top `TOP_SHARE` as the retention priority and save that list." ] }, { @@ -1029,85 +1013,96 @@ " ENTITY\n", " ANCHOR_TIMESTAMP\n", " True_PROB\n", + " retention_priority\n", " \n", " \n", " \n", " \n", - " 1897\n", - " 2218\n", + " 1065\n", + " 1911\n", " 2025-07-01\n", - " 0.103908\n", + " 0.599451\n", + " True\n", " \n", " \n", - " 890\n", - " 1911\n", + " 694\n", + " 2483\n", " 2025-07-01\n", - " 0.101261\n", + " 0.459822\n", + " True\n", " \n", " \n", - " 885\n", - " 1814\n", + " 1412\n", + " 1139\n", " 2025-07-01\n", - " 0.094809\n", + " 0.443655\n", + " True\n", " \n", " \n", - " 1890\n", - " 2064\n", + " 1484\n", + " 1814\n", " 2025-07-01\n", - " 0.093637\n", + " 0.418969\n", + " True\n", " \n", " \n", - " 1315\n", - " 2483\n", + " 1087\n", + " 2130\n", " 2025-07-01\n", - " 0.090609\n", + " 0.380728\n", + " True\n", " \n", " \n", - " 1298\n", - " 2130\n", + " 1633\n", + " 3179\n", " 2025-07-01\n", - " 0.090488\n", + " 0.336966\n", + " True\n", " \n", " \n", - " 1626\n", - " 725\n", + " 1081\n", + " 2064\n", " 2025-07-01\n", - " 0.090171\n", + " 0.332438\n", + " True\n", " \n", " \n", - " 255\n", - " 1139\n", + " 111\n", + " 1033\n", " 2025-07-01\n", - " 0.090048\n", + " 0.296899\n", + " True\n", " \n", " \n", - " 1519\n", - " 2586\n", + " 508\n", + " 725\n", " 2025-07-01\n", - " 0.089504\n", + " 0.289124\n", + " True\n", " \n", " \n", - " 648\n", - " 1033\n", + " 1419\n", + " 1197\n", " 2025-07-01\n", - " 0.087281\n", + " 0.285499\n", + " True\n", " \n", " \n", "\n", "" ], "text/plain": [ - " ENTITY ANCHOR_TIMESTAMP True_PROB\n", - "1897 2218 2025-07-01 0.103908\n", - "890 1911 2025-07-01 0.101261\n", - "885 1814 2025-07-01 0.094809\n", - "1890 2064 2025-07-01 0.093637\n", - "1315 2483 2025-07-01 0.090609\n", - "1298 2130 2025-07-01 0.090488\n", - "1626 725 2025-07-01 0.090171\n", - "255 1139 2025-07-01 0.090048\n", - "1519 2586 2025-07-01 0.089504\n", - "648 1033 2025-07-01 0.087281" + " ENTITY ANCHOR_TIMESTAMP True_PROB retention_priority\n", + "1065 1911 2025-07-01 0.599451 True\n", + "694 2483 2025-07-01 0.459822 True\n", + "1412 1139 2025-07-01 0.443655 True\n", + "1484 1814 2025-07-01 0.418969 True\n", + "1087 2130 2025-07-01 0.380728 True\n", + "1633 3179 2025-07-01 0.336966 True\n", + "1081 2064 2025-07-01 0.332438 True\n", + "111 1033 2025-07-01 0.296899 True\n", + "508 725 2025-07-01 0.289124 True\n", + "1419 1197 2025-07-01 0.285499 True" ] }, "metadata": {}, @@ -1116,15 +1111,16 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Saved predictions: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-934wo6cn/output/predictions.parquet\n" - ] + "text": "Saved 258 retention contacts to /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-mjw5dbid/output/retention_contacts.parquet\n" } ], "source": [ "assert len(scores) == len(scoring_population) and set(scores.ENTITY) == set(active_ids)\n", - "display(scores.sort_values(\"True_PROB\", ascending=False)[[\"ENTITY\", \"ANCHOR_TIMESTAMP\", \"True_PROB\"]].head(10))\n", - "print(\"Saved predictions:\", output_dir / \"predictions.parquet\")" + "contact_count = round(TOP_SHARE * len(scores))\n", + "scores[\"retention_priority\"] = scores.True_PROB.rank(method=\"first\", ascending=False) <= contact_count\n", + "scores[scores.retention_priority].to_parquet(output_dir / \"retention_contacts.parquet\", index=False)\n", + "display(scores.sort_values(\"True_PROB\", ascending=False)[[\"ENTITY\", \"ANCHOR_TIMESTAMP\", \"True_PROB\", \"retention_priority\"]].head(10))\n", + "print(f\"Saved {contact_count} retention contacts to\", output_dir / \"retention_contacts.parquet\")" ] }, { @@ -1134,7 +1130,7 @@ "source": [ "## Retain and reuse\n", "\n", - "Reload the saved model bundle without retraining. Keep the source tables and graph definition with the bundle: a later run rebuilds the graph and snapshot, then predicts as above. See [publish and retrieve models](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/publish-and-retrieve-models)." + "Reload the saved model bundle without retraining and confirm it reproduces the batch predictions. Keep the source tables and graph definition with the bundle: a later run rebuilds the graph and snapshot, then predicts as above. See [publish and retrieve models](https://docs.nvidia.com/sdgm/kumo-engine/user-code-guide/train-a-model/publish-and-retrieve-models)." ] }, { @@ -1146,14 +1142,19 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Reloaded model from: /home/ubuntu/telco-churn/nb/kumo_example_runs/telco_churn-934wo6cn/output/model\n" - ] + "text": "Reloaded model reproduces the batch predictions.\n" } ], "source": [ "reloaded_trainer = kumo.Trainer.load(output_dir / \"model\")\n", - "print(\"Reloaded model from:\", output_dir / \"model\")" + "reloaded_scores = reloaded_trainer.predict(\n", + " prediction_table, snapshot=scoring_snapshot, binary_classification_threshold=DECISION_THRESHOLD\n", + ").frame\n", + "compared = scores[[\"ENTITY\", \"True_PROB\"]].merge(reloaded_scores[[\"ENTITY\", \"True_PROB\"]], on=\"ENTITY\",\n", + " suffixes=(\"\", \"_reloaded\"))\n", + "assert len(compared) == len(scores)\n", + "assert np.allclose(compared.True_PROB, compared.True_PROB_reloaded, atol=1e-5)\n", + "print(\"Reloaded model reproduces the batch predictions.\")" ] }, { @@ -1186,7 +1187,7 @@ "source": [ "## Use your own data\n", "\n", - "Replace the generation cells with your own tables, one folder of Parquet files per table under `source_dir`, and update the `graph` definition to match your columns and keys. The query, training and scoring steps stay the same. Retrain periodically and rescore the active base as each month of data lands." + "Replace the generation cells with your own tables: one folder of Parquet files per table under `source_dir`, then update the `graph` definition to match your columns and keys. The query and training steps stay the same. `tables` and `new_rows` exist only in this simulation, so for batch predictions add each new period as new Parquet files in the same folders and build `scoring_population` from your own list of active customer IDs and scoring time. Retrain periodically and rescore as each month of data lands." ] } ], diff --git a/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb b/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb index a096fae57..355632c7d 100644 --- a/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb +++ b/examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb @@ -84,9 +84,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Outputs: /tmp/telco-churn-tabular-tnz9fz39\n" - ] + "text": "Outputs: /tmp/telco-churn-tabular-uw7crf8k\n" } ], "source": [ @@ -125,13 +123,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - " rows churn_rate\n", - "cohort \n", - "context 3000 0.148000\n", - "heldout 1500 0.146667\n", - "score 1000 NaN\n" - ] + "text": " rows churn_rate\ncohort \ncontext 3000 0.148000\nheldout 1500 0.146667\nscore 1000 NaN\n" }, { "data": { @@ -332,9 +324,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Context: 3000 | Held-out: 1500 | Scoring: 1000\n" - ] + "text": "Context: 3000 | Held-out: 1500 | Scoring: 1000\n" } ], "source": [ @@ -485,9 +475,14 @@ " return prediction.numerical.detach().cpu().numpy()[:, class_names.index(\"1\")]\n", "\n", "\n", + "def top_share(probabilities):\n", + " contact_count = round(TOP_SHARE * len(probabilities))\n", + " return pd.Series(probabilities).rank(method=\"first\", ascending=False).to_numpy() <= contact_count\n", + "\n", + "\n", "p_heldout = churn_probability(model.predict(x_heldout))\n", "\n", - "top = p_heldout >= np.quantile(p_heldout, 1 - TOP_SHARE)\n", + "top = top_share(p_heldout)\n", "metrics = pd.Series({\n", " \"AUROC\": roc_auc_score(y_heldout, p_heldout),\n", " \"Average precision\": average_precision_score(y_heldout, p_heldout),\n", @@ -525,7 +520,7 @@ "source": [ "### Inspect and save the scores\n", "\n", - "Attach the probabilities to customer IDs in input order and flag the top `TOP_SHARE` as a retention priority. The flag is an illustrative campaign cut-off, not a calibrated decision threshold." + "Attach the probabilities to customer IDs in input order and flag exactly the top `TOP_SHARE` of customers as a retention priority. The flag is an illustrative campaign cut-off, not a calibrated decision threshold." ] }, { @@ -616,9 +611,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Saved 1000 predictions to /tmp/telco-churn-tabular-tnz9fz39/predictions.parquet\n" - ] + "text": "Saved 1000 predictions to /tmp/telco-churn-tabular-uw7crf8k/predictions.parquet\n" } ], "source": [ @@ -626,7 +619,7 @@ " \"customer_id\": scoring.customer_id,\n", " \"snapshot_date\": scoring.snapshot_date,\n", " \"churn_probability\": p_score,\n", - " \"retention_priority\": p_score >= np.quantile(p_score, 1 - TOP_SHARE),\n", + " \"retention_priority\": top_share(p_score),\n", "})\n", "assert len(predictions) == len(scoring)\n", "predictions.to_parquet(run_dir / \"predictions.parquet\", index=False)\n", @@ -641,7 +634,7 @@ "source": [ "## Explain the highest-risk customers\n", "\n", - "[GradientExplainer](https://nvidia.github.io/structured-data-models/api/generated/sdm.explain.GradientExplainer.html) measures how sensitive each churn probability is to each input. For the riskiest customers here, late payments and billing contacts dominate, which points the retention team to billing-focused offers. Sensitivity is not a causal effect." + "[GradientExplainer](https://nvidia.github.io/structured-data-models/api/generated/sdm.explain.GradientExplainer.html) measures how sensitive each churn probability is to each input, including the categorical `plan` and `region`. For the riskiest customers here, late payments and billing contacts dominate, which points the retention team to billing-focused offers. Sensitivity is not a causal effect." ] }, { @@ -681,7 +674,7 @@ " \n", " \n", " billing_tickets_90d\n", - " 1.099764\n", + " 1.099765\n", " \n", " \n", " network_tickets_90d\n", @@ -707,6 +700,26 @@ " usage_change_pct_90d\n", " 0.047037\n", " \n", + " \n", + " monthly_charge\n", + " 0.037219\n", + " \n", + " \n", + " price_change_pct_12m\n", + " 0.036575\n", + " \n", + " \n", + " contract_months_left\n", + " 0.030719\n", + " \n", + " \n", + " plan\n", + " 0.023524\n", + " \n", + " \n", + " region\n", + " 0.004345\n", + " \n", " \n", "\n", "" @@ -714,13 +727,18 @@ "text/plain": [ " mean |gradient|\n", "late_payments_6m 1.454464\n", - "billing_tickets_90d 1.099764\n", + "billing_tickets_90d 1.099765\n", "network_tickets_90d 0.401423\n", "num_lines 0.299152\n", "tenure_months 0.271113\n", "dropped_calls_30d 0.076381\n", "data_usage_gb_30d 0.047753\n", - "usage_change_pct_90d 0.047037" + "usage_change_pct_90d 0.047037\n", + "monthly_charge 0.037219\n", + "price_change_pct_12m 0.036575\n", + "contract_months_left 0.030719\n", + "plan 0.023524\n", + "region 0.004345" ] }, "metadata": {}, @@ -734,7 +752,7 @@ "gradients = attributions.numerical.detach().cpu().numpy()[churn_class]\n", "drivers = pd.DataFrame(np.abs(gradients), columns=list(attributions.columns[sdm.Stype.numerical]),\n", " index=scoring.customer_id.iloc[riskiest])\n", - "display(drivers.mean().sort_values(ascending=False).head(8).rename(\"mean |gradient|\").to_frame())" + "display(drivers.mean().sort_values(ascending=False).rename(\"mean |gradient|\").to_frame())" ] }, { @@ -756,9 +774,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Rebuilt model reproduces the saved predictions.\n" - ] + "text": "Rebuilt model reproduces the saved predictions.\n" } ], "source": [ @@ -794,9 +810,7 @@ { "name": "stdout", "output_type": "stream", - "text": [ - "Outputs retained in /tmp/telco-churn-tabular-tnz9fz39\n" - ] + "text": "Outputs retained in /tmp/telco-churn-tabular-uw7crf8k\n" } ], "source": [ @@ -812,7 +826,7 @@ "source": [ "## Use your own data\n", "\n", - "Replace the generation cell with your own customer snapshots: one row per customer, your feature columns and a 0/1 churn label for past snapshots. Update `feature_columns` and `target`; the rest of the notebook runs as written. If your data lives in several related tables, use the [Kumo Engine churn notebook](../../../../kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb)." + "Replace the generation cell with a `customers` DataFrame of your own snapshots. It needs `customer_id`, `snapshot_date`, a `cohort` column with the values `\"context\"`, `\"heldout\"` and `\"score\"`, your feature columns and a 0/1 `churned_30d` label that is missing for `\"score\"` rows. Update `feature_columns`; the rest of the notebook runs as written. If your data lives in several related tables, use the [Kumo Engine churn notebook](../../../../kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb)." ] } ],