Skip to content

Add synthetic telco churn example notebooks - #388

Merged
aliu65803 merged 4 commits into
mainfrom
feat/telco-churn-example-notebooks
Nov 19, 2026
Merged

aliu65803 merged 4 commits into
mainfrom
feat/telco-churn-example-notebooks

Conversation

@aliu65803

@aliu65803 aliu65803 commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds two local example notebooks for telco customer churn on synthetic data, for the telco churn use case:

  • KumoTabular: structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb uses pretrained direct KumoTabular on a single engineered feature table. It covers context fitting, held-out evaluation, batch predictions, gradient explanations and context reuse.
  • Kumo Engine: kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb predicts 30-day cancellation from seven related tables. It covers a FAST plan, explanations, batch predictions on newly appended data and model reload.
  • Both notebooks are indexed under a new Telecommunications section.

Change map: #388 (comment)

Testing Done

  • Both notebooks were executed end to end on an L4 GPU with kumo-engine 1.0.1 and structured-data-models 0.2.1.
  • Saved outputs were trimmed of logs and progress bars.
  • Metrics are labeled demonstration only: tabular held-out AUROC 0.80; Engine test AUROC 0.76 and AUPRC 0.16. Engine training is seeded, but GPU training still varies slightly between runs (two seeded runs gave AUROC 0.75 and 0.76).

Checklist

  • No breaking changes
  • Updated example indexes
  • Unit tests: not applicable (example notebooks)

🤖 Generated with Claude Code

@copy-pr-bot

copy-pr-bot Bot commented Nov 18, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@aliu65803
aliu65803 temporarily deployed to gha-kumo-workflow-tests November 18, 2026 03:43 — with GitHub Actions Inactive
@aliu65803

aliu65803 commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor Author

Change map

Requested: runnable synthetic telco churn examples for both KumoTabular (single feature table) and Kumo Engine (relational tables). Previously there was no telco vertical. Smallest change: two notebooks and their index entries. Verification: full execution on an L4 GPU with the released packages. Stopping condition: both notebooks run cleanly and are indexed.

File Function / scope What changed Why
examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb New notebook Requires a GPU; explains when to use pretrained KumoTabular; synthetic churn snapshots; KumoTabular fit, evaluation, exact top-share retention flag, all-feature explanations, context reuse, own-data schema Pretrained tabular churn example
examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb New notebook Requires a supported GPU; explains when to use Kumo Engine; seven synthetic telco tables; 30-day cancellation query, seeded FAST training, selected-trial explanations, top-share retention list, reload with prediction check, own-data guidance Relational Engine churn example
examples/kumo-example-notebooks/README.md, examples/kumo-example-notebooks/kumo-engine/README.md, examples/kumo-example-notebooks/structured-data-models/README.md Example indexes Added Telecommunications sections and table of contents links Make the notebooks discoverable

@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Explain Change →

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 6479d36 and 0221481.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review

No actionable comments were generated in the recent review.

Walkthrough

The change adds two synthetic telecommunications churn notebooks. The Kumo Engine notebook generates historical and later-period data, trains and evaluates a model, and scores active customers. The KumoTabular notebook prepares context, held-out, and scoring cohorts, evaluates and explains predictions, and checks predictions after rebuilding the model from saved context. README updates link to both examples.

Sequence Diagram(s)

sequenceDiagram
  participant Notebook as Kumo Engine notebook
  participant Data as Synthetic telco data and Parquet tables
  participant Graph as Time-aware graph and snapshots
  participant Engine as Kumo Engine
  participant Artifacts as Model and prediction outputs
  Notebook->>Data: Generate historical and later-period records
  Notebook->>Graph: Build graph and create historical snapshot
  Graph->>Engine: Provide training rows and graph data
  Engine->>Artifacts: Export trained model and metrics
  Notebook->>Graph: Append later records and create scoring snapshot
  Graph->>Engine: Provide eligible customers for scoring
  Engine->>Artifacts: Save customer predictions
Loading
sequenceDiagram
  participant Notebook as KumoTabular notebook
  participant Cohorts as Synthetic customer cohorts and tensors
  participant Tabular as Pretrained KumoTabular
  participant Outputs as Metrics and predictions
  participant Context as Saved context
  Notebook->>Cohorts: Generate context, held-out, and scoring data
  Cohorts->>Tabular: Provide tensors for evaluation and scoring
  Tabular->>Outputs: Return metrics and churn scores
  Notebook->>Tabular: Request gradient explanations for high-risk scores
  Notebook->>Context: Save labeled context
  Context->>Tabular: Rebuild model context for prediction check
Loading
Merge Risk: Low · previous review 5849f

These are example notebooks with no changes to library code. The remaining issues are documentation clarity: the tabular driver table covers numerical features only while the surrounding text suggests it covers all inputs, and the replacement-data requirements may be incomplete. Both are minor, so the PR is mergeable with small follow-ups.

Architecture risk: Low · up to 5849f

The change affects 1 system.

Changed systems: examples

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — examples (service) was modified; 5 changed files map to changed impact.

Before / after behavior

  • observed — Modified behavior in examples/kumo-example-notebooks/README.md: The Kumo Engine table of contents adds a link to the Telecommunications section.
  • observed — Modified behavior in examples/kumo-example-notebooks/README.md: The tabular Structured Data Models table of contents adds a link to the Telecommunications section.
  • observed — Modified behavior in examples/kumo-example-notebooks/kumo-engine/README.md: Adds a Telecommunications section and a Notebooks table linking the local synthetic telco churn notebook, labeled as synthetic temporal binary classification.
  • observed — Modified behavior in examples/kumo-example-notebooks/structured-data-models/README.md: Adds a Telecommunications category under Tabular, with a Notebooks listing for the direct KumoTabular telco churn example and its deployment and scope labels.
Pre-merge checks: Passed 2 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check name Status Explanation Resolution
Change Map Design Review Inconclusive The check reviews the PR conversation comment that begins with <!-- kumo-pr-change-map -->. That comment is not in the supplied context. The provided review comments list no CodeRabbit threads, and … Provide the body of the PR conversation comment that begins with <!-- kumo-pr-change-map --> (PR #388, comment 6968369811), or make the GitHub conversation comments available to this check. Then re-run the review. The review should test e…
Passed Passed checks: 2 passed

Check name Status Explanation
Pr Size Passed The PR size check counts changed files and fails only above 40. The reviewed range (refs/coderabbit/pre-merge-check/base to head) changes 5 files: 3 modified READMEs (examples/kumo-example-notebooks/R…
Public Resource Access Passed The PR adds external resource references in two notebooks and modifies two README indexes. I checked every added external URL with an anonymous curl request that followed redirects. All returned HTTP …
Full details: Pr Size

Explanation

The PR size check counts changed files and fails only above 40. The reviewed range (refs/coderabbit/pre-merge-check/base to head) changes 5 files: 3 modified READMEs (examples/kumo-example-notebooks/README.md, kumo-engine/README.md, structured-data-models/README.md) and 2 added notebooks (kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb and structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb). git diff --name-status and --name-only both agree on 5 files. Line counts are ignored by this check. The count is well below the 40-file threshold, so the check passes.

Full details: Public Resource Access

Explanation

The PR adds external resource references in two notebooks and modifies two README indexes. I checked every added external URL with an anonymous curl request that followed redirects. All returned HTTP 200. Added external references: - Engine notebook (kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb): pip install -q kumo-engine (PyPI project page, 200). Documentation links on docs.nvidia.com (linux-machine-deployment, gpu-sizing, binary-classification, graphs, predictive-queries, training-tables, training, model-explainability, making-predictions, publish-and-retrieve-models), all 200. - Tabular notebook (structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb): pip install -q structured-data-models (PyPI project page, 200). Pretrained weights at https://huggingface.co/nvidia/Kumo-Tabular (200), which the notebook downloads at runtime, so it is required to use the code. API docs on nvidia.github.io for KumoTabular and GradientExplainer (200). - README files: the added links are repository-relative paths within this PR's repository. They are not external references. Each added reference is public and reachable without credentials. Since no added reference is confirmed inaccessible, the check does not fail.

Full details: Change Map Design Review

Explanation

The check reviews the PR conversation comment that begins with &lt;!-- kumo-pr-change-map --&gt;. That comment is not in the supplied context. The provided review comments list no CodeRabbit threads, and the PR description links to the comment only by URL (issuecomment-6968369811). I could not fetch it because no GitHub API or conversation access is available in this run. The repository checkout does not contain the comment body. A search for the marker found only docs/development/review.md. That file describes the process: one comment with the marker, file coverage, and a link from the description. It does not contain the map itself. The committed range changes five files, which matches the raw summary: two README indexes, two README sections, and two notebooks (local_kumo_engine_synthetic_telco_churn.ipynb and local_structured_data_models_telco_churn.ipynb). File coverage alone does not satisfy this check. I did not assess the Change map's rationale, necessity, abstraction level, verification, or stopping condition, because the map text was unavailable. I did not find a proven defect in the change map. I also did not establish that it is correct.

Resolution

Provide the body of the PR conversation comment that begins with &lt;!-- kumo-pr-change-map --&gt; (PR #388, comment 6968369811), or make the GitHub conversation comments available to this check. Then re-run the review. The review should test each proposed change against the observed failure or release requirement, the existing behavior in the repository, the smallest sufficient change, focused verification, and the stopping condition. It should also check whether the explanations omit material notebook behavior.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between ad021c9 and 5849fae.

Files selected for processing: 2

  • examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb
  • examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

Other comments: 3

examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb: 2

1034-1036: Minor · Functional Correctness · Quick win

Select the explanation file from the trial that was chosen, not from an unsorted rglob result.

rglob("explain_test/scores.csv") searches the whole run directory. A run can contain several trial directories under models/task/automl/, and each can hold its own explain_test/scores.csv. The code takes score_files[0], which depends on enumeration order and is not guaranteed to be sorted. The sensitivity table can then show the scores of a trial other than the selected model.

Resolve the scores file from the selected trial's directory, or read the gradient scores through the explanation result's public accessor. Do not select the file by its position in an rglob result.


123-123: Minor · Functional Correctness · Quick win

Explain the effect of DECISION_THRESHOLD on this output.

At DECISION_THRESHOLD = 0.5, TARGET_PRED is positive for at most one of the 2581 scored customers. The highest True_PROB is 0.500259, and the next is 0.424149. A reader who uses TARGET_PRED as the retention list gets an almost empty list.

The markdown in the "Inspect the results" cell only calls the threshold "untuned". Either choose a threshold from the validation split, or select a top-k or top-percentile of True_PROB. If the threshold stays at 0.5, state that TARGET_PRED is not useful for this model.

examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb: 1

835-837: Minor · Functional Correctness · Quick win

State that the driver table covers numerical features only.

The driver table is built only from attributions.numerical and attributions.columns[sdm.Stype.numerical]. The categorical features plan and region cannot appear in it. In the data generator, plan == "sim_only_rolling" is a strong churn driver. The markdown above the cell says the output shows "which signals drive the riskiest customers' scores". A reader can take this table as the complete driver ranking. Add a sentence to the markdown cell saying that the table covers numerical features only and that categorical attributions are not shown. Alternatively, include the categorical attributions if GradientExplainer returns them.

Nitpick comments: 1

examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb: 1

1334-1335: Trivial · Functional Correctness · Quick win

Use reloaded_trainer to verify the reload.

The cell loads the bundle and prints a path. It never calls reloaded_trainer.predict. A load that succeeds but produces a broken or mismatched model would not be caught. The user would not learn that the bundle is reusable, which is the claim of the "Retain and reuse" section.

Score the same prediction_table with reloaded_trainer on scoring_snapshot. Compare True_PROB with scores for each ENTITY, for example with np.allclose after sorting both frames by ENTITY. This confirms the saved bundle gives the same predictions.

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between b53781b and 880f253.

Files selected for processing: 5

  • examples/kumo-example-notebooks/README.md
  • examples/kumo-example-notebooks/kumo-engine/README.md
  • examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb
  • examples/kumo-example-notebooks/structured-data-models/README.md
  • examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@aliu65803
aliu65803 temporarily deployed to gha-kumo-workflow-tests November 18, 2026 16:40 — with GitHub Actions Inactive
@aliu65803
aliu65803 temporarily deployed to gha-kumo-workflow-tests November 18, 2026 17:09 — with GitHub Actions Inactive

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

Minor · Limit retention selection to the contact budget. · local_structured_data_models_telco_churn.ipynb:629

examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb:629
Minor · Functional Correctness · Quick win

Limit retention selection to the contact budget.

If scores tie at the percentile cutoff, >= can select more than TOP_SHARE permits. With ten scores [0.1, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2], this selects nine customers instead of one.

Select an explicit number of highest-scoring rows with deterministic tie-breaking. Use the same selection rule for the held-out metrics at Line 490.

Other comments: 2

examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb: 1

1189-1189: Minor · Functional Correctness · Quick win

Document the in-memory data required for scoring.

If users replace the generation cells with Parquet folders as instructed, the scoring steps still require new_rows at Line 924 and tables at Lines 956–957. Neither variable comes from the updated graph, so this procedure stops with NameError.

Explain how to construct these variables from user data, or update scoring to read the Parquet tables directly.

examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb: 1

815-815: Minor · Functional Correctness · Quick win

Document the complete replacement-data schema.

A table with only the documented features and historical label cannot run unchanged. Line 348 requires cohort with values "context", "heldout" and "score". The prediction export also requires customer_id and snapshot_date.

List these required columns and cohort values. Explain that scoring rows need an unknown target value. Otherwise, following these instructions can stop execution before context preparation.

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between c0f889e and ad021c9.

Files selected for processing: 2

  • examples/kumo-example-notebooks/kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb
  • examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

@leena-kang

leena-kang commented Nov 18, 2026 •

Copy link
Copy Markdown

@aliu65803 - Consider seeding Engine training for more stable tutorial result. Otherwise, the notebooks both run e2e on my side (in brev)

@aliu65803
aliu65803 temporarily deployed to gha-kumo-workflow-tests November 18, 2026 22:58 — with GitHub Actions Inactive
@aliu65803

Copy link
Copy Markdown
Contributor Author

@leena-kang Thanks for running both notebooks! Engine training is now seeded (plan.training_job.manual_seed = RANDOM_SEED, in 5849fae). GPU training still varies a little: two seeded runs gave test AUROC 0.752 and 0.764. The notebook now says the seed reduces variation rather than making reruns identical. The same commit also addresses the CodeRabbit notes.

@leena-kang leena-kang self-assigned this Nov 18, 2026
@leena-kang
leena-kang self-requested a review November 18, 2026 23:48
@aliu65803
aliu65803 temporarily deployed to gha-kumo-workflow-tests November 18, 2026 23:56 — with GitHub Actions Inactive
aliu65803 and others added 4 commits November 18, 2026 18:13
Add a direct KumoTabular notebook on a single engineered feature table and
a Kumo Engine notebook on seven related tables, and index both under a new
Telecommunications section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seed Engine training, read explanations from the selected trial, flag a
top-share retention list instead of relying on the 0.5 threshold, verify
the reloaded model's predictions, show every Tabular feature attribution,
select exactly the top share without ties, and list the data each
notebook needs when using your own data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aliu65803
aliu65803 force-pushed the feat/telco-churn-example-notebooks branch from 6479d36 to 0221481 Compare November 19, 2026 00:13
@aliu65803
aliu65803 temporarily deployed to gha-kumo-workflow-tests November 19, 2026 00:13 — with GitHub Actions Inactive
@aliu65803
aliu65803 merged commit d8aaf2b into main Nov 19, 2026
122 checks passed
@aliu65803
aliu65803 deleted the feat/telco-churn-example-notebooks branch November 19, 2026 00:38

This branch was previously deployed

1 inactive deployment
gha-kumo-workflow-tests — 0221481c Deployed Nov 19, 2026 by aliu65803 via import #2650
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants