You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add synthetic telco churn example notebooks - #388
Adds two local example notebooks for telco customer churn on synthetic data, for the telco churn use case:
KumoTabular:structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb uses pretrained direct KumoTabular on a single engineered feature table. It covers context fitting, held-out evaluation, batch predictions, gradient explanations and context reuse.
Kumo Engine:kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb predicts 30-day cancellation from seven related tables. It covers a FAST plan, explanations, batch predictions on newly appended data and model reload.
Both notebooks are indexed under a new Telecommunications section.
Both notebooks were executed end to end on an L4 GPU with kumo-engine 1.0.1 and structured-data-models 0.2.1.
Saved outputs were trimmed of logs and progress bars.
Metrics are labeled demonstration only: tabular held-out AUROC 0.80; Engine test AUROC 0.76 and AUPRC 0.16. Engine training is seeded, but GPU training still varies slightly between runs (two seeded runs gave AUROC 0.75 and 0.76).
Requested: runnable synthetic telco churn examples for both KumoTabular (single feature table) and Kumo Engine (relational tables). Previously there was no telco vertical. Smallest change: two notebooks and their index entries. Verification: full execution on an L4 GPU with the released packages. Stopping condition: both notebooks run cleanly and are indexed.
Reviewing files that changed from the base of the PR and between 6479d36 and 0221481.
You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.
Use the checkbox below for a quick retry:
Trigger review
No actionable comments were generated in the recent review.
Walkthrough
The change adds two synthetic telecommunications churn notebooks. The Kumo Engine notebook generates historical and later-period data, trains and evaluates a model, and scores active customers. The KumoTabular notebook prepares context, held-out, and scoring cohorts, evaluates and explains predictions, and checks predictions after rebuilding the model from saved context. README updates link to both examples.
Sequence Diagram(s)
sequenceDiagram
participant Notebook as Kumo Engine notebook
participant Data as Synthetic telco data and Parquet tables
participant Graph as Time-aware graph and snapshots
participant Engine as Kumo Engine
participant Artifacts as Model and prediction outputs
Notebook->>Data: Generate historical and later-period records
Notebook->>Graph: Build graph and create historical snapshot
Graph->>Engine: Provide training rows and graph data
Engine->>Artifacts: Export trained model and metrics
Notebook->>Graph: Append later records and create scoring snapshot
Graph->>Engine: Provide eligible customers for scoring
Engine->>Artifacts: Save customer predictions
Loading
sequenceDiagram
participant Notebook as KumoTabular notebook
participant Cohorts as Synthetic customer cohorts and tensors
participant Tabular as Pretrained KumoTabular
participant Outputs as Metrics and predictions
participant Context as Saved context
Notebook->>Cohorts: Generate context, held-out, and scoring data
Cohorts->>Tabular: Provide tensors for evaluation and scoring
Tabular->>Outputs: Return metrics and churn scores
Notebook->>Tabular: Request gradient explanations for high-risk scores
Notebook->>Context: Save labeled context
Context->>Tabular: Rebuild model context for prediction check
Loading
Merge Risk: Low · previous review 5849f
These are example notebooks with no changes to library code. The remaining issues are documentation clarity: the tabular driver table covers numerical features only while the surrounding text suggests it covers all inputs, and the replacement-data requirements may be incomplete. Both are minor, so the PR is mergeable with small follow-ups.
Architecture risk: Low · up to 5849f
The change affects 1 system.
Changed systems:examples
Architecture concerns
No architecture-level concerns identified.
Review details
Systems and components
observed — examples (service) was modified; 5 changed files map to changed impact.
Before / after behavior
observed — Modified behavior in examples/kumo-example-notebooks/README.md: The Kumo Engine table of contents adds a link to the Telecommunications section.
observed — Modified behavior in examples/kumo-example-notebooks/README.md: The tabular Structured Data Models table of contents adds a link to the Telecommunications section.
observed — Modified behavior in examples/kumo-example-notebooks/kumo-engine/README.md: Adds a Telecommunications section and a Notebooks table linking the local synthetic telco churn notebook, labeled as synthetic temporal binary classification.
observed — Modified behavior in examples/kumo-example-notebooks/structured-data-models/README.md: Adds a Telecommunications category under Tabular, with a Notebooks listing for the direct KumoTabular telco churn example and its deployment and scope labels.
Pre-merge checks: 2 · 1
Failed checks: 1 inconclusive
Check name
Status
Explanation
Resolution
Change Map Design Review
The check reviews the PR conversation comment that begins with <!-- kumo-pr-change-map -->. That comment is not in the supplied context. The provided review comments list no CodeRabbit threads, and …
Provide the body of the PR conversation comment that begins with <!-- kumo-pr-change-map --> (PR #388, comment 6968369811), or make the GitHub conversation comments available to this check. Then re-run the review. The review should test e…
Passed checks: 2 passed
Check name
Status
Explanation
Pr Size
The PR size check counts changed files and fails only above 40. The reviewed range (refs/coderabbit/pre-merge-check/base to head) changes 5 files: 3 modified READMEs (examples/kumo-example-notebooks/R…
Public Resource Access
The PR adds external resource references in two notebooks and modifies two README indexes. I checked every added external URL with an anonymous curl request that followed redirects. All returned HTTP …
Full details: Pr Size
Explanation
The PR size check counts changed files and fails only above 40. The reviewed range (refs/coderabbit/pre-merge-check/base to head) changes 5 files: 3 modified READMEs (examples/kumo-example-notebooks/README.md, kumo-engine/README.md, structured-data-models/README.md) and 2 added notebooks (kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb and structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb). git diff --name-status and --name-only both agree on 5 files. Line counts are ignored by this check. The count is well below the 40-file threshold, so the check passes.
Full details: Public Resource Access
Explanation
The PR adds external resource references in two notebooks and modifies two README indexes. I checked every added external URL with an anonymous curl request that followed redirects. All returned HTTP 200. Added external references: - Engine notebook (kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynb): pip install -q kumo-engine (PyPI project page, 200). Documentation links on docs.nvidia.com (linux-machine-deployment, gpu-sizing, binary-classification, graphs, predictive-queries, training-tables, training, model-explainability, making-predictions, publish-and-retrieve-models), all 200. - Tabular notebook (structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb): pip install -q structured-data-models (PyPI project page, 200). Pretrained weights at https://huggingface.co/nvidia/Kumo-Tabular (200), which the notebook downloads at runtime, so it is required to use the code. API docs on nvidia.github.io for KumoTabular and GradientExplainer (200). - README files: the added links are repository-relative paths within this PR's repository. They are not external references. Each added reference is public and reachable without credentials. Since no added reference is confirmed inaccessible, the check does not fail.
Full details: Change Map Design Review
Explanation
The check reviews the PR conversation comment that begins with <!-- kumo-pr-change-map -->. That comment is not in the supplied context. The provided review comments list no CodeRabbit threads, and the PR description links to the comment only by URL (issuecomment-6968369811). I could not fetch it because no GitHub API or conversation access is available in this run. The repository checkout does not contain the comment body. A search for the marker found only docs/development/review.md. That file describes the process: one comment with the marker, file coverage, and a link from the description. It does not contain the map itself. The committed range changes five files, which matches the raw summary: two README indexes, two README sections, and two notebooks (local_kumo_engine_synthetic_telco_churn.ipynb and local_structured_data_models_telco_churn.ipynb). File coverage alone does not satisfy this check. I did not assess the Change map's rationale, necessity, abstraction level, verification, or stopping condition, because the map text was unavailable. I did not find a proven defect in the change map. I also did not establish that it is correct.
Resolution
Provide the body of the PR conversation comment that begins with <!-- kumo-pr-change-map --> (PR #388, comment 6968369811), or make the GitHub conversation comments available to this check. Then re-run the review. The review should test each proposed change against the observed failure or release requirement, the existing behavior in the repository, the smallest sufficient change, focused verification, and the stopping condition. It should also check whether the explanations omit material notebook behavior.
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
1034-1036: Minor · Functional Correctness · Quick win
Select the explanation file from the trial that was chosen, not from an unsorted rglob result.
rglob("explain_test/scores.csv") searches the whole run directory. A run can contain several trial directories under models/task/automl/, and each can hold its own explain_test/scores.csv. The code takes score_files[0], which depends on enumeration order and is not guaranteed to be sorted. The sensitivity table can then show the scores of a trial other than the selected model.
Resolve the scores file from the selected trial's directory, or read the gradient scores through the explanation result's public accessor. Do not select the file by its position in an rglob result.
123-123: Minor · Functional Correctness · Quick win
Explain the effect of DECISION_THRESHOLD on this output.
At DECISION_THRESHOLD = 0.5, TARGET_PRED is positive for at most one of the 2581 scored customers. The highest True_PROB is 0.500259, and the next is 0.424149. A reader who uses TARGET_PRED as the retention list gets an almost empty list.
The markdown in the "Inspect the results" cell only calls the threshold "untuned". Either choose a threshold from the validation split, or select a top-k or top-percentile of True_PROB. If the threshold stays at 0.5, state that TARGET_PRED is not useful for this model.
835-837: Minor · Functional Correctness · Quick win
State that the driver table covers numerical features only.
The driver table is built only from attributions.numerical and attributions.columns[sdm.Stype.numerical]. The categorical features plan and region cannot appear in it. In the data generator, plan == "sim_only_rolling" is a strong churn driver. The markdown above the cell says the output shows "which signals drive the riskiest customers' scores". A reader can take this table as the complete driver ranking. Add a sentence to the markdown cell saying that the table covers numerical features only and that categorical attributions are not shown. Alternatively, include the categorical attributions if GradientExplainer returns them.
The cell loads the bundle and prints a path. It never calls reloaded_trainer.predict. A load that succeeds but produces a broken or mismatched model would not be caught. The user would not learn that the bundle is reusable, which is the claim of the "Retain and reuse" section.
Score the same prediction_table with reloaded_trainer on scoring_snapshot. Compare True_PROB with scores for each ENTITY, for example with np.allclose after sorting both frames by ENTITY. This confirms the saved bundle gives the same predictions.
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Note
Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.
Caution
Some comments are outside the diff and can’t be posted inline due to GitHub limitations.
⚠️ Outside diff range comments (1)
Minor · Limit retention selection to the contact budget. · local_structured_data_models_telco_churn.ipynb:629
examples/kumo-example-notebooks/structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynb:629 Minor · Functional Correctness · Quick win
Limit retention selection to the contact budget.
If scores tie at the percentile cutoff, >= can select more than TOP_SHARE permits. With ten scores [0.1, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2], this selects nine customers instead of one.
Select an explicit number of highest-scoring rows with deterministic tie-breaking. Use the same selection rule for the held-out metrics at Line 490.
1189-1189: Minor · Functional Correctness · Quick win
Document the in-memory data required for scoring.
If users replace the generation cells with Parquet folders as instructed, the scoring steps still require new_rows at Line 924 and tables at Lines 956–957. Neither variable comes from the updated graph, so this procedure stops with NameError.
Explain how to construct these variables from user data, or update scoring to read the Parquet tables directly.
815-815: Minor · Functional Correctness · Quick win
Document the complete replacement-data schema.
A table with only the documented features and historical label cannot run unchanged. Line 348 requires cohort with values "context", "heldout" and "score". The prediction export also requires customer_id and snapshot_date.
List these required columns and cohort values. Explain that scoring rows need an unknown target value. Otherwise, following these instructions can stop execution before context preparation.
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.
@leena-kang Thanks for running both notebooks! Engine training is now seeded (plan.training_job.manual_seed = RANDOM_SEED, in 5849fae). GPU training still varies a little: two seeded runs gave test AUROC 0.752 and 0.764. The notebook now says the seed reduces variation rather than making reruns identical. The same commit also addresses the CodeRabbit notes.
Add a direct KumoTabular notebook on a single engineered feature table and
a Kumo Engine notebook on seven related tables, and index both under a new
Telecommunications section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seed Engine training, read explanations from the selected trial, flag a
top-share retention list instead of relying on the 0.5 threshold, verify
the reloaded model's predictions, show every Tabular feature attribution,
select exactly the top share without ties, and list the data each
notebook needs when using your own data.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two local example notebooks for telco customer churn on synthetic data, for the telco churn use case:
structured-data-models/tabular/telco/notebooks/local_structured_data_models_telco_churn.ipynbuses pretrained direct KumoTabular on a single engineered feature table. It covers context fitting, held-out evaluation, batch predictions, gradient explanations and context reuse.kumo-engine/telco/notebooks/local_kumo_engine_synthetic_telco_churn.ipynbpredicts 30-day cancellation from seven related tables. It covers a FAST plan, explanations, batch predictions on newly appended data and model reload.Change map: #388 (comment)
Testing Done
kumo-engine1.0.1 andstructured-data-models0.2.1.Checklist
🤖 Generated with Claude Code