From 5aa26858d2239b2350235519b8a31c98c3826d11 Mon Sep 17 00:00:00 2001 From: Jana Gagacheva <39542883+JGagacheva@users.noreply.github.com> Date: Mon, 9 Nov 2026 10:15:06 -0800 Subject: [PATCH] Extend the Databricks notebook examples with XAI (#71) Extend notebook to include xai definitions and examples --------- Signed-off-by: Jana Gagacheva (cherry picked from commit 0f95e84097035ee6c52bbfa445d1909b8fa5f023) --- .../kumo_relational_quickstart.ipynb | 175 ++++++++++++++++++ .../databricks/kumo_tabular_quickstart.ipynb | 154 +++++++++++++++ 2 files changed, 329 insertions(+) diff --git a/examples/databricks/kumo_relational_quickstart.ipynb b/examples/databricks/kumo_relational_quickstart.ipynb index 8a6f1b9c8..06fd88f8b 100644 --- a/examples/databricks/kumo_relational_quickstart.ipynb +++ b/examples/databricks/kumo_relational_quickstart.ipynb @@ -464,6 +464,10 @@ "\n", "A comparison after the target turns it into a yes-or-no question: `COUNT(sales_order_headers.*, 0, 90, days) = 0` asks whether a customer places no order in the next 90 days.\n", "A `WHERE` inside the aggregation counts only some rows: `COUNT(sales_order_headers.* WHERE sales_order_headers.TotalDue > 1000, 0, 90, days)` counts only orders over $1,000.\n", + "\n", + "**Explanations**\n", + "\n", + "`EXPLAIN PREDICT …` returns the same predictions, and also explanations of them; see Example 5.\n", "" ] }, @@ -884,6 +888,177 @@ "This lets a team ask questions about the measures it already defines in metric views." ] }, + { + "cell_type": "markdown", + "id": "620b2a07b418435c90d8e7dca1fd869f", + "metadata": {}, + "source": [ + "### Example 5: Explain the predictions\n", + "\n", + "We predicted which customers are likely to place no orders in the next 90 days. Now we'll explore patterns in the historical examples and inspect how one customer's prediction responds to small input changes.\n", + "\n", + "Explanations are available with NIM version 1.1+ and client version 1.1.0+.\n", + "\n", + "The result carries explanations at two levels:\n", + "\n", + "- `result.explanations.columns`, computed by the client from the historical examples the model reads. It groups customers by ranges or categories in each column. Each group reports its share of the examples and the fraction who placed no orders in the following 90 days.\n", + "- `result.explanations.gradients`, returned by the model for each explained customer. A gradient describes how the predicted probability responds to a small change in a preprocessed numerical input. It does not measure how much that input contributed to the prediction.\n", + "\n", + "Add `EXPLAIN` to request both column analysis and gradients." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b2ee7bc6cf2e43d8b7e7e3207117ea3d", + "metadata": {}, + "outputs": [], + "source": [ + "# Explain the 25 customers Example 1 rated riskiest.\n", + "riskiest_25 = churn.loc[riskiest[:25], 'id'].tolist()\n", + "request = PredictiveQuery(\n", + " 'EXPLAIN PREDICT COUNT(sales_order_headers.*, 0, 90, days) = 0 '\n", + " 'FOR EACH customers.CustomerID',\n", + " schema,\n", + ").materialize(indices=riskiest_25, anchor_time=PAST, num_context_samples=5000)\n", + "result = client.predict(request, model=model)" + ] + }, + { + "cell_type": "markdown", + "id": "dce952ea80cc428fb83782dd23aa2b3f", + "metadata": {}, + "source": [ + "#### What patterns do we see in the historical data?\n", + "\n", + "The column analysis groups customers by ranges or categories in each column. For each group, the table shows what percentage of the historical examples it represents and what percentage placed no orders in the following 90 days.\n", + "\n", + "For numerical columns in linked tables, values are averaged per customer at each hop before grouping customers into cohorts. For categorical and timestamp columns, individual linked rows are grouped instead. For `OrderDate`, each order falls into a window of time before the customer's anchor date. The sampled history is split into ten equal windows, and their width and unit (hours, days, weeks or months) depend on how far back it reaches. A customer whose orders fall into several windows is split across them by their share of orders: with three of four orders in one window, they count as three quarters there and one quarter in the window holding the fourth order." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2b5bb9ac990d40baab07c0a0cefab7e9", + "metadata": {}, + "outputs": [], + "source": [ + "columns = result.explanations.columns.to_pandas()\n", + "\n", + "# True means the customer placed no orders in the following 90 days.\n", + "groups = columns.loc[\n", + " columns['class'] == True, # noqa: E712\n", + " ['table', 'hop', 'column', 'cohort', 'share', 'mean'],\n", + "].copy()\n", + "\n", + "display(groups)" + ] + }, + { + "cell_type": "markdown", + "id": "7e07694e4f5c42a191903997b324abfc", + "metadata": {}, + "source": [ + "Each row answers: **How common is this group, and how often did customers in it stop ordering?**\n", + "\n", + "For example, a row with `0.30` under `share` and `0.45` under `mean` would mean that the group contains 30% of the historical examples, and 45% of that group placed no orders.\n", + "\n", + "#### Do customers with older ordering activity stop ordering more often?\n", + "\n", + "Zoom in on `OrderDate`. Each group is a window of time before the anchor date, and its percentages are weighted by how much of each customer's orders fall into it. Customers with no orders before the anchor date do not appear in these groups." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c2becede8cda4ba09b497c1d48b39369", + "metadata": {}, + "outputs": [], + "source": [ + "order_dates = groups[\n", + " (groups['table'] == 'sales_order_headers')\n", + " & (groups['column'] == 'OrderDate')\n", + "]\n", + "display(order_dates)" + ] + }, + { + "cell_type": "markdown", + "id": "27b22710d2de41ff9da8cc017d6dfb61", + "metadata": {}, + "source": [ + "Compare the percentages across time windows within the same hop. If older groups have higher percentages of customers placing no orders, that is a pattern in these historical examples. The group sizes show how much of the data the pattern covers.\n", + "\n", + "This helps you understand the data behind the predictions. It does not establish why a particular customer received their risk score.\n", + "\n", + "#### How would small input changes affect one customer's predicted probability?\n", + "\n", + "Gradients describe how the model output responds to small changes near the current input values. They use the model's numerical inputs after preprocessing, which may differ from the original columns.\n", + "\n", + "For the riskiest customer, inspect ten entries with the largest absolute gradients:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7b914676ad1e4ecbac19bf90c24bb735", + "metadata": {}, + "outputs": [], + "source": [ + "gradients = result.explanations.gradients.to_pandas()\n", + "customer_gradients = gradients[\n", + " (gradients['id'] == riskiest_25[0])\n", + " & (gradients['output'] == 'probability_True')\n", + "]\n", + "display(\n", + " customer_gradients.loc[\n", + " customer_gradients['gradient'].abs().nlargest(10).index\n", + " ]\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "a7b667632bfa4a48be8e94665ef115ca", + "metadata": {}, + "source": [ + "A positive gradient means a small increase in that preprocessed value would locally increase the predicted probability of placing no orders; a negative gradient means it would decrease it.\n", + "\n", + "The sizes depend on the input scales, so these ten entries are not necessarily the inputs that contributed most to the prediction.\n", + "\n", + "Use this table to investigate sensitivity—for example, when checking a possibly incorrect source value. Correct the value using the source record, rerun the prediction, and check whether the customer's risk score changes enough to affect your outreach decision. Gradients alone do not tell you which business action would prevent churn." + ] + }, + { + "cell_type": "markdown", + "id": "be17fd43", + "metadata": {}, + "source": [ + "#### 5.2 Ask with outputs\n", + "\n", + "The `EXPLAIN` prefix and the `outputs` argument ask for the same thing.\n", + "To keep the query text fixed and decide per call, pass `outputs=['prediction', 'probabilities', 'explanation']`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1575e783", + "metadata": {}, + "outputs": [], + "source": [ + "# The same request and explanations, without the EXPLAIN prefix.\n", + "request = PredictiveQuery(\n", + " 'PREDICT COUNT(sales_order_headers.*, 0, 90, days) = 0 '\n", + " 'FOR EACH customers.CustomerID',\n", + " schema,\n", + ").materialize(indices=riskiest_25, anchor_time=PAST, num_context_samples=5000)\n", + "result = client.predict(\n", + " request, model=model, outputs=['prediction', 'probabilities', 'explanation']\n", + ")\n", + "print(f'{len(result.explanations.gradients)} predictions explained.')" + ] + }, { "cell_type": "markdown", "id": "064697a1bad1b5348afae26f2d9acec4", diff --git a/examples/databricks/kumo_tabular_quickstart.ipynb b/examples/databricks/kumo_tabular_quickstart.ipynb index 00eb0c3be..aebe6bc0a 100644 --- a/examples/databricks/kumo_tabular_quickstart.ipynb +++ b/examples/databricks/kumo_tabular_quickstart.ipynb @@ -572,6 +572,160 @@ "2. The printed line counts the correct predictions for the same 60 products as Example 1, from the same context rows, so it matches Example 1's count." ] }, + { + "cell_type": "markdown", + "id": "723085f0ff6f44b5b0dfe42d2b5ceb5c", + "metadata": {}, + "source": [ + "### Example 4: Explain the predictions\n", + "\n", + "We predicted the missing product colors. Now we'll explore patterns in the labelled examples and inspect how one product's prediction responds to small input changes.\n", + "\n", + "Explanations are available with NIM version 1.1+ and client version 1.1.0+.\n", + "The result carries explanations at two levels:\n", + "\n", + "- `result.explanations.columns`, computed by the client from the labelled examples sent with the request. It groups products by ranges or categories in each column. Each group reports its share of the examples and the fraction with each color.\n", + "- `result.explanations.gradients`, returned by the model for each explained product. A gradient describes how a predicted color probability responds to a small change in a preprocessed numerical input. It does not measure how much that input contributed to the prediction.\n", + "\n", + "#### 4.1 Ask with the EXPLAIN prefix\n", + "\n", + "Ask the question of Example 3 again with `EXPLAIN`, using the same context rows, for the first 25 of its 60 products." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c2b144db005e4007b736b07327635d98", + "metadata": {}, + "outputs": [], + "source": [ + "# The first 25 of the 60 products Example 3 asked about.\n", + "to_explain = ask.head(25)\n", + "request = PredictiveQuery(\n", + " f'EXPLAIN PREDICT products.{TARGET} FOR EACH products.ProductID', schema\n", + ").materialize(\n", + " predict=to_explain.drop(columns=[TARGET, 'predicted']), model=model\n", + ")\n", + "result = client.predict(request, model=model)" + ] + }, + { + "cell_type": "markdown", + "id": "2d235bbbc5014c188350db902322dac1", + "metadata": {}, + "source": [ + "**What patterns do we see in the labelled examples?**\n", + "\n", + "The column analysis groups products by ranges or categories in each column. For example, `ProductLine` groups products by their product line. For each group and color, `share` is the fraction of examples in the group, and `mean` is the fraction of that group with that color." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1c6ab73514aa4619ae9920119137e7f8", + "metadata": {}, + "outputs": [], + "source": [ + "columns = result.explanations.columns.to_pandas()\n", + "display(columns)" + ] + }, + { + "cell_type": "markdown", + "id": "9e8280d327e342b9b20d39fe5a61d0ec", + "metadata": {}, + "source": [ + "**How does product line relate to color?**\n", + "\n", + "Zoom in on `ProductLine` for the most common color in the labelled examples." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a7ea75bd12541bc951cd34c2e3581c0", + "metadata": {}, + "outputs": [], + "source": [ + "line = columns[\n", + " (columns['column'] == 'ProductLine') & (columns['class'] == most_common)\n", + "]\n", + "display(line)" + ] + }, + { + "cell_type": "markdown", + "id": "6283031b7cd94cac9e8ed6589febbcf4", + "metadata": {}, + "source": [ + "Compare `mean` across the product lines: which have a larger fraction of products with this color? Check `share` to see how much of the data each line represents.\n", + "\n", + "This shows a pattern in the labelled examples. It does not establish why a particular product received its predicted color.\n", + "\n", + "**How would small input changes affect one product's predicted color probability?**\n", + "\n", + "Gradients describe how the model output responds to small changes near the current input values. They use the model's numerical inputs after preprocessing, which may differ from the original columns.\n", + "\n", + "For the first product, inspect ten entries with the largest absolute gradients for the color predicted in Example 3." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a6bdaedb262542d3891889e49ae4cf8e", + "metadata": {}, + "outputs": [], + "source": [ + "gradients = result.explanations.gradients.to_pandas()\n", + "first = gradients[\n", + " (gradients['id'] == to_explain['ProductID'].iloc[0])\n", + " & (gradients['output'] == f'probability_{to_explain[\"predicted\"].iloc[0]}')\n", + "]\n", + "display(first.loc[first['gradient'].abs().nlargest(10).index])" + ] + }, + { + "cell_type": "markdown", + "id": "5e62cda0a14d4db5a8bcf0d7421b5be6", + "metadata": {}, + "source": [ + "A positive gradient means a small increase in that preprocessed value would locally increase the predicted probability of this color; a negative gradient means it would decrease it.\n", + "\n", + "The sizes depend on the input scales, so these ten entries are not necessarily the inputs that contributed most to the prediction.\n", + "\n", + "Use this table to investigate sensitivity—for example, when checking a possibly incorrect product attribute. Correct the value using the source record, rerun the prediction, and check whether the predicted color or its probability changes. This checks how the model responds to a data correction; it does not show that changing an attribute would change the product's actual color." + ] + }, + { + "cell_type": "markdown", + "id": "98b3f19452054386a12523a3856674b7", + "metadata": {}, + "source": [ + "#### 4.2 Ask with outputs\n", + "\n", + "The `EXPLAIN` prefix and the `outputs` argument ask for the same thing.\n", + "To keep the query text fixed and decide per call, pass `outputs=['prediction', 'probabilities', 'explanation']`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6b99b712a6394627adea279bbf9416ea", + "metadata": {}, + "outputs": [], + "source": [ + "# The same request and explanations, without the EXPLAIN prefix.\n", + "request = PredictiveQuery(\n", + " f'PREDICT products.{TARGET} FOR EACH products.ProductID', schema\n", + ").materialize(\n", + " predict=to_explain.drop(columns=[TARGET, 'predicted']), model=model\n", + ")\n", + "result = client.predict(\n", + " request, model=model, outputs=['prediction', 'probabilities', 'explanation']\n", + ")\n", + "print(f'{len(result.explanations.gradients)} predictions explained.')" + ] + }, { "cell_type": "markdown", "id": "35191b5835272b7849ed573c1de6294e",