From 63e8811ff8a2d378e803b65e69ca906624d29ea6 Mon Sep 17 00:00:00 2001 From: Jana Gagacheva Date: Thu, 5 Nov 2026 16:24:21 -0800 Subject: [PATCH 1/2] add skills --- skills/relational/SKILL.md | 30 +++++++++++++++ .../references/beyond-one-answer.md | 38 +++++++++++++++++++ skills/tabular/SKILL.md | 26 +++++++++++++ 3 files changed, 94 insertions(+) diff --git a/skills/relational/SKILL.md b/skills/relational/SKILL.md index 1ab1a346e..b0157493e 100644 --- a/skills/relational/SKILL.md +++ b/skills/relational/SKILL.md @@ -192,6 +192,36 @@ A ranking is a call per candidate and a forecast a call per step; an explanation Endpoint support and task restrictions are described in [`references/beyond-one-answer.md`](references/beyond-one-answer.md). +An explanation is kept out of `to_pandas()`. +The gradients are on `result.explanations.gradients`, a mapping from each row's key to its entry. +The key is the entity's `id` when IDs are unique, or `row_index` when IDs are missing or repeated. +Forecast keys add the step: `(id, forecast_step)` or `(row_index, forecast_step)`. +One entry covers the entity's own row and every related row gathered for it, each under its table's name in `related`: + +```python +entry = result.explanations.gradients[1] # the entity whose id is 1 +entry['outputs'] # the explained scores, in order +entry['related']['users'] # the entity's table; the entity's own row is at hop 0 +block = entry['related']['orders'] # one block per related table +block['rows'] # each row's values as sent +block['keys'] # each row's key, when the table has a key of its own +block['columns'] # the model's preprocessed numerical columns +block['gradients'] # [output][row][column] +result.explanations.gradients.to_pandas() # one line per (entity, table, row, output, column) +analysis = result.explanations.columns +if analysis is not None: + cohorts = analysis.to_pandas() # one line per (table, hop, column, cohort, class) +``` + +Column explanations are on `result.explanations.columns`. +This client computes them from the labelled context `materialize()` gathered; the endpoint does not. +Column explanations group labelled context examples by input values and summarize their observed outcomes. +They describe associations, not how much a column contributed to an individual prediction. +See [beyond-one-answer.md](references/beyond-one-answer.md) for interpreting `mean`, `class`, and `share`. +`result.explanations.columns` is `None` unless `explanation` was asked for. +It can also be `None` when statistics are unavailable, so check it before calling `to_pandas()`. +Gradients and column explanations both need a deployment that serves `explanation`. + ## Asking the same context more than once `fit()` uploads a context once and pins it in the endpoint's memory; every diff --git a/skills/relational/references/beyond-one-answer.md b/skills/relational/references/beyond-one-answer.md index 138d06e24..ef12bffa2 100644 --- a/skills/relational/references/beyond-one-answer.md +++ b/skills/relational/references/beyond-one-answer.md @@ -123,8 +123,46 @@ Source keys can identify related rows; synthetic transport keys are not user key Only scored rows and their sampled related rows receive gradients, not the labelled context examples. A gradient measures sensitivity to a preprocessed feature, not a causal effect. +```python +entry = result.explanations.gradients[1] # the entity whose id is 1 +block = entry['related']['orders'] # one block per table, by name +block['rows'][0] # the first row's values, as sent +block['gradients'][0][0] # first output, first row: one gradient per name in block['columns'] +``` + Unique prediction IDs key the collection; missing/repeated IDs use `row_index`. Forecast keys include the step, and merging preserves distinct rows and steps. `result.explanations.gradients.to_pandas()` flattens entries with table, hop, related-row identity, output and column. `EXPLAIN` ranking is rejected before gathering; explanations requested later through `outputs` are checked before prediction. + +```python +result.explanations.gradients[(1, 2)] # on a forecast: entity 1, step 2 +result.explanations.gradients.to_pandas() # id, forecast_step on a forecast, table, hop, key, row, output, column, gradient +``` + +Column explanations are on `result.explanations.columns`, which is `None` unless `explanation` was asked for. +This client computes them from the labelled context `materialize()` gathered; the endpoint does not. +That context includes the rows held back from the call because they carried the answer. +Column explanations group the labelled entities into cohorts by each column of each table they reach. +Key columns and the columns a relationship joins on are left out. +Each table also gets `COUNT(*)`, how many of its rows each entity reaches. +`share` is the fraction of the population included in that column analysis that belongs to the cohort; `share: 0.2` means 20%. +Missing numerical values are excluded, so this population can differ between columns. +Consider how well each cohort is represented before interpreting differences; a small share does not necessarily mean a small absolute sample size. +For regression, `mean` is the average observed target within the cohort, in the target's units, and `class` is `None`. +For example, for an order total in dollars, `mean: 120` means an average observed total of $120. +For classification, `class` names the target label and `mean` is the proportion of the cohort with that observed label. +For example, `class: "churned"` and `mean: 0.3` means 30% of the cohort's labelled examples churned; it is not an individual prediction's probability. +Compare means within the same column and class, and the same table and hop. +These are associations in the context, not gradients and not causes. +A forecast gets one analysis, over the context of every step. +If the statistics cannot be computed, `result.explanations.columns` is `None` and a warning says so; the predictions still come back. + +```python +analysis = result.explanations.columns +if analysis is not None: + cohorts = analysis.to_pandas() # table, hop, column, stype, cohort, class, share, mean, score + cohorts[cohorts['table'] == 'orders'] # the cohorts of each column of orders +``` + See [the result contract](../../../docs/reference/prediction-output.md). diff --git a/skills/tabular/SKILL.md b/skills/tabular/SKILL.md index 93f5f2d5b..2691ba98e 100644 --- a/skills/tabular/SKILL.md +++ b/skills/tabular/SKILL.md @@ -55,6 +55,32 @@ The optional Python model runtime also supports them. `EXPLAIN` adds explanation output without dropping ordinary outputs; `outputs=[]` remains invalid. See [the result contract](../../docs/reference/prediction-output.md). +```python +# With missing or repeated IDs, keys are row_index values. +entry = result.explanations.gradients[0] # row_index 0 under that assumption +entry['outputs'] # the explained scores, in order +entry['instance']['columns'] # the model's preprocessed numerical columns +entry['instance']['gradients'] # [output][column] +result.explanations.gradients.to_pandas() # one line per (row, output, column) +analysis = result.explanations.columns +if analysis is not None: + cohorts = analysis.to_pandas() # one line per (column, cohort, class) +``` + +Column explanations are on `result.explanations.columns`, which is `None` unless `explanation` was asked for or when statistics are unavailable, so check it before calling `to_pandas()`. +This client computes them from the labelled rows sent with the request; the endpoint does not. +Column explanations group the labelled rows into cohorts by each column, leaving out the key, the target, and timestamps. +`share` is the fraction of the population included in that column analysis that belongs to the cohort; `share: 0.2` means 20%. +Missing numerical values are excluded, so this population can differ between columns. +Consider how well each cohort is represented before interpreting differences; a small share does not necessarily mean a small absolute sample size. +For regression, `mean` is the average observed target within the cohort, in the target's units, and `class` is `None`. +For example, for an order total in dollars, `mean: 120` means an average observed total of $120. +For classification, `class` names the target label and `mean` is the proportion of the cohort with that observed label. +For example, `class: "churned"` and `mean: 0.3` means 30% of the cohort's labelled examples churned; it is not an individual prediction's probability. +Compare means within the same column and class. +These are associations in the labelled rows, not gradients and not causes. +A session makes them once, at `fit()`, from the context it pins. + ## The whole interface ```python From 68ffb111ec249af18f0a4d3a714fc622f4663422 Mon Sep 17 00:00:00 2001 From: Jana Gagacheva Date: Fri, 6 Nov 2026 11:10:40 -0800 Subject: [PATCH 2/2] Explain fractional entity weighting in cohort statistic --- skills/relational/references/beyond-one-answer.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/skills/relational/references/beyond-one-answer.md b/skills/relational/references/beyond-one-answer.md index ef12bffa2..e2e1e1e41 100644 --- a/skills/relational/references/beyond-one-answer.md +++ b/skills/relational/references/beyond-one-answer.md @@ -146,13 +146,15 @@ That context includes the rows held back from the call because they carried the Column explanations group the labelled entities into cohorts by each column of each table they reach. Key columns and the columns a relationship joins on are left out. Each table also gets `COUNT(*)`, how many of its rows each entity reaches. +An entity whose rows fall in several cohorts is divided among them in proportion to its rows: a user with one category-A order and nine category-B orders counts as 0.1 of an entity in cohort A and 0.9 in cohort B. +`share` and `mean` are computed over these fractional weights, so they measure how the entities' rows are associated with each cohort, not the plain rate among entities that ever touched it. `share` is the fraction of the population included in that column analysis that belongs to the cohort; `share: 0.2` means 20%. Missing numerical values are excluded, so this population can differ between columns. Consider how well each cohort is represented before interpreting differences; a small share does not necessarily mean a small absolute sample size. For regression, `mean` is the average observed target within the cohort, in the target's units, and `class` is `None`. For example, for an order total in dollars, `mean: 120` means an average observed total of $120. For classification, `class` names the target label and `mean` is the proportion of the cohort with that observed label. -For example, `class: "churned"` and `mean: 0.3` means 30% of the cohort's labelled examples churned; it is not an individual prediction's probability. +For example, `class: "churned"` and `mean: 0.3` means a weighted 30% of the cohort churned; it is not an individual prediction's probability. Compare means within the same column and class, and the same table and hop. These are associations in the context, not gradients and not causes. A forecast gets one analysis, over the context of every step.