Skip to content

feat(engine): display batch prediction XAI in a notebook widget - #410

Draft
devanj08 wants to merge 1 commit into
feat/training-xai-widgetfrom
feat/batch-xai-widget
Draft

devanj08 wants to merge 1 commit into
feat/training-xai-widgetfrom
feat/batch-xai-widget

Conversation

@devanj08

@devanj08 devanj08 commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Display saved batch-prediction explanations with prediction.visualize() or PredictionExplanations.load(path).visualize().
Reuse the training widget renderer for cohort charts, sampled subgraphs and offline exports.
Identify the batch, model, snapshot, task and recorded row count in the header.

What Happens and Why

The batch XAI API exposes retained cohorts and graph samples but does not provide an integrated notebook view.
Batch predictions have no observed targets, so their widget must omit training diagnostics, outcome comparisons and target-derived error labels.

The Fix

Add a thin adapter over the validated prediction manifest and typed readers, honoring completed-producer allowlists and independent failed/disabled/unsupported states.
Extend the shared card and snapshot rendering only where batch data lacks outcomes or requires a single-tab layout.
Opening the widget reads saved artifacts and never reruns prediction or explanation generation.

Blast Radius

This PR stacks on the training widget PR #409 and the batch XAI API in #289; merge those prerequisites first.
It changes only display adapters and entrypoints, with no changes to prediction generation or its output schema.

Verification

The batch validation report records passing artifact/status tests and Chromium, Firefox and WebKit checks of the batch print document alongside shared training export tests.
Native notebook print/download dialogs and the full host/OS matrix were not validated by these local checks.

Change map.

@copy-pr-bot

copy-pr-bot Bot commented Nov 19, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@devanj08
devanj08 deployed to gha-kumo-workflow-tests November 19, 2026 06:50 — with GitHub Actions Active
@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Pre-merge checks: Passed 2 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Change Map Design Review Warning The change map is the PR comment with marker <!-- kumo-pr-change-map --> (id 7002013964, author devanj08). I fetched its full body and checked its 13 rows against the diff 0cde40b5..af77350f. The … Correct the change map and the design as follows. - For _columns.py (group_view, groups): state that batch cohort charts drop the MAX_GROUPS = 10 cap. Give the reason and the expected cohort bound. Either restore a bound for batch m…
Passed Passed checks: 2 passed

Check name Status Explanation
Pr Size Passed The PR #410 (NVIDIA/kumo-engine, head feat/batch-xai-widget at af77350, base feat/training-xai-widget) changes 13 files. The count is below the threshold of 40, so the PR size check passes. Evide…
Public Resource Access Passed The PR head is af77350, and its parent is 0cde40b, the training widget commit. I diffed the two revisions across 13 files (628 insertions, 76 deletions). The added lines contain no URL, git, package…
Full details: Pr Size

Explanation

The PR #410 (NVIDIA/kumo-engine, head feat/batch-xai-widget at af77350, base feat/training-xai-widget) changes 13 files. The count is below the threshold of 40, so the PR size check passes. Evidence: - The base branch feat/training-xai-widget is not present in the local checkout. The local history places commit 0cde40b ("Add training diagnostics and explainability widget") directly before head commit af77350 ("Display retained batch prediction explanations in notebooks"). This matches the PR description, which says the PR stacks on the training widget PR #409. - git diff --name-status -M HEAD~1 HEAD lists 13 files. git diff --shortstat HEAD~1 HEAD reports 13 files changed, 628 insertions, 76 deletions. Files include explainability.md, _explainability/_prediction.py (added), _columns.py, _snapshot.py, _subgraphs.py, static JS and CSS, explain/results.py, trainer/trainer.py, and test and validation files. - Caveat: the base tip is inferred from local history, not read from the provider. If the remote base has later commits, the GitHub file count could differ. Diffing against the full main line (main...HEAD) gives 1633 files, but that reflects the stacked prerequisite history, not this PR's changes against its target branch. Line counts were ignored, as the check requires.

Full details: Public Resource Access

Explanation

The PR head is af77350, and its parent is 0cde40b, the training widget commit. I diffed the two revisions across 13 files (628 insertions, 76 deletions). The added lines contain no URL, git, package-registry, CDN, or container reference. I searched for ://, http, www., CDN hosts, &lt;script src&gt;/&lt;link href&gt;, JS imports, Python URL or download calls, and package install commands. The only hits were: - http://www.w3.org/2000/svg SVG namespace strings in _snapshot.py and charts.js. These already existed on the base and are not added or modified lines. The SVG namespace is an identifier, not a fetched resource. - Local file URLs in widget_exports.cjs (pathToFileURL). These are local test fixtures. - Documentation examples, including /path/to/prediction/run and NODE_PATH=&lt;directory-containing-playwright&gt;. These are illustrative placeholders. The PR description links to PR #409, a commit blob, and a PR comment. These links are in the description, not in the changed repository content. They are not required to build, test, or use the changed code, so they fall outside this check's scope. No added external resource reference required public-access verification.

Full details: Change Map Design Review

Explanation

The change map is the PR comment with marker &lt;!-- kumo-pr-change-map --&gt; (id 7002013964, author devanj08). I fetched its full body and checked its 13 rows against the diff 0cde40b5..af77350f. The file list matches the diff. Most rows are accurate. These claims hold in code: batch cohorts are target-free (_columns.py has_targets, group_view), graph samples use the producer allowlist (results.py samples()/sample() via collector_card(..., explanations)), batch sidecars are skipped (output = {}), the single-tab layout is limited to kind == 'prediction' (shell.js mount), and visualize() runs no generation. The check fails on omitted material behavior: 1. Omitted removal of the group cap. _columns.py:193 sets limit = MAX_GROUPS if run is not None else len(entry['cohorts']). MAX_GROUPS = 10 (_columns.py:27) no longer applies to batch cohort charts. The names, pops, pred, and actual slices all use limit. The change map row for _columns.py says only "Allow cohort charts without observed targets." It does not state this behavior change, its rationale, or its bound. No test in test_prediction_explanations.py covers more than 10 cohorts. Whether the producer bounds the cohort count was not established. This is a proven omission in the table. The rendering risk is an uncertain design question. 2. Omitted shared-path changes. The _subgraphs.py row does not mention the new branch in collector_card. When task_type is empty, the function now infers task_type from the first stored sample and sets model_type from it. This branch sits in a shared function. _columns.py also changes the share calculation for all callers (if total_score else 0) and reformats the group_view dict literal. The table describes none of these. 3. No stopping condition. The comment states a feature request and the reuse of existing readers. It does not state a stopping condition or scope boundary. The check requires one in the same comment. 4. Requirement source. "Requested feature" names no linked issue or requirement. The map gives no source for the request. Uncertain design question (not a proven defect): batch mode uses run: Optional[Run] as a sentinel in shared functions (group_view, groups, collector_card). Those functions now branch in many places. A separate adapter may keep the training path simpler. The author should justify this choice. Verification claims in widget_validation.md (224 tests, browser runs, fingerprints) were not re-run. They are outside the change map.

Resolution

Correct the change map and the design as follows. - For _columns.py (group_view, groups): state that batch cohort charts drop the MAX_GROUPS = 10 cap. Give the reason and the expected cohort bound. Either restore a bound for batch mode (for example, keep MAX_GROUPS when run is None) or justify the unbounded render. Add a test with more than 10 cohorts. - For _columns.py: list the shared-path changes. These are the zero-total guard in share and the training-visible reformatting of the groups dict. Keep the reformatting out of the behavior description or revert it. - For _subgraphs.py (collector_card): add a row for the task and model inference when task_type is empty. Confirm that the training path never reaches it, or add a test for it. - Add a stopping condition to the change map. For example: the batch widget is complete when the listed artifact, status, and browser checks pass. Name the linked request or issue that requires the feature. - Keep the widget_exports.cjs batch case in a separate optional section, as the check requires. - Decide whether a separate batch adapter is simpler than the run: Optional[Run] sentinel in shared functions. Record the decision and its reason.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@devanj08

Copy link
Copy Markdown
Contributor Author

Change map

Requested feature: display retained batch XAI using the shared widget.
Existing artifact readers and shared widget shell are reused; no training or prediction generation is added.
Focused tests cover artifact and rendering contracts; local browser checks cover exports.

File Function / scope What changed Why
packages/kumo-engine/explainability.md widget display and saved-snapshot guide Document supported display/export behavior, retained-artifact requirements and inactive-notebook output. Explain the display APIs, exports and retained-artifact limitations.
packages/kumo-engine/kumo/engine/_explainability/_columns.py group_view; groups Allow cohort charts without observed targets, using sampled-row shares and omitting outcome/error labels. Make saved cohort distributions navigable with or without importance scores.
packages/kumo-engine/kumo/engine/_explainability/_prediction.py PredictionExplainabilityView Adapt the existing validated prediction XAI API into the shared widget without scoring, targets or evaluation metrics. Reuse the shared view while preserving independent producer outcomes.
packages/kumo-engine/kumo/engine/_explainability/_snapshot.py Snapshot.init; _cohorts; svg Render a batch-specific static title and retained cohort/subgraph coverage without individual customer graphs. Retain useful diagnostics when no kernel is active.
packages/kumo-engine/kumo/engine/_explainability/_subgraphs.py collector_card Use prediction manifest allowlists for graph samples and omit training-only prediction sidecars. Display saved graph samples without fabricating scores or reading unlisted batch artifacts.
packages/kumo-engine/kumo/engine/_explainability/static/shell.js KX.cards.groups; KX.cards.status; mount Render target-free cohorts, batch identity and producer status, with single-tab layout limited to prediction widgets. Show batch identity and producer outcomes while preserving training navigation.
packages/kumo-engine/kumo/engine/_explainability/static/subgraphs.js batch outcome details and coverage note Render target-free batch outcomes and describe retained prediction coverage in the graph note. Identify which batch artifacts the displayed graph describes.
packages/kumo-engine/kumo/engine/_explainability/static/widget.css batch title and identity styles Style the batch identity fields and larger batch title within the shared widget. Keep batch identity fields readable within the shared header.
packages/kumo-engine/kumo/engine/explain/results.py PredictionExplanations.visualize Expose batch widget display on existing PredictionExplanations objects without altering generation. Make loaded prediction explanations directly displayable.
packages/kumo-engine/kumo/engine/trainer/trainer.py PredictionResult.visualize Expose the requested visualize entrypoint on PredictionResult. Expose widget display on returned results without rerunning jobs.
packages/kumo-engine/test/test_prediction_explanations.py producer status, relocation and widget tests Extend existing batch fixtures with widget behavior for producer outcomes, task families, malformed artifacts, row counts and reopened runs. Verify manifest boundaries, producer failures and target-free batch rendering.
packages/kumo-engine/test/widget_exports.cjs optional batch export case Add the optional batch print-document check to the shared three-browser export harness. Verify batch plotted output and its single widget tab using the same browser checks.
packages/kumo-engine/test/widget_validation.md batch results Record batch test outcomes and browser evidence alongside the shared training results. Link reviewers to measured batch validation and its limitations.

This branch was successfully deployed

1 active deployment
gha-kumo-workflow-tests — af77350f Deployed Nov 19, 2026 by devanj08 via import #2679
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant