Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
7bd9710
Publish the kumo-data-science skill to the NVIDIA skills catalog
kumo-isaac Nov 6, 2026
113fde0
nvskills-publish: post the change map and name /ok to test
kumo-isaac Nov 6, 2026
49ff664
Publish from main: commit the catalog copy with each skill change
kumo-isaac Nov 6, 2026
87151c3
Merge origin/main into igutierrez/nvskills-publish
kumo-isaac Nov 9, 2026
65a7ea2
Refresh the published copy for #250's kumo-model-explanations change
kumo-isaac Nov 9, 2026
f3fb77c
publish_nvskills: refuse catalog collisions and signed names at any d…
kumo-isaac Nov 9, 2026
13c6910
agent-skills README: list what the first publish needs; name /ok to t…
kumo-isaac Nov 9, 2026
8976fad
generic runtime: keep each location's project, catalog, warehouse and…
kumo-isaac Nov 9, 2026
a251cc3
kumo-data-science: verify name-based keys, name what a run replaces, …
kumo-isaac Nov 9, 2026
9a35780
Merge origin/main into igutierrez/nvskills-publish
kumo-isaac Nov 12, 2026
a8d8016
Refresh the published copy for #195's skill changes
kumo-isaac Nov 12, 2026
b73963a
test_publish_nvskills: read and write the router as UTF-8
kumo-isaac Nov 12, 2026
7b63c2b
preflight: probe declared primary keys too
kumo-isaac Nov 12, 2026
d9d0ab8
kumo-graph-visualization: never overwrite a file with a default export
kumo-isaac Nov 12, 2026
2507a7f
kumoai-reference: generation-plan fields and destination timestamp types
kumo-isaac Nov 12, 2026
21c1f7a
kumo-data-science: bind population replacement to its name; stage it …
kumo-isaac Nov 12, 2026
0feaf88
kumo-data-science: restate the binary threshold for local predictions…
kumo-isaac Nov 12, 2026
465899b
publish_nvskills: publish each nested skill as its own catalog skill
kumo-isaac Nov 12, 2026
2105a3a
kumo-graph-visualization: rerunning a script redraws its own diagram
kumo-isaac Nov 12, 2026
ecf2318
agent-skills: address CodeRabbit's 2026-11-12 follow-ups
kumo-isaac Nov 13, 2026
dc9b3bf
Merge origin/main into igutierrez/nvskills-publish
kumo-isaac Nov 14, 2026
7ace05d
Merge branch 'main' into igutierrez/nvskills-publish
kumo-isaac Nov 17, 2026
2ab10b0
agent-skills: parse a formatted CSV time column when staging a popula…
kumo-isaac Nov 17, 2026
1dc2be3
agent-skills: publish one directory per skill, as the NVSkills catalo…
kumo-isaac Nov 17, 2026
432514d
agent-skills: skill cards and evals for the NVSkills catalog
kumo-isaac Nov 17, 2026
9126db0
Merge origin/main into igutierrez/nvskills-publish
kumo-isaac Nov 17, 2026
1a2852d
agent-skills: cast a time column first; link skill cards to the publi…
kumo-isaac Nov 17, 2026
53c3892
Merge remote-tracking branch 'origin/main' into igutierrez/nvskills-p…
kumo-isaac Nov 17, 2026
c7554e3
agent-skills: quote BigQuery and Snowflake names in generated SQL
kumo-isaac Nov 18, 2026
9680c1a
Attach NVSkills validation signatures
svc-nvskills-signing Nov 18, 2026
2857ed7
agent-skills: keep the skill card NVSkills CI writes when it signs
kumo-isaac Nov 18, 2026
5026ce4
agent-skills: align a test signature for flake8
kumo-isaac Nov 18, 2026
7cbda7a
agent-skills: let sf_name keep an already-quoted Snowflake name
kumo-isaac Nov 18, 2026
f22b610
Attach NVSkills validation signatures
svc-nvskills-signing Nov 18, 2026
83ce9a8
agent-skills: stage a population time column in its declared format f…
kumo-isaac Nov 18, 2026
8aaa6cb
agent-skills: ask the persistence evals for notebook code
kumo-isaac Nov 18, 2026
7218a19
Attach NVSkills validation signatures
svc-nvskills-signing Nov 18, 2026
bfbf737
agent-skills: read a CSV time column as text only for a format Arrow …
kumo-isaac Nov 18, 2026
bef5ab4
Attach NVSkills validation signatures
svc-nvskills-signing Nov 19, 2026
2bbedbe
agent-skills: null CSV tokens in the time column only; retry %z witho…
kumo-isaac Nov 19, 2026
7c56017
Attach NVSkills validation signatures
svc-nvskills-signing Nov 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/agent-skills.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ on: # yamllint disable-line rule:truthy
- main
paths:
- "agent-skills/**"
# The copy NVIDIA/skills publishes; it must match agent-skills/ (publish_nvskills.py).
- "skills/**"
# The skills cite modules and names from these packages; a rename must fail here.
- "packages/kumo-engine/**"
- "packages/kumo-core/**"
Expand All @@ -20,6 +22,8 @@ on: # yamllint disable-line rule:truthy
pull_request:
paths:
- "agent-skills/**"
# The copy NVIDIA/skills publishes; it must match agent-skills/ (publish_nvskills.py).
- "skills/**"
# The skills cite modules and names from these packages; a rename must fail here.
- "packages/kumo-engine/**"
- "packages/kumo-core/**"
Expand Down
83 changes: 83 additions & 0 deletions agent-skills/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,3 +126,86 @@ The skills cite Kumo code by module path, as the agent sees it in a notebook
resolves each one against `packages/`, and checks every `kumo.<name>` against `kumo.engine.__all__`,
so the workflow also runs on changes to `kumo-engine`, `kumo-core`, `kumo-api` and `kumo-snowflake`.
`kumoml.*` lives outside this repository and is not checked.

## Publishing to the NVIDIA skills catalog

[NVIDIA/skills](https://github.com/NVIDIA/skills) pulls every product's skills from its source repo
once an hour. It accepts only content that NVSkills CI has evaluated and signed. NVSkills CI runs
only on PRs that change a root-level `skills/` directory. So the catalog reads a built copy in
`skills/` at the repository root, not `agent-skills/skills/`.

The catalog does not allow a skill inside another skill. The package nests seven skills inside the
router (`kumoai-reference/` and the six `kumo-*/` capability skills), so the published copy lays
all eight out side by side: `skills/kumo-data-science/`, `skills/kumoai-reference/`,
`skills/kumo-model-tuning/` and so on, each its own catalog skill. Their links to one another
(`../kumo-model-persistence/SKILL.md`) resolve the same way in both layouts. A backticked path that
starts with a skill's name (`kumoai-reference/references/outputs.md`) names that skill's directory,
as the router's *Paths in these files* says; any other path is relative to the skill that names it.
`publish_nvskills.py` checks every reference in the published layout too, so a path that resolves
only when nested (`accept/preflight.md` written in `kumoai-reference/`) fails the build: write it
`kumo-data-science/accept/preflight.md`.

The catalog sync deletes any skill directory that lacks its own `skill.oms.sig`, `skill-card.md`
and eval dataset:

| Path, in each of the eight skill directories | Written by |
|---|---|
| everything else | `agent-skills/scripts/publish_nvskills.py`: the `generic` package |
| `evals/` | copied from `agent-skills/nvskills/<skill>/`, e.g. `agent-skills/nvskills/kumo-model-tuning/evals/evals.json` |
| `skill-card.md` | first from `agent-skills/nvskills/<skill>/`; NVSkills CI rewrites it when it signs, adding the evaluation results, and from then on a refresh leaves it alone. To change a signed card, edit `skills/<skill>/skill-card.md` in your PR |
| `BENCHMARK.md`, `skill.oms.sig` | NVSkills CI, in your PR; never by hand |

Once `agent-skills/nvskills/` exists, the build fails unless every skill has both a card and an
`evals/*.json`, so a new skill cannot be published without them. Until then
`publish_nvskills.py` lists what is missing.

Whatever is on `main` is what the catalog publishes. To change the skill:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not true yet: agent-skills/nvskills/kumo-data-science/ (named on line 140) does not exist, NVIDIA/skills has no components.d/kumo-engine.yml, and NVSKILLS_CI_DISPATCH_TOKEN is unset. List what must exist before the first publish here (skill-card.md, evals/evals.json, the secret, the onboarding PR, a PASS BENCHMARK.md) instead of this sentence.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 13c6910: the sentence is replaced by a "Before the first publish" checklist: skill-card.md, evals/evals.json, the NVSKILLS_CI_DISPATCH_TOKEN secret, a signed copy on main with a PASS BENCHMARK.md, and the onboarding PR adding components.d/kumo-engine.yml.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed at 2105a3a: the checklist at README.md:168 lists the card and evals per skill, the secret, the signed copy and the onboarding PR.

1. Edit `agent-skills/`, never `skills/`.
2. Run `python3 agent-skills/scripts/publish_nvskills.py` and commit `skills/` with your change.
`tests/test_publish_nvskills.py` fails the PR if you forget. It ignores the `SKILL-BUILD`
stamp, so a rebuild on an unrelated commit is not required.
3. A maintainer comments `/nvskills-ci` on the PR. NVSkills CI runs the evals and pushes
`BENCHMARK.md` and `skill.oms.sig` as a commit by `nv-skills-ci[bot]`. copy-pr-bot does not
trust that commit, so `kumo-core-gpu-ci` runs only after a maintainer comments
`/ok to test <sha of that commit>`. A later push to the PR needs another `/nvskills-ci`.
4. Merge. The catalog picks the change up within the hour.

A change merged without step 3, a revert included, is "sig drift" to the catalog sync: it keeps
the last signed copy, so the catalog serves content that `main` no longer has until the next signed
merge. Run `/nvskills-ci` on reverts too.

### Before the first publish

The catalog publishes nothing until all of this is done. Each skill has its own card and evals
under `agent-skills/nvskills/`. The cards are rendered with the catalog's `skill-card-generator`
(template in NVIDIA/skills `docs/skill-cards.mdx`) and pass its `validate_submission.py`. The
evals use the `evals/evals.json` schema of published catalog skills, and assert only behaviour the
skill's own files document. Each set has one negative case (`expected_skill: null`).

| Skill | Card | Evals |
|---|---|---|
| `kumo-data-science` (router) | [x] `kumo-data-science/skill-card.md` | [x] `kumo-data-science/evals/evals.json` |
| `kumoai-reference` | [x] `kumoai-reference/skill-card.md` | [x] `kumoai-reference/evals/evals.json` |
| `kumo-custom-populations` | [x] `kumo-custom-populations/skill-card.md` | [x] `kumo-custom-populations/evals/evals.json` |
| `kumo-graph-visualization` | [x] `kumo-graph-visualization/skill-card.md` | [x] `kumo-graph-visualization/evals/evals.json` |
| `kumo-model-explanations` | [x] `kumo-model-explanations/skill-card.md` | [x] `kumo-model-explanations/evals/evals.json` |
| `kumo-model-persistence` | [x] `kumo-model-persistence/skill-card.md` | [x] `kumo-model-persistence/evals/evals.json` |
| `kumo-model-tuning` | [x] `kumo-model-tuning/skill-card.md` | [x] `kumo-model-tuning/evals/evals.json` |
| `kumo-training-diagnostics` | [x] `kumo-training-diagnostics/skill-card.md` | [x] `kumo-training-diagnostics/evals/evals.json` |

- [x] the `NVSKILLS_CI_DISPATCH_TOKEN` repository secret, from the NVSkills CI maintainers;
`request-nvskills-ci.yml` fails without it
- [x] kumo-engine registered with NVSkills CI (NVIDIA/nvskills-ci#112: `main`, external profile,
no fork PRs) and the `nv-skills-ci` GitHub App installed
- [x] the catalog PR to NVIDIA/skills adding `components.d/kumo-engine.yml`, with one `skills:`
entry per directory (`path: skills/<skill>/`, `catalog_dir: <skill>`), all eight:
NVIDIA/skills#885, a draft. Its `verify-source-onboarding` check verifies only the paths the
file lists, and passes only once the signed copy is on `main`
- [x] `request-nvskills-ci.yml` on `main` (NVIDIA/kumo-engine#371). `/nvskills-ci` is an
`issue_comment` trigger, so it runs only from `main`'s copy
- [ ] confirmation from the NVSkills CI maintainers that a skill may link to files in a sibling
skill directory (`kumo-data-science/` reads `kumoai-reference/`). Each directory is signed
and published as its own skill
- [ ] a signed copy on `main`: a PR through steps 1–4 whose `BENCHMARK.md` says PASS in every
skill directory
72 changes: 72 additions & 0 deletions agent-skills/nvskills/kumo-custom-populations/evals/evals.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
{
"skill_name": "kumo-custom-populations",
"evals": [
{
"id": "kpop-01",
"prompt": "Train my confirmed Kumo churn query on our own labels instead of generated ones. The Snowflake table ML.LABELS.CURATED_LABELS has ENTITY1, TARGET1, TIMESTAMP, WEIGHT and SPLIT (0 train, 1 validation, 2 test). `pq` and `snapshot` are already built in the notebook.",
"expected_output": "Before any code it replies with the plan and stops for confirmation: the table and its fields, WEIGHT and SPLIT as the weight and split columns, what it validated, and the constructor `kumo.TrainingTable.from_source_table(population_connector['CURATED_LABELS'], TrainingTableSpec(weight_col='WEIGHT', split_col='SPLIT'), pq=pq, snapshot=snapshot)`, reusing the confirmed pq and snapshot, with `suggest_model_plan(..., train_table_spec=spec)`.",
"assertions": [
"Names kumo.TrainingTable.from_source_table as the constructor",
"Uses TrainingTableSpec with weight_col='WEIGHT' and split_col='SPLIT'",
"Reuses the existing pq and snapshot",
"Passes the same spec to suggest_model_plan as train_table_spec",
"Presents the plan and stops for the user's confirmation before training"
],
"expected_skill": "kumo-custom-populations",
"expected_script": null
},
{
"id": "kpop-02",
"prompt": "Our Kumo labels table ML.LABELS.CURATED_LABELS has a SPLIT_LABEL column with the values 'train', 'val' and 'test'. Use it as the split column and train.",
"expected_output": "Reports SPLIT_LABEL as the offending field: the split column reads only 0, 1 and 2, so string labels match no split and their rows are skipped (the build fails with 'custom-TT source kept no rows' if none remain). It shows the labels and their counts from a read-only query and asks for either a prepared table with 0/1/2 or explicit approval of a shown mapping, which would then be applied in a read-only connector.sql(...) source. It writes nothing back.",
"assertions": [
"Says the split column must hold 0, 1 or 2 and that 'train'/'val'/'test' will not match",
"Explains that unmatched rows are skipped, so training could keep no rows",
"Asks for a prepared 0/1/2 table or explicit approval of a mapping it shows",
"Proposes the mapping only as a read-only connector.sql(...) source, after approval",
"Does not UPDATE the table, CREATE TABLE AS, or write a derived table"
],
"expected_skill": "kumo-custom-populations",
"expected_script": null
},
{
"id": "kpop-03",
"prompt": "For my Kumo custom training table, just normalize the WEIGHT column to mean 1 and multiply positives by 10. You have my approval, go ahead.",
"expected_output": "An approval given before the formula and distribution are shown is not approval. It states the exact formula and the columns it reads, shows the resulting weight distribution (min, mean, max, counts per label) from a read-only query, and stops for explicit approval. Only then would it apply the formula in a read-only connector.sql(...) source; it never writes to the source table.",
"assertions": [
"Does not apply the weight change straight away despite the up-front approval",
"Shows the exact weight formula and the columns it reads",
"Shows or proposes a read-only query for the resulting weight distribution",
"Stops for explicit approval after showing them",
"Does not UPDATE the source table or write a derived table"
],
"expected_skill": "kumo-custom-populations",
"expected_script": null
},
{
"id": "kpop-04",
"prompt": "Score exactly the accounts in ML.EVAL.EVAL_ACCOUNTS (ACCOUNT_ID, ANCHOR_TS, CHURNED) with my fitted Kumo trainer, so I can compare against CHURNED afterwards.",
"expected_output": "The scoring population must not carry the target, so the table cannot be passed as is. Evaluation labels (CHURNED) stay separate and are joined to the scores afterwards. It proposes, for approval, a read-only connector.sql(...) source keeping only the entity and TIMESTAMP columns under their contract names, or asks for a table without the target, and scores with `kumo.PredictionTable.from_source_table(..., pq=pq, snapshot=snapshot, plan=trainer.training_table_generation_plan)` then `trainer.predict(population, snapshot=snapshot, ...)`, comparing output entity and anchor coverage with the population.",
"assertions": [
"Says the scoring population must not contain the target column CHURNED",
"Keeps CHURNED separate for joining to the scores afterwards",
"Proposes a read-only connector.sql(...) source with the entity and timestamp columns, or asks for a table without the target",
"Uses kumo.PredictionTable.from_source_table with pq and snapshot",
"Says output rows or coverage will be compared with the population rather than assumed equal"
],
"expected_skill": "kumo-custom-populations",
"expected_script": null
},
{
"id": "kpop-05",
"prompt": "Split my pandas DataFrame into stratified train, validation and test sets (70/15/15) with scikit-learn.",
"expected_output": "A scikit-learn data-splitting task unrelated to Kumo. The agent answers with train_test_split, without loading the Kumo custom-populations skill.",
"assertions": [
"Does not load or follow the kumo-custom-populations skill",
"Uses scikit-learn train_test_split with stratification to produce 70/15/15 splits"
],
"expected_skill": null,
"expected_script": null
}
]
}
45 changes: 45 additions & 0 deletions agent-skills/nvskills/kumo-custom-populations/skill-card.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
## Description: <br>
Train with caller-supplied labels, weights and splits or score exact entity/anchor rows through Snowflake source tables using the local Kumo API. <br>

This skill is ready for commercial/non-commercial use. <br>

## Owner
NVIDIA <br>

### License/Terms of Use: <br>
Apache 2.0 <br>
## Use Case: <br>
Developers and data scientists use this skill with coding agents when generated Kumo training or prediction tables do not represent the rows they need, for example curated labels, manual splits, sample weights or an exact scoring population. <br>

### Deployment Geography for Use: <br>
Global <br>

## Requirements / Dependencies: <br>
**Requires API Key or External Credential:** [Yes] <br>
**Credential Type(s):** [Other [Snowflake connection or Snowpark session of the user's notebook]] <br>

Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. See skill body for more details. <br>

## Known Risks and Mitigations: <br>
Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. <br>
Mitigation: Review and scan skill before deployment. <br>

## Reference(s): <br>
- [references/populations.md](https://github.com/NVIDIA/kumo-engine/blob/main/skills/kumo-custom-populations/references/populations.md) <br>
- [NVIDIA/kumo-engine](https://github.com/NVIDIA/kumo-engine) <br>


## Skill Output: <br>
**Output Type(s):** [Code, Analysis] <br>
**Output Format:** [Markdown with inline Python and SQL code blocks] <br>
**Output Parameters:** [1D] <br>
**Other Properties Related to Output:** [Reads source tables only through read-only queries; never writes back to them.] <br>

## Skill Version(s): <br>
Unversioned; published from the main branch of NVIDIA/kumo-engine (source: repository; no skill version, CHANGELOG entry or release tag covers the skill files). The signed copy is identified by the skill.oms.sig file in this directory. <br>

## Ethical Considerations: <br>
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>

(For Release on NVIDIA Platforms Only) <br>
Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail). <br>
Loading
Loading