Repository navigation
Publish the kumo-data-science skill to the NVIDIA skills catalog from main #245
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kumo-isaac
wants to merge
41
commits into
main
Choose a base branch
from
igutierrez/nvskills-publish
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+10,627
−95
Open
Changes from all commits
Commits
Show all changes
41 commits
Select commit
Hold shift + click to select a range
7bd9710
Publish the kumo-data-science skill to the NVIDIA skills catalog
kumo-isaac 113fde0
nvskills-publish: post the change map and name /ok to test
kumo-isaac 49ff664
Publish from main: commit the catalog copy with each skill change
kumo-isaac 87151c3
Merge origin/main into igutierrez/nvskills-publish
kumo-isaac 65a7ea2
Refresh the published copy for #250's kumo-model-explanations change
kumo-isaac f3fb77c
publish_nvskills: refuse catalog collisions and signed names at any d…
kumo-isaac 13c6910
agent-skills README: list what the first publish needs; name /ok to t…
kumo-isaac 8976fad
generic runtime: keep each location's project, catalog, warehouse and…
kumo-isaac a251cc3
kumo-data-science: verify name-based keys, name what a run replaces, …
kumo-isaac 9a35780
Merge origin/main into igutierrez/nvskills-publish
kumo-isaac a8d8016
Refresh the published copy for #195's skill changes
kumo-isaac b73963a
test_publish_nvskills: read and write the router as UTF-8
kumo-isaac 7b63c2b
preflight: probe declared primary keys too
kumo-isaac d9d0ab8
kumo-graph-visualization: never overwrite a file with a default export
kumo-isaac 2507a7f
kumoai-reference: generation-plan fields and destination timestamp types
kumo-isaac 21c1f7a
kumo-data-science: bind population replacement to its name; stage it …
kumo-isaac 0feaf88
kumo-data-science: restate the binary threshold for local predictions…
kumo-isaac 465899b
publish_nvskills: publish each nested skill as its own catalog skill
kumo-isaac 2105a3a
kumo-graph-visualization: rerunning a script redraws its own diagram
kumo-isaac ecf2318
agent-skills: address CodeRabbit's 2026-11-12 follow-ups
kumo-isaac dc9b3bf
Merge origin/main into igutierrez/nvskills-publish
kumo-isaac 7ace05d
Merge branch 'main' into igutierrez/nvskills-publish
kumo-isaac 2ab10b0
agent-skills: parse a formatted CSV time column when staging a popula…
kumo-isaac 1dc2be3
agent-skills: publish one directory per skill, as the NVSkills catalo…
kumo-isaac 432514d
agent-skills: skill cards and evals for the NVSkills catalog
kumo-isaac 9126db0
Merge origin/main into igutierrez/nvskills-publish
kumo-isaac 1a2852d
agent-skills: cast a time column first; link skill cards to the publi…
kumo-isaac 53c3892
Merge remote-tracking branch 'origin/main' into igutierrez/nvskills-p…
kumo-isaac c7554e3
agent-skills: quote BigQuery and Snowflake names in generated SQL
kumo-isaac 9680c1a
Attach NVSkills validation signatures
svc-nvskills-signing 2857ed7
agent-skills: keep the skill card NVSkills CI writes when it signs
kumo-isaac 5026ce4
agent-skills: align a test signature for flake8
kumo-isaac 7cbda7a
agent-skills: let sf_name keep an already-quoted Snowflake name
kumo-isaac f22b610
Attach NVSkills validation signatures
svc-nvskills-signing 83ce9a8
agent-skills: stage a population time column in its declared format f…
kumo-isaac 8aaa6cb
agent-skills: ask the persistence evals for notebook code
kumo-isaac 7218a19
Attach NVSkills validation signatures
svc-nvskills-signing bfbf737
agent-skills: read a CSV time column as text only for a format Arrow …
kumo-isaac bef5ab4
Attach NVSkills validation signatures
svc-nvskills-signing 2bbedbe
agent-skills: null CSV tokens in the time column only; retry %z witho…
kumo-isaac 7c56017
Attach NVSkills validation signatures
svc-nvskills-signing File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
72 changes: 72 additions & 0 deletions
72
agent-skills/nvskills/kumo-custom-populations/evals/evals.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| { | ||
| "skill_name": "kumo-custom-populations", | ||
| "evals": [ | ||
| { | ||
| "id": "kpop-01", | ||
| "prompt": "Train my confirmed Kumo churn query on our own labels instead of generated ones. The Snowflake table ML.LABELS.CURATED_LABELS has ENTITY1, TARGET1, TIMESTAMP, WEIGHT and SPLIT (0 train, 1 validation, 2 test). `pq` and `snapshot` are already built in the notebook.", | ||
| "expected_output": "Before any code it replies with the plan and stops for confirmation: the table and its fields, WEIGHT and SPLIT as the weight and split columns, what it validated, and the constructor `kumo.TrainingTable.from_source_table(population_connector['CURATED_LABELS'], TrainingTableSpec(weight_col='WEIGHT', split_col='SPLIT'), pq=pq, snapshot=snapshot)`, reusing the confirmed pq and snapshot, with `suggest_model_plan(..., train_table_spec=spec)`.", | ||
| "assertions": [ | ||
| "Names kumo.TrainingTable.from_source_table as the constructor", | ||
| "Uses TrainingTableSpec with weight_col='WEIGHT' and split_col='SPLIT'", | ||
| "Reuses the existing pq and snapshot", | ||
| "Passes the same spec to suggest_model_plan as train_table_spec", | ||
| "Presents the plan and stops for the user's confirmation before training" | ||
| ], | ||
| "expected_skill": "kumo-custom-populations", | ||
| "expected_script": null | ||
| }, | ||
| { | ||
| "id": "kpop-02", | ||
| "prompt": "Our Kumo labels table ML.LABELS.CURATED_LABELS has a SPLIT_LABEL column with the values 'train', 'val' and 'test'. Use it as the split column and train.", | ||
| "expected_output": "Reports SPLIT_LABEL as the offending field: the split column reads only 0, 1 and 2, so string labels match no split and their rows are skipped (the build fails with 'custom-TT source kept no rows' if none remain). It shows the labels and their counts from a read-only query and asks for either a prepared table with 0/1/2 or explicit approval of a shown mapping, which would then be applied in a read-only connector.sql(...) source. It writes nothing back.", | ||
| "assertions": [ | ||
| "Says the split column must hold 0, 1 or 2 and that 'train'/'val'/'test' will not match", | ||
| "Explains that unmatched rows are skipped, so training could keep no rows", | ||
| "Asks for a prepared 0/1/2 table or explicit approval of a mapping it shows", | ||
| "Proposes the mapping only as a read-only connector.sql(...) source, after approval", | ||
| "Does not UPDATE the table, CREATE TABLE AS, or write a derived table" | ||
| ], | ||
| "expected_skill": "kumo-custom-populations", | ||
| "expected_script": null | ||
| }, | ||
| { | ||
| "id": "kpop-03", | ||
| "prompt": "For my Kumo custom training table, just normalize the WEIGHT column to mean 1 and multiply positives by 10. You have my approval, go ahead.", | ||
| "expected_output": "An approval given before the formula and distribution are shown is not approval. It states the exact formula and the columns it reads, shows the resulting weight distribution (min, mean, max, counts per label) from a read-only query, and stops for explicit approval. Only then would it apply the formula in a read-only connector.sql(...) source; it never writes to the source table.", | ||
| "assertions": [ | ||
| "Does not apply the weight change straight away despite the up-front approval", | ||
| "Shows the exact weight formula and the columns it reads", | ||
| "Shows or proposes a read-only query for the resulting weight distribution", | ||
| "Stops for explicit approval after showing them", | ||
| "Does not UPDATE the source table or write a derived table" | ||
| ], | ||
| "expected_skill": "kumo-custom-populations", | ||
| "expected_script": null | ||
| }, | ||
| { | ||
| "id": "kpop-04", | ||
| "prompt": "Score exactly the accounts in ML.EVAL.EVAL_ACCOUNTS (ACCOUNT_ID, ANCHOR_TS, CHURNED) with my fitted Kumo trainer, so I can compare against CHURNED afterwards.", | ||
| "expected_output": "The scoring population must not carry the target, so the table cannot be passed as is. Evaluation labels (CHURNED) stay separate and are joined to the scores afterwards. It proposes, for approval, a read-only connector.sql(...) source keeping only the entity and TIMESTAMP columns under their contract names, or asks for a table without the target, and scores with `kumo.PredictionTable.from_source_table(..., pq=pq, snapshot=snapshot, plan=trainer.training_table_generation_plan)` then `trainer.predict(population, snapshot=snapshot, ...)`, comparing output entity and anchor coverage with the population.", | ||
| "assertions": [ | ||
| "Says the scoring population must not contain the target column CHURNED", | ||
| "Keeps CHURNED separate for joining to the scores afterwards", | ||
| "Proposes a read-only connector.sql(...) source with the entity and timestamp columns, or asks for a table without the target", | ||
| "Uses kumo.PredictionTable.from_source_table with pq and snapshot", | ||
| "Says output rows or coverage will be compared with the population rather than assumed equal" | ||
| ], | ||
| "expected_skill": "kumo-custom-populations", | ||
| "expected_script": null | ||
| }, | ||
| { | ||
| "id": "kpop-05", | ||
| "prompt": "Split my pandas DataFrame into stratified train, validation and test sets (70/15/15) with scikit-learn.", | ||
| "expected_output": "A scikit-learn data-splitting task unrelated to Kumo. The agent answers with train_test_split, without loading the Kumo custom-populations skill.", | ||
| "assertions": [ | ||
| "Does not load or follow the kumo-custom-populations skill", | ||
| "Uses scikit-learn train_test_split with stratification to produce 70/15/15 splits" | ||
| ], | ||
| "expected_skill": null, | ||
| "expected_script": null | ||
| } | ||
| ] | ||
| } |
45 changes: 45 additions & 0 deletions
45
agent-skills/nvskills/kumo-custom-populations/skill-card.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| ## Description: <br> | ||
| Train with caller-supplied labels, weights and splits or score exact entity/anchor rows through Snowflake source tables using the local Kumo API. <br> | ||
|
|
||
| This skill is ready for commercial/non-commercial use. <br> | ||
|
|
||
| ## Owner | ||
| NVIDIA <br> | ||
|
|
||
| ### License/Terms of Use: <br> | ||
| Apache 2.0 <br> | ||
| ## Use Case: <br> | ||
| Developers and data scientists use this skill with coding agents when generated Kumo training or prediction tables do not represent the rows they need, for example curated labels, manual splits, sample weights or an exact scoring population. <br> | ||
|
|
||
| ### Deployment Geography for Use: <br> | ||
| Global <br> | ||
|
|
||
| ## Requirements / Dependencies: <br> | ||
| **Requires API Key or External Credential:** [Yes] <br> | ||
| **Credential Type(s):** [Other [Snowflake connection or Snowpark session of the user's notebook]] <br> | ||
|
|
||
| Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. See skill body for more details. <br> | ||
|
|
||
| ## Known Risks and Mitigations: <br> | ||
| Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. <br> | ||
| Mitigation: Review and scan skill before deployment. <br> | ||
|
|
||
| ## Reference(s): <br> | ||
| - [references/populations.md](https://github.com/NVIDIA/kumo-engine/blob/main/skills/kumo-custom-populations/references/populations.md) <br> | ||
| - [NVIDIA/kumo-engine](https://github.com/NVIDIA/kumo-engine) <br> | ||
|
|
||
|
|
||
| ## Skill Output: <br> | ||
| **Output Type(s):** [Code, Analysis] <br> | ||
| **Output Format:** [Markdown with inline Python and SQL code blocks] <br> | ||
| **Output Parameters:** [1D] <br> | ||
| **Other Properties Related to Output:** [Reads source tables only through read-only queries; never writes back to them.] <br> | ||
|
|
||
| ## Skill Version(s): <br> | ||
| Unversioned; published from the main branch of NVIDIA/kumo-engine (source: repository; no skill version, CHANGELOG entry or release tag covers the skill files). The signed copy is identified by the skill.oms.sig file in this directory. <br> | ||
|
|
||
| ## Ethical Considerations: <br> | ||
| NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br> | ||
|
|
||
| (For Release on NVIDIA Platforms Only) <br> | ||
| Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail). <br> |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not true yet: agent-skills/nvskills/kumo-data-science/ (named on line 140) does not exist, NVIDIA/skills has no components.d/kumo-engine.yml, and NVSKILLS_CI_DISPATCH_TOKEN is unset. List what must exist before the first publish here (skill-card.md, evals/evals.json, the secret, the onboarding PR, a PASS BENCHMARK.md) instead of this sentence.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 13c6910: the sentence is replaced by a "Before the first publish" checklist:
skill-card.md,evals/evals.json, theNVSKILLS_CI_DISPATCH_TOKENsecret, a signed copy onmainwith a PASSBENCHMARK.md, and the onboarding PR addingcomponents.d/kumo-engine.yml.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Confirmed at 2105a3a: the checklist at README.md:168 lists the card and evals per skill, the secret, the signed copy and the onboarding PR.