diff --git a/.github/ISSUE_TEMPLATE/migration-feedback.md b/.github/ISSUE_TEMPLATE/migration-feedback.md new file mode 100644 index 0000000..3c9817c --- /dev/null +++ b/.github/ISSUE_TEMPLATE/migration-feedback.md @@ -0,0 +1,82 @@ +--- +name: Migration Skill Feedback +about: Report a bug, suggest a pattern, or submit a failure report for the serverless migration skill +title: "[migration-skill] " +labels: migration-skill +assignees: '' +--- + + + +## Category + + + +- [ ] Pattern not detected (the skill missed a classic-compute construct) +- [ ] Wrong fix suggested (the fix didn't work or was incorrect) +- [ ] Migration succeeded but output differs (code runs, data is different) +- [ ] Documentation unclear +- [ ] Failure report (attach JSON from `~/.databricks-migration-skill/reports/`) +- [ ] New pattern suggestion + +## Pre-submission checklist + +- [ ] I have removed all customer code and replaced with a minimal reproducible example +- [ ] I have removed all credentials, tokens, and secret scope names +- [ ] I have removed all workspace URLs, catalog names, and user emails +- [ ] I have removed any customer-identifying information + +## Environment + +- Skill version: +- Agent: +- Model: +- Databricks Runtime of source workload: + +## Description + + + +## Minimal reproducible example + + +```python +# Classic compute code (minimal example, no customer data) +``` + +## Expected migration + + + +## Actual migration + + + +## Failure report (optional) + + + +
+Failure report JSON + +```json + +``` + +
+ +## Additional context + + diff --git a/README.md b/README.md index 5e8467f..c3b7190 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,7 @@ Run this command in chat: ## Available Skills - **databricks-apps** - Build full-stack TypeScript apps on Databricks using AppKit +- **databricks-serverless-migration** - Migrate classic-compute workloads (notebooks, jobs, pipelines) to serverless compute. Detects compatibility blockers, provides concrete fixes, and produces an anonymized failure report you can file as a GitHub issue when migration cannot complete. Also installable inside a Databricks workspace via Genie Code Agent mode — see [the install reference](skills/databricks-serverless-migration/references/install-in-databricks-genie-code.md). ## Structure diff --git a/manifest.json b/manifest.json index 54ec72f..af17620 100644 --- a/manifest.json +++ b/manifest.json @@ -1,6 +1,6 @@ { "version": "2", - "updated_at": "2026-04-30T11:02:41Z", + "updated_at": "2026-05-12T07:54:37Z", "skills": { "databricks-apps": { "version": "0.1.1", @@ -152,7 +152,7 @@ "version": "0.1.0", "description": "Migrate Databricks workloads from classic compute to serverless compute, including compatibility checks and concrete fixes", "experimental": false, - "updated_at": "2026-04-24T15:10:23Z", + "updated_at": "2026-05-12T07:54:37Z", "files": [ "SKILL.md", "agents/openai.yaml", @@ -161,6 +161,8 @@ "references/code-patterns.md", "references/compatibility-checks.md", "references/configuration-guide.md", + "references/failure-reporting.md", + "references/install-in-databricks-genie-code.md", "references/networking-and-security.md", "references/streaming-migration.md" ] diff --git a/skills/databricks-serverless-migration/SKILL.md b/skills/databricks-serverless-migration/SKILL.md index 859d01a..ae86ede 100644 --- a/skills/databricks-serverless-migration/SKILL.md +++ b/skills/databricks-serverless-migration/SKILL.md @@ -21,6 +21,15 @@ Analyze existing Databricks code for serverless compute compatibility and guide - Troubleshooting serverless-specific errors after migration - Choosing between Performance-Optimized and Standard mode +## Where to Run This Skill + +This skill is published as an Agent Skill (agentskills.io) and runs in any compatible client: + +- **Claude Code, Cursor, or any agentskills.io client on your laptop** — the default. Install via `databricks experimental aitools install` or follow the per-client docs. +- **Inside a Databricks workspace via Genie Code Agent mode** — drop the skill into `/Workspace/Users//.assistant/skills/databricks-serverless-migration/` (per-user) or `/Workspace/.assistant/skills/databricks-serverless-migration/` (workspace-wide, admin only). See [Install in Databricks Genie Code](references/install-in-databricks-genie-code.md) for the three install methods and the **important serverless-compute caveat** (the Databricks CLI isn't pre-installed on serverless, so some deploy steps need adjustment). + +If you finish a migration analysis for a user who's currently running you from a laptop client, mention the Genie Code option once at the end — many users prefer iterating on migrations inside the workspace where the workload lives. + ## Understanding Migration Blockers Migration blockers fall into three categories. Focus your effort on category 2 — that's where this skill helps most. @@ -545,21 +554,28 @@ See [Configuration Guide](references/configuration-guide.md) for the full error ## Failure Reporting Protocol -When migration fails irrecoverably, generate a structured failure report to help improve the skill. This applies when: +When migration fails or hits an unmigratable pattern, generate a structured failure report and offer the user a one-step path to file it as a GitHub issue. This feedback loop is how the skill learns — without it, gaps go undiscovered. + +See [Failure Reporting](references/failure-reporting.md) for the full redaction checklist, URL-encoding recipe, and example pre-filled link. + +### Decision tree: when to offer to file an issue -- All retry attempts are exhausted (typically 5) -- An unknown pattern is encountered that isn't in the compatibility checks -- A fix was applied but didn't resolve the underlying issue -- The workload hits a Category 3 blocker the user wasn't aware of +**Offer to file a GitHub issue any time the workload cannot be fully migrated to serverless as-is.** Reports from "known" patterns (R, Scala, custom JAR data sources, JVM access, third-party connectors) are just as valuable as reports from unknown patterns — they tell maintainers which gaps users hit most often, which drives prioritization. -### When to generate a report +Concretely, ALWAYS offer if **any** of these is true: -Generate a report at the end of a migration attempt if **any** of: -- `retry_count >= max_retries` and final status is FAILED -- A pattern was detected but no fix is available in the skill -- The user explicitly requests a failure report (`/migration-report`) +1. **Workload contains any Category 3 (classic-only) blocker** — R, Scala notebook cells, custom JAR data sources, JVM/Py4J access, third-party connectors without serverless equivalents, native binary dependencies, etc. The fact that the pattern is "documented as Cat 3" is not a reason to skip the offer. +2. **Retries exhausted** — `retry_count >= max_retries` (typically 5) and final status is FAILED +3. **Unknown pattern** — a classic-compute construct was detected that isn't in the skill's catalog +4. **Fix didn't resolve** — a known fix was applied but the workload still fails on serverless +5. **Explicit request** — the user invokes `/migration-report` -### How to generate +**Do NOT offer to file** only when: + +- The migration succeeded fully (even after retries), or +- The workload is already serverless-compatible and required no changes. + +### How to generate the report Write a JSON file to `~/.databricks-migration-skill/reports/failure-.json`. Create the directory if it doesn't exist. @@ -567,7 +583,7 @@ Write a JSON file to `~/.databricks-migration-skill/reports/failure-", "skill_version": "", "timestamp": "", @@ -578,7 +594,7 @@ Write a JSON file to `~/.databricks-migration-skill/reports/failure-", "attempt_number": 1, "outcome": "failed"} ], - "final_error_category": "unknown_api | missing_library | data_access | permission | custom_data_source | other", + "final_error_category": "unknown_api | missing_library | data_access | permission | custom_data_source | jvm_access | unsupported_language | other", "final_error_signature": "", "retry_count": 5, "total_duration_seconds": 245, @@ -594,34 +610,85 @@ Write a JSON file to `~/.databricks-migration-skill/reports/failure-"` | -### After generating the report +The `notebook_characteristics` fields are the only safe surface for workload metadata. Do not add new fields without expanding this checklist. -Tell the user: +### After generating the report — REQUIRED output template + +**This is not optional.** When the decision tree above says "offer to file", you MUST produce all three of these in your final response to the user, in order: + +1. The local report file path (literally include the string `~/.databricks-migration-skill/reports/failure-.json`, with `` filled in). +2. **Option A** — a complete pre-filled `https://github.com/databricks/databricks-agent-skills/issues/new?template=migration-feedback.md&title=<…>&body=<…>` URL, both parameters URL-encoded. +3. **Option B** — the literal `gh issue create --repo databricks/databricks-agent-skills …` command, body-file pointing at the local report. + +If you write the JSON inline in your response but skip the URL and the `gh` command, the protocol is **not satisfied**. The whole point is to make filing one click. + +Use this exact wrap-up template, replacing `<…>` placeholders: ``` -Migration failed after attempts. A failure report has been generated at: +Migration could not complete. A failure report has been generated at: ~/.databricks-migration-skill/reports/failure-.json -This report contains anonymized diagnostic data (detected patterns, error categories, retry count) and no code content or PII. You can: +The report contains anonymized diagnostic data (detected pattern IDs, error +category, retry count, notebook characteristics) and no code content or PII. +Submission is optional and opt-in. + +To help improve this skill, file the report as a GitHub issue: -1. Review the JSON to confirm no sensitive information is present -2. Share it via GitHub Issue to help improve the skill: - https://github.com/databricks/databricks-agent-skills/issues/new?template=migration-feedback.md + Option A — One-click in browser (pre-filled): + -Submission is optional and opt-in. We use reports to prioritize new patterns and fix detection gaps. + Option B — From the terminal (if you have the GitHub CLI installed): + gh issue create \ + --repo databricks/databricks-agent-skills \ + --title "" \ + --body-file ~/.databricks-migration-skill/reports/failure-<timestamp>.json \ + --label migration-skill + +Before submitting, please open the JSON and confirm nothing sensitive +slipped through. We never transmit reports automatically. ``` -**Never transmit the report automatically.** The user owns their data and must review before sharing. +Build `<PREFILLED_ISSUE_URL>` like this: + +1. **Title**: `[migration-skill] <final_error_category> in <failure_phase> phase` + Example: `[migration-skill] custom_data_source in migrate phase` +2. **Body**: the issue template's markdown skeleton (Category, Environment, Description, Failure report JSON fenced in ` ```json `) with the report JSON inlined. +3. **URL-encode** both title and body (`%20` for spaces, `%23` for `#`, `%0A` for newline, etc.). +4. **Final URL**: + `https://github.com/databricks/databricks-agent-skills/issues/new?template=migration-feedback.md&title=<URL-encoded title>&body=<URL-encoded body>` + +If your runtime cannot actually write the file (sandboxed, no filesystem write), still show the path the file WOULD be at and produce Options A and B. The user can write the JSON to disk themselves. + +The full recipe with a worked example is in [Failure Reporting](references/failure-reporting.md). + +**Never transmit the report automatically.** The user owns their data and must review before sharing. If the user declines, do not press them — log the local report path and move on. ## Reference Guides @@ -632,6 +699,8 @@ For detailed workarounds and code examples beyond the quick fixes above: - [Networking and Security](references/networking-and-security.md) — VPC peering to NCCs, Private Link, firewall setup - [Code Patterns](references/code-patterns.md) — Complete before/after code examples for every migration pattern - [Configuration Guide](references/configuration-guide.md) — Supported Spark configs, Environments setup, budget policies +- [Failure Reporting](references/failure-reporting.md) — Redaction checklist + pre-filled GitHub issue URL recipe (for when migration cannot complete) +- [Install in Databricks Genie Code](references/install-in-databricks-genie-code.md) — Run this skill inside a Databricks workspace ## Documentation diff --git a/skills/databricks-serverless-migration/references/failure-reporting.md b/skills/databricks-serverless-migration/references/failure-reporting.md new file mode 100644 index 0000000..11c7038 --- /dev/null +++ b/skills/databricks-serverless-migration/references/failure-reporting.md @@ -0,0 +1,162 @@ +# Failure Reporting — Filing a Migration Skill Issue + +Reference for the [Failure Reporting Protocol](../SKILL.md#failure-reporting-protocol) in `SKILL.md`. Read this when migration could not complete and you need to file a GitHub issue with anonymized context. + +## Why this exists + +The skill detects ~40 patterns across 7 categories today. Every new pattern in the wild that the skill doesn't recognize, every fix that didn't work, every Cat 3 blocker surfaced late — these are the inputs that close detection gaps. Reports are opt-in and never auto-submitted; the user owns the data. + +## Redaction checklist (apply before writing the JSON) + +Walk every string-typed field in the report and confirm none of these appear. If anything matches, drop the field rather than partially redacting. + +- **Identifiers**: emails, employee names, Slack user/channel IDs (`U…`, `C…`), customer / company names, account IDs, workspace IDs. +- **Paths and URLs**: `dbfs:/`, `/dbfs/`, `s3://`, `abfss://`, `gs://`, `wasbs://`, notebook paths, workspace URLs (`*.cloud.databricks.com`, `adb-*.azuredatabricks.net`), git remote URLs. +- **Internal references**: `go/` links, internal codenames not in public docs, Confluence page IDs, Google Doc IDs (`1[A-Za-z0-9_-]{20,}`), Slack channel names, PROD-* / SEV-* / SC-* tickets. +- **Catalog / schema / table / column names** from the analyzed notebook. Pattern IDs from this skill's catalog are fine; literals from the workload are not. +- **Credentials**: tokens, API keys, connection strings, JDBC URLs, service principal IDs, secret scope names. +- **Stack frame contents**: hash the top 3 frames with SHA-256, never include the frames themselves. +- **Error messages**: store only the `final_error_category` enum, never the raw error text. + +If a field is ambiguous, drop it. `notebook_characteristics` is the safe surface for workload metadata — do not invent new fields here without first updating this checklist and the schema in `SKILL.md`. + +## Building the pre-filled GitHub issue URL + +GitHub's `issues/new` endpoint accepts URL-encoded `title=` and `body=` query parameters. Combined with the `template=migration-feedback.md` parameter, you can produce a one-click link that drops the user straight into a pre-filled issue. + +### Title format + +``` +[migration-skill] <final_error_category> in <failure_phase> phase +``` + +Examples: +- `[migration-skill] custom_data_source in migrate phase` +- `[migration-skill] unknown_api in analyze phase` +- `[migration-skill] jvm_access in test phase` + +### Body skeleton + +Use this Markdown template. Replace the placeholders, then URL-encode the whole thing. + +```markdown +## Category + +- [x] Failure report (see JSON below) + +## Pre-submission checklist + +- [x] I reviewed the JSON below and confirmed no PII slipped through + +## Environment + +- Skill version: <skill_version from report> +- Agent: Claude Code / Cursor / other +- Databricks Runtime of source workload: <databricks_runtime_source from report> + +## Description + +Migration failed in the `<failure_phase>` phase. Final error category: +`<final_error_category>`. Retry count: <retry_count>. + +## Failure report JSON + +<details> +<summary>failure-<timestamp>.json</summary> + +```json +<paste the full report JSON here> +``` + +</details> +``` + +### URL-encoding + +Encode the title and body with standard `application/x-www-form-urlencoded` rules (space → `%20`, newline → `%0A`, `#` → `%23`, `<` → `%3C`, `>` → `%3E`, backtick → `%60`, `[` → `%5B`, `]` → `%5D`, etc.). In Python: + +```python +import urllib.parse +url = ( + "https://github.com/databricks/databricks-agent-skills/issues/new" + "?template=migration-feedback.md" + f"&title={urllib.parse.quote(title)}" + f"&body={urllib.parse.quote(body)}" +) +``` + +GitHub accepts URLs up to ~8 KB. The failure report JSON is well under 2 KB after redaction, so the encoded URL fits. + +### Worked example + +Title: `[migration-skill] custom_data_source in migrate phase` + +Body (abridged): + +```markdown +## Category +- [x] Failure report (see JSON below) +## Environment +- Skill version: 0.1.0 +- Agent: Claude Code +- Databricks Runtime: 14.3.x-scala2.12 +## Description +Migration failed in the `migrate` phase. Final error category: `custom_data_source`. Retry count: 5. +## Failure report JSON +<details><summary>failure-2026-05-12T08-00-00Z.json</summary> + +```json +{ + "report_version": "1.1", + "report_id": "5b7c8e8e-...", + "skill_version": "0.1.0", + "failure_phase": "migrate", + "detected_patterns": [{"category": "E", "pattern_id": "custom_jar_datasource", "count": 1}], + "attempted_fixes": [], + "final_error_category": "custom_data_source", + "final_error_signature": "a91b5e...", + "retry_count": 5, + "notebook_characteristics": {"lines_of_code": 73, "language": "python", "uses_streaming": false, "uses_ml_libraries": false, "databricks_runtime_source": "14.3.x-scala2.12"} +} +\`\`\` + +</details> +``` + +Encoded URL (truncated for readability): + +``` +https://github.com/databricks/databricks-agent-skills/issues/new?template=migration-feedback.md + &title=%5Bmigration-skill%5D%20custom_data_source%20in%20migrate%20phase + &body=%23%23%20Category%0A-%20%5Bx%5D%20Failure%20report... +``` + +## CLI alternative (`gh`) + +If the user has the GitHub CLI on their PATH (`which gh`), offer: + +```bash +gh issue create \ + --repo databricks/databricks-agent-skills \ + --title "[migration-skill] <final_error_category> in <failure_phase> phase" \ + --body-file ~/.databricks-migration-skill/reports/failure-<timestamp>.json \ + --label migration-skill +``` + +This works but skips the issue template's checklist sections. Prefer the browser URL when the user is unfamiliar with the contribution flow. + +## What we do with reports + +Reports are triaged by the skill maintainers (`@lennartkats-db @simonfaltum` plus the migration-skill code owners) and used to: + +1. Prioritize new patterns to add to `references/compatibility-checks.md` +2. Identify fixes that don't work in practice and need correction +3. Spot Cat 3 blockers that need clearer up-front detection so users hit them in analyze rather than migrate + +Reports never leave the public GitHub issues thread. We do not aggregate them externally. + +## Troubleshooting + +- **The pre-filled URL is too long for the browser** — GitHub caps at ~8 KB. If the report is unusually large (many `attempted_fixes`), fall back to the `gh` CLI command, or open the issue manually and paste the JSON in. +- **The browser opens an empty issue** — the `template=` parameter requires the template file to exist in the repo's `.github/ISSUE_TEMPLATE/`. If it's missing, file with the title and body only; the maintainers will update the template. +- **The user wants to share more context** — that's fine, but ask them to add it as a follow-up comment, not in the initial report body. Initial body should be the anonymized JSON only. diff --git a/skills/databricks-serverless-migration/references/install-in-databricks-genie-code.md b/skills/databricks-serverless-migration/references/install-in-databricks-genie-code.md new file mode 100644 index 0000000..eb0e11d --- /dev/null +++ b/skills/databricks-serverless-migration/references/install-in-databricks-genie-code.md @@ -0,0 +1,87 @@ +# Install this skill in a Databricks workspace (Genie Code Agent mode) + +You can run this skill **inside Databricks** — not just from Claude Code or Cursor on your laptop. Databricks Genie Code Agent mode uses the same Agent Skills standard ([agentskills.io](https://agentskills.io/specification)) as Claude Code, so this skill is drop-in compatible. Once installed, the skill auto-loads in Genie Code Agent chats based on its `description` frontmatter, or you can invoke it explicitly. + +## Where skills live in a workspace + +Workspace files at one of two paths: + +| Scope | Path | Who can install | +|-------|------|-----------------| +| **Per-user** | `/Workspace/Users/<your-email>/.assistant/skills/<skill-name>/` | Any user | +| **Workspace-wide** | `/Workspace/.assistant/skills/<skill-name>/` | Workspace admin only | + +Genie Code automatically picks up skills from these paths the next time the user starts an Agent-mode chat — no manifest registration, no separate UI step. + +## Three install methods + +### Option 1 — Notebook installer (easiest for one user) + +The `databricks-solutions/ai-dev-kit` repo ships a notebook that pulls skills from GitHub and writes them to your `/Workspace/Users/<you>/.assistant/skills/` path. + +1. Import [`install_genie_code_skills.py`](https://github.com/databricks-solutions/ai-dev-kit/blob/main/databricks-skills/install_genie_code_skills.py) into your workspace. +2. Edit the constants at the top of the notebook: + - `REPO = "databricks/databricks-agent-skills"` + - `BRANCH = "main"` + - `SKILLS = ["databricks-serverless-migration"]` (plus any others you want) +3. Attach the notebook to any running cluster (classic or serverless, any size). +4. Run All. The installer uses `WorkspaceClient().workspace.import_()` to upload `SKILL.md` + every file listed in this skill's entry in `manifest.json`. +5. Open a Genie Code Agent chat. The skill loads on demand based on its `description`. + +### Option 2 — Shell installer (for repeat installs) + +From a local clone of `databricks-solutions/ai-dev-kit`: + +```bash +./databricks-skills/install_skills.sh --local --install-to-genie +``` + +Add `--profile <YOUR_PROFILE>` if your `databricks` CLI default isn't the target workspace. Requires the `databricks` CLI installed and authenticated locally. + +### Option 3 — Databricks App (workspace-wide) + +The `mcp-ai-dev-kit` app deploys once per workspace, pulls skills from a configured GitHub repo on every redeploy, and writes to the workspace-wide `/Workspace/.assistant/skills/` path so every user in the workspace sees them. Recommended if you're rolling this skill out to a team. + +This requires workspace admin. Ping `#india-gcc-fe-team` for the latest deploy instructions. + +## Format compatibility + +Zero conversion needed. Genie Code uses the same Agent Skills layout as Claude Code: + +``` +databricks-serverless-migration/ +├── SKILL.md # frontmatter + instructions (this skill) +├── agents/ # optional agent configs +├── assets/ # static assets +└── references/ # supplementary docs loaded on demand (you're reading one) +``` + +The installer copies the entire skill directory as listed in `manifest.json`. + +## Caveats specific to this skill + +⚠️ **Databricks CLI is not pre-installed on serverless compute.** This skill's frontmatter declares `compatibility: Requires databricks CLI (>= v0.292.0)` because several recommended commands shell out to `databricks bundle deploy`, `databricks jobs update`, and `databricks fs cp`. + +Inside Genie Code on serverless compute, those commands will fail with `databricks: command not found`. Three options to reconcile: + +1. **Run the Agent-mode chat against a classic cluster** that has the CLI installed (you can `%pip install databricks-cli` in a setup cell, or bake it into a cluster init script). +2. **Use the Databricks SDK equivalents** the skill mentions where available — `WorkspaceClient().jobs.update(...)`, `WorkspaceClient().workspace.import_(...)`. The skill's recommendations are CLI-flavored today; SDK calls work equivalently in a notebook context. +3. **Skip the deploy steps** entirely. The skill's value is the analyze + migrate phases — code transformation, blocker detection, and fix suggestions. Those work without the CLI. You can apply the migrated job/DABs changes by hand or from a separate environment. + +Other notes: + +- **Genie Code's underlying model is not publicly specified** and may not be Claude. The skill is authored against Claude Code defaults; behavior parity is not guaranteed. Smoke-test on a non-critical workload first. +- **Skills are knowledge + scripts, not arbitrary tools.** Genie Code skills can't make network calls outside the workspace boundary. The skill's GitHub-issue-filing flow ([Failure Reporting](failure-reporting.md)) still works because it generates a URL the user clicks rather than auto-submitting. +- **File-edit sandboxing**: Genie Code can write files in the workspace, so the migration outputs (migrated notebooks, updated job JSON, env spec YAML) land where you expect. + +## Pointers and follow-ups + +| Resource | Link | +|----------|------| +| Public docs: Extend Genie Code with agent skills | https://docs.databricks.com/aws/en/genie-code/skills | +| Public docs: Agent skills overview | https://docs.databricks.com/aws/en/agent-skills/ | +| Notebook + shell installer | https://github.com/databricks-solutions/ai-dev-kit | +| This skill's source | https://github.com/databricks/databricks-agent-skills/tree/main/skills/databricks-serverless-migration | +| Agent Skills spec | https://agentskills.io/specification | + +For Genie Code runtime questions, the internal Slack channel is `#eng-genie-code-team`. For the workspace-wide app deployment pattern, `#india-gcc-fe-team`.