Skip to content

Stabilize smart routing integration coverage - #1222

Merged
sunishsheth2009 merged 1 commit into
mainfrom
sunish/aigtwy-4848-routing-ci
Oct 22, 2026
Merged

sunishsheth2009 merged 1 commit into
mainfrom
sunish/aigtwy-4848-routing-ci

Conversation

@sunishsheth2009

@sunishsheth2009 sunishsheth2009 commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Stabilizes smart-routing CI against the model menus currently published by the US and Canadian managed workspaces, and makes rejected routing decisions diagnosable without changing the hook protocol.

  • uses explicit checked-in Codex fixtures for the US GPT-6 and Canada GPT-5.6 menus
  • allows another managed workspace to select a validated checked-in fixture through UCODE_TEST_CODEX_ROUTING_FIXTURE; the suite never guesses a menu or changes workspaces
  • reports a rejected routing decision on stderr while preserving the empty stdout hook response
  • waits for the child’s own persisted answer instead of accepting only a parent-reported result
  • correlates Codex routing notices with the tagged prompt, or with its immediately adjacent native Spawned row after compaction
  • retains audit correlation, model-use evidence, and completed parent/child task assertions

This is independent from the MCP/Skills browser work in #1211. The branch and diff contain only routing diagnostics, fixtures, and routing evidence/tests.

Before and after

When the gateway rejects a routing request, the hook contract remains unchanged: no stdout response is emitted. The difference is an actionable stderr diagnostic.

Before

stdout: <empty>
stderr: <empty>

After

stdout: <empty>
stderr: Smart routing did not select a model: <gateway decision reason>

Unit coverage verifies that the diagnostic does not contain the bearer token or task prompt.

Why the fixtures differ

The US task_v3 policy requires GPT-6 Sol/Luna. The Canadian managed workspace returns 404 for GPT-6 and publishes GPT-5.6 Sol/Terra/Luna. A single common menu therefore cannot exercise both real workspaces. Claude’s Opus 4.8 / Sonnet 4.6 / Haiku 4.5 fixture works in both.

The mapping is explicit and test-only. Unknown workspaces fail with instructions to set UCODE_TEST_CODEX_ROUTING_FIXTURE; there is no fallback, retry, or automatic workspace selection.

Validation

  • 138 passed: Codex routing, integration evidence, and integration contract tests
  • Ruff check and format check pass for every changed Python file
  • The cumulative routing revision previously passed all 52 checks, including Linux/Windows unit tests and full Claude/Codex journeys: run 39510850961
  • A fresh routing-only CI run is executing after restoring this PR’s routing-only head

No skips, xfails, retries in product/test code, or weakened child/audit assertions were added.

sunishsheth2009 added a commit that referenced this pull request Oct 21, 2026
Consumers cannot compare MCP services in `ug mcp add` because discovery
drops their descriptions and audit metadata. `ug skills add` only
previews a short description. Both pickers now show a compact Select /
Name / UC location / Status table and a shared, scrollable detail pane.

Fixes https://databricks.atlassian.net/browse/AIGTWY-4848

Reviewed the ticket description, all three Jira comments, and all 47
replies in the linked [Slack
thread](https://databricks.slack.com/archives/C09V48HFPDW/p1790619709443169?thread_ts=1789543972.420209&cid=C09V48HFPDW)
before implementation. One PR covers MCPs and Skills because they share
the picker and metadata presentation.

This browser-only PR is stacked on #1222, which contains the independent
smart-routing CI repairs requested during review. After #1222 lands,
this PR can be retargeted to `main` without changing its browser diff.

- Existing paginated MCP `BASIC` responses supply metadata for initial
and streamed choices; there are no per-service requests.
- `[ ]` / `[x]` represent selection. ✓ / ✗ represent current status:
Configured, Not configured, Managed, Installed, or Not installed.
- Skills show ✓ Installed only when the exact UC FQN, bundle,
metastore/workspace attribution, every target root, and `SKILL.md` all
match.
- Details use `Full name` rather than the command-line `location` term,
and dates render as compact UTC calendar dates.
- Wide terminals keep the columns left-aligned in a compact 96-column
table; narrow terminals retain the status glyph and full details.
- Page Up/Down changes the visible detail content on the first key
press.
- A streamed rich MCP row replaces a matching plain configured row in
place, preserving its value-based selection while adding its full name
and details.
- MCP filtering uses the full dotted name without a common `MCP:`
prefix, so typing `mcp` no longer matches every row.
- No dependencies or lockfiles changed.

### Before and after

These are rendered component captures using identical synthetic
metadata; they are not live-workspace results. Selection returns the
same values before and after.

**MCPs — before**

```text
 MCP: (space to toggle, ctrl-a all, enter to save, type to filter)
 › ○ MCP: system.ai.docs
```

**MCPs — after**

```text
 MCP: (space toggle, ^A all, enter save, type filter, pgup/pgdn details)
  Select Name                       UC location                 Status
› [ ]    docs                       system.ai                   ✗ Not configured
  [-]    tickets                    system.ai                   ✓ Configured
  [-]    managed                    system.ai                   ✓ Managed

  Full name: system.ai.docs
  Description: Search internal documentation and return relevant pages.
  Created by: team@example.com
  Created: 2026-09-01
  Updated: 2026-10-15
  Status: ✗ Not configured
```

**Skills — before**

```text
 Skills: (space to toggle, ctrl-a all, enter to save, type to filter)
 › ○ demo.team.triage (on disk)
   ○ demo.team.review

  triage: Summarize a ticket, identify the owner, and recommend next steps.
```

**Skills — after**

```text
 Skills: (space toggle, ^A all, enter save, type filter, pgup/pgdn details)
  Select Name                       UC location                 Status
› [ ]    triage                     demo.team                   ✓ Installed
  [ ]    review                     demo.team                   ✗ Not installed

  Full name: demo.team.triage
  Bundle: triage
  Description: Summarize a ticket, identify the owner, and recommend next steps.
  Created by: team@example.com
  Created: 2026-09-01
  Updated: 2026-10-15
  Status: ✓ Installed
```

### Validation

All local commands used a disposable HOME.

- Review follow-up: **135 passed** across the shared picker, catalog
browser, Skills download, and managed-row behavior.
- `ruff check src/ tests/`: pass.
- `ruff format --check src/ tests/`: 253 files formatted.
- `ty check src/`: pass.
- Before the split, the cumulative browser + routing revision passed all
**52** GitHub checks, including Linux/Windows unit tests and full
Claude/Codex journeys: [run
39510850961](https://github.com/databricks/unity-gateway/actions/runs/39510850961).
- Earlier broad local comparison (excluding a baseline-stalling auth
test): main 4,907 passed / 21 failed / 9 errors / 40 skipped; browser
revision 4,927 passed / 21 failed / 9 errors / 40 skipped. Failure/error
identifiers were identical, with zero new failures.

No skips or xfails were added.
@sunishsheth2009
sunishsheth2009 force-pushed the sunish/aigtwy-4848-routing-ci branch from ef64366 to dd834bf Compare October 21, 2026 18:30
@sunishsheth2009
sunishsheth2009 added this pull request to the merge queue Oct 22, 2026
Merged via the queue into main with commit 081b7c0 Oct 22, 2026
255 of 269 checks passed
@sunishsheth2009
sunishsheth2009 deleted the sunish/aigtwy-4848-routing-ci branch October 22, 2026 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants