Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 39 additions & 24 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,35 +1,50 @@
name: CI
on:
pull_request:
branches: [main]
push:
branches: ['**']

branches: [main]
permissions:
contents: read
packages: read

concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
build:
node:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
node: ["20.19.0", "22.23.1", "24.18.0"]
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: ${{ matrix.node }}
- run: npm install --no-audit --no-fund --ignore-scripts

Check warning on line 20 in .github/workflows/ci.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Using dependencies without locking resolved versions is security-sensitive.

See more on https://sonarcloud.io/project/issues?id=Quantum-L9_LLM-Router&issues=AZ-HvhDzcjTW3YD8VZcN&open=AZ-HvhDzcjTW3YD8VZcN&pullRequest=12
- run: npm run build
- run: npm run verify:types
- run: npm run lint
- run: npm run lint:boundary
- run: npm test
- run: npm audit --audit-level=high --omit=dev
- run: npm run verify:package
- name: Pack artifact
if: matrix.node == '20.19.0'
run: |
npm pack --ignore-scripts
sha256sum quantum-l9-llm-router-*.tgz > package.sha256
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: matrix.node == '20.19.0'
with:
persist-credentials: false
- uses: actions/setup-node@v4
name: package-contract
path: |
quantum-l9-llm-router-*.tgz
package.sha256
if-no-files-found: error
artifact-roundtrip:
needs: node
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4
with:
node-version: '20'
cache: 'npm'
registry-url: 'https://npm.pkg.github.com'
scope: '@quantum-l9'
- run: npm ci
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Build (tsc -> dist/)
run: npm run build
- name: Typecheck
run: npm run verify:types
name: package-contract
path: artifacts/downloaded
- run: cd artifacts/downloaded && sha256sum -c package.sha256
53 changes: 9 additions & 44 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -1,56 +1,21 @@
name: Publish Package
name: Publish
on:
push:
branches: [main]
paths: ['package.json']
workflow_dispatch:

tags: ['v*']
permissions:
contents: read
packages: write

concurrency:
group: publish-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false

jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
persist-credentials: false
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
registry-url: 'https://npm.pkg.github.com'
scope: '@quantum-l9'
- run: npm ci
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- run: npm run build

# Gate publish on a version bump: npm refuses to republish an existing
# name@version, so manual dispatch or any non-bump push to package.json
# would otherwise fail the job. Skip cleanly when this version already
# exists on the registry.
- name: Check whether this version is already published
id: published
run: |
VERSION="$(node -p "require('./package.json').version")"
if npm view "@quantum-l9/llm-router@${VERSION}" version >/dev/null 2>&1; then
echo "exists=true" >> "$GITHUB_OUTPUT"
echo "@quantum-l9/llm-router@${VERSION} is already published — skipping publish."
else
echo "exists=false" >> "$GITHUB_OUTPUT"
echo "@quantum-l9/llm-router@${VERSION} not found — will publish."
fi
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Publish
if: steps.published.outputs.exists == 'false'
run: npm publish
node-version: 24.18.0
registry-url: https://npm.pkg.github.com
- run: npm install --no-audit --no-fund --ignore-scripts

Check warning on line 17 in .github/workflows/publish.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Using dependencies without locking resolved versions is security-sensitive.

See more on https://sonarcloud.io/project/issues?id=Quantum-L9_LLM-Router&issues=AZ-HvhEDcjTW3YD8VZcO&open=AZ-HvhEDcjTW3YD8VZcO&pullRequest=12
- run: npm run verify:all
- run: npm publish
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
25 changes: 25 additions & 0 deletions .github/workflows/supply-chain.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Supply Chain
on:
pull_request:
push:
branches: [main]
permissions:
contents: read
jobs:
sbom:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 24.18.0
- run: npm install --no-audit --no-fund --ignore-scripts

Check warning on line 16 in .github/workflows/supply-chain.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Using dependencies without locking resolved versions is security-sensitive.

See more on https://sonarcloud.io/project/issues?id=Quantum-L9_LLM-Router&issues=AZ-HvhDccjTW3YD8VZcM&open=AZ-HvhDccjTW3YD8VZcM&pullRequest=12
- run: npm sbom --sbom-format cyclonedx > sbom.cdx.json
- run: sha256sum sbom.cdx.json > sbom.sha256
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: sbom
path: |
sbom.cdx.json
sbom.sha256
if-no-files-found: error
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
node_modules/
dist/
*.log
*.tgz
artifacts/
coverage/
25 changes: 25 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Architecture

`@quantum-l9/llm-router` is a reusable TypeScript routing library. The root `L9LLMRouter` is the supported execution surface.

## Runtime flow

```text
validated TaskDescriptor
-> pure route resolution
-> atomic process-local budget reservation
-> provider-family-safe downgrade
-> per-provider circuit permit
-> provider dispatch
-> cost reconciliation and audit log
```

Provider clients live under `src/providers/`. Production modules outside `src/index.ts` and `src/providers/` may not import them. Existing provider subpath exports remain available during the 1.x line for compatibility, but direct use is deprecated because it bypasses budget and circuit controls.

## State scope

The built-in budget tracker and circuit breaker are process-local. They prevent concurrent overspend and provider stampedes inside one process. Distributed enforcement requires an external persistence adapter and is intentionally not claimed here.

## Runtime floor

The package runtime floor is Node 20.19.0. Type definitions track the Node 20 line so compilation cannot silently authorize APIs that are unavailable to supported consumers. CI also exercises the current active LTS as a secondary runtime.
181 changes: 9 additions & 172 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,183 +1,20 @@
# @quantum-l9/llm-router

> The shared intelligence routing layer for all L9 bots. One module, all models, zero waste.
Shared deterministic model routing, budget enforcement, provider resilience, and visual QA for L9 bots.

## What This Is

A standalone, reusable TypeScript module that any L9 bot imports to get optimal model selection, budget enforcement, and multi-provider routing — without each bot implementing its own LLM integration.

```typescript
import { L9LLMRouter, TaskType, TaskComplexity } from '@quantum-l9/llm-router';
```ts
import { L9LLMRouter, TaskComplexity, TaskType } from '@quantum-l9/llm-router';

const router = new L9LLMRouter({
perplexityApiKey: process.env.PERPLEXITY_API_KEY!,
openrouterApiKey: process.env.OPENROUTER_API_KEY!,
appName: 'L9-SEO-Bot',
});

router.initClient('safehavenrr', { monthlyBudgetPerClient: 200 });

const response = await router.execute(
{
clientId: 'safehavenrr',
type: TaskType.CONTENT_GENERATION,
complexity: TaskComplexity.MEDIUM,
description: 'Write blog post about roof repair costs in Houston',
},
'You are an expert roofing content writer...',
'Write a 1200-word blog post about...',
router.initClient('tenant-a', { monthlyBudgetPerClient: 200 });
const result = await router.execute(
{ clientId: 'tenant-a', type: TaskType.CONTENT_GENERATION, complexity: TaskComplexity.MEDIUM },
'You are a careful writer.',
'Draft the article.',
);
```

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│ L9LLMRouter.execute() │
├─────────────────────────────────────────────────────────────┤
│ 1. Classify task → TaskType × TaskComplexity │
│ 2. Check budget → BudgetTracker.evaluateTask() │
│ 3. Route to matrix: │
│ ├── Search tasks → Perplexity Matrix → Sonar models │
│ ├── Vision tasks → Vision Matrix → GPT-4o/Claude/Gemini │
│ └── General tasks → General Matrix → Best model per job │
│ 4. Execute via provider client │
│ 5. Record spend + log routing decision │
└─────────────────────────────────────────────────────────────┘
```

## Three Matrices

### 1. Perplexity Matrix (Search-Grounded Tasks)

Ported from the [Enrichment.Inference.Engine](https://github.com/cryptoxdog/Enrichment.Inference.Engine) search optimizer. Maps task complexity to Sonar model + search depth:

| Complexity | Model | Search Context | Cost/Call |
|---|---|---|---|
| Trivial/Low | `sonar` | `low` | ~$0.001 |
| Medium | `sonar-pro` | `medium` | ~$0.01 |
| High | `sonar-pro` | `high` | ~$0.03 |
| Critical | `sonar-deep-research` | `high` | ~$0.05 |

Includes consensus mode (multiple variations for high-stakes research).

### 2. General Matrix (All Other Tasks)

Maps `TaskType × TaskComplexity` to the optimal model across providers:

| Task Type | Low | Medium | High | Critical |
|---|---|---|---|---|
| Classification | GPT-4o-mini | GPT-4o-mini | GPT-4o | Claude Sonnet |
| Content Generation | Claude Haiku | Claude Sonnet | Claude Sonnet | Claude Opus |
| Strategic Reasoning | GPT-4o | Claude Sonnet | Claude Sonnet | O3 |
| Code Generation | Claude Haiku | Claude Sonnet | Claude Sonnet | O3 |
| Extraction | GPT-4o-mini | GPT-4o | GPT-4o | Claude Sonnet |

Each model has a 2-deep fallback chain for resilience.

### 3. Vision Matrix (Visual QA Tasks)

| Complexity | Model | Detail | Cost/Call |
|---|---|---|---|
| Low (quick check) | Gemini Flash Vision | `low` | ~$0.001 |
| Medium (layout) | GPT-4o Vision | `auto` | ~$0.015 |
| High (detailed) | GPT-4o Vision | `high` | ~$0.02 |
| Multi-image comparison | Claude Sonnet Vision | `high` | ~$0.03 |

## Budget Engine

No daily hard cap. Trajectory-based throttling with surge awareness:

- **Monthly budget**: $200/client (configurable per-client)
- **Weekly target**: $50/week soft target
- **Weekly ceiling**: $100/week hard safety net
- **Surge**: If week-to-date spend < 60% by Thursday, allow burst up to ceiling
- **Critical override**: CRITICAL tasks ALWAYS proceed regardless of budget
- **Downgrade, don't kill**: Under throttle, tasks get cheaper models instead of being blocked

## Vision QA (Site Visual Validation)

The router includes a full Visual QA system that lets bots "see" websites:

```typescript
// Generate QA plan for a site
const tasks = router.planVisualQA({
pages: ['https://safehavenrr.com', 'https://safehavenrr.com/services'],
viewports: [VIEWPORTS.desktop_1440, VIEWPORTS.mobile_iphone],
competitorUrl: 'https://competitor.com',
conversionAudit: true,
});

// Bot takes screenshots, then executes each task
for (const task of tasks) {
const result = await router.execute(
{ clientId: 'safehavenrr', type: TaskType.LAYOUT_VALIDATION, complexity: TaskComplexity.MEDIUM },
task.prompt,
'Analyze this screenshot',
{ images: [screenshotUrl] },
);
}
```

Cost: ~$0.40 per full site audit (5 pages × 3 viewports + competitor + conversion).

## Consuming This Module

### From L9 SEO Bot

```typescript
// In l9-seo-bot/package.json
"dependencies": {
"@quantum-l9/llm-router": "file:../l9-llm-router"
}
```

### From L9 Website Factory

```typescript
// In l9-website-factory/package.json
"dependencies": {
"@quantum-l9/llm-router": "file:../l9-llm-router"
}
```

### From Any Future Bot

Same pattern. Import, configure with API keys, call `execute()`.

## Environment Variables

```env
OPENROUTER_API_KEY=sk-or-v1-...
PERPLEXITY_API_KEY=pplx-...
```

That's it. Two API keys give you access to every model.

## File Structure

```
src/
├── index.ts # Main router + re-exports
├── types.ts # All types, enums, interfaces
├── matrices/
│ ├── perplexity-matrix.ts # Search task → Sonar model resolver
│ └── general-matrix.ts # General task → model resolver
├── vision/
│ └── index.ts # Visual QA engine + prompts
├── budget/
│ └── index.ts # Budget tracker + throttle engine
└── providers/
├── perplexity.ts # Perplexity API client
└── openrouter.ts # OpenRouter API client
```

## Design Principles

1. **Deterministic routing** — No LLM call needed to decide which LLM to call
2. **Budget-aware, not budget-killed** — Downgrade models under pressure, never block critical work
3. **Provider-agnostic** — Bots don't know or care which provider serves the response
4. **Surge-friendly** — Quiet weeks allow burst activity without throttling
5. **Consensus-capable** — High-stakes research runs multiple variations for reliability
6. **Vision-native** — Visual QA is a first-class capability, not an afterthought
7. **Portable** — Any L9 bot imports this module identically
Direct imports from `./openrouter` or `./perplexity` are retained for 1.x compatibility but are deprecated. They bypass router-level budget and circuit controls.
Loading
Loading