Skip to content

feat(ai-gateway): automated recipe update #5615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kong-documentation-app wants to merge 1 commit into from
Closed

feat(ai-gateway): automated recipe update #5615

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 7 additions & 8 deletions app/_cookbooks/basic-llm-routing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Basic LLM Routing
description: Route requests to any supported LLM provider through {{site.ai_gateway_name}} with Consumer authentication and per-request model selection.
description: Route requests to any supported LLM provider through Kong AI Gateway with Consumer authentication and per-request model selection.

Check failure on line 3 in app/_cookbooks/basic-llm-routing.md

View workflow job for this annotation

GitHub Actions / Vale 

[vale] reported by reviewdog 🐶
[base.Kongterms] Use '{{site.ai_gateway_name}}' instead of 'Kong AI Gateway'.

Raw Output:
{"message": "[base.Kongterms] Use '{{site.ai_gateway_name}}' instead of 'Kong AI Gateway'.", "location": {"path": "app/_cookbooks/basic-llm-routing.md", "range": {"start": {"line": 3, "column": 67}}}, "severity": "ERROR"}
url: "/cookbooks/basic-llm-routing/"
content_type: cookbook
layout: cookbook
Expand All @@ -15,7 +15,6 @@
gateway: '3.14'
categories:
- llm
- llm-routing
featured: false
popular: false

Expand All @@ -37,7 +36,7 @@
skip_product: true
skip_tool: true
inline:
- title: "{{site.konnect_product_name}}"
- title: Kong Konnect
content: |
This tutorial uses {{site.konnect_product_name}}. You will provision a recipe-scoped Control Plane and local Data Plane via the [quickstart script](https://get.konghq.com/quickstart).

Expand All @@ -60,7 +59,7 @@
content: |
This tutorial uses [kongctl](/kongctl/) and [decK](/deck/) to manage Kong configuration.

1. Install **kongctl** from [developer.konghq.com/kongctl](/kongctl/).
1. Install **kongctl** from [developer.konghq.com/kongctl](https://developer.konghq.com/kongctl/).

Check failure on line 62 in app/_cookbooks/basic-llm-routing.md

View workflow job for this annotation

GitHub Actions / Vale 

[vale] reported by reviewdog 🐶
[base.Relativeurls] Use relative URLs for docs URLs. For example, use [Get Started](/gateway/latest/get-started/) instead of [Get Started](https://developer.konghq.com/gateway/3.3.x/get-started/). If a direct URL is necessary, use {{ site.links.web }} instead of `https://developer.konghq.com`.

Raw Output:
{"message": "[base.Relativeurls] Use relative URLs for docs URLs. For example, use [Get Started](/gateway/latest/get-started/) instead of [Get Started](https://developer.konghq.com/gateway/3.3.x/get-started/). If a direct URL is necessary, use {{ site.links.web }} instead of `https://developer.konghq.com`.", "location": {"path": "app/_cookbooks/basic-llm-routing.md", "range": {"start": {"line": 62, "column": 37}}}, "severity": "ERROR"}
1. Install **decK** version 1.43 or later from [docs.konghq.com/deck](https://docs.konghq.com/deck/).
1. Verify both are installed:

Expand Down Expand Up @@ -205,7 +204,7 @@
{% mermaid %}
sequenceDiagram
participant C as Client
participant K as {{site.ai_gateway_name}}
participant K as Kong AI Gateway
participant L as LLM Provider

C->>K: POST /basic-llm-routing (apikey, model: fast or smart)
Expand Down Expand Up @@ -265,7 +264,7 @@
```
{:.no-copy-code}

**`key_names: [apikey]`**. The headers (or query parameters) the Plugin looks in for the API key. The recipe uses `apikey` because the Key Auth Plugin performs an exact string match on the header value and does not inspect `Authorization` for Bearer tokens. The OpenAI SDK's `api_key` field always serializes as `Authorization: Bearer <key>`, which Kong would read as the literal string `Bearer <key>` and fail to match against any stored credential. The "Try it out" section below points at a pre-function pattern that bridges the SDK's Bearer token to the `apikey` header server-side; the [Authenticate OpenAI SDK clients with Key Auth](/how-to/authenticate-openai-sdk-clients-with-key-auth/) guide has the full pattern.
**`key_names: [apikey]`**. The headers (or query parameters) the Plugin looks in for the API key. The recipe uses `apikey` because the Key Auth Plugin performs an exact string match on the header value and does not inspect `Authorization` for Bearer tokens. The OpenAI SDK's `api_key` field always serializes as `Authorization: Bearer <key>`, which Kong would read as the literal string `Bearer <key>` and fail to match against any stored credential. The "Try it out" section below points at a pre-function pattern that bridges the SDK's Bearer token to the `apikey` header server-side; the [Authenticate OpenAI SDK clients with Key Auth](https://developer.konghq.com/how-to/authenticate-openai-sdk-clients-with-key-auth/) guide has the full pattern.

Check failure on line 267 in app/_cookbooks/basic-llm-routing.md

View workflow job for this annotation

GitHub Actions / Vale 

[vale] reported by reviewdog 🐶
[base.Relativeurls] Use relative URLs for docs URLs. For example, use [Get Started](/gateway/latest/get-started/) instead of [Get Started](https://developer.konghq.com/gateway/3.3.x/get-started/). If a direct URL is necessary, use {{ site.links.web }} instead of `https://developer.konghq.com`.

Raw Output:
{"message": "[base.Relativeurls] Use relative URLs for docs URLs. For example, use [Get Started](/gateway/latest/get-started/) instead of [Get Started](https://developer.konghq.com/gateway/3.3.x/get-started/). If a direct URL is necessary, use {{ site.links.web }} instead of `https://developer.konghq.com`.", "location": {"path": "app/_cookbooks/basic-llm-routing.md", "range": {"start": {"line": 267, "column": 592}}}, "severity": "ERROR"}

**`hide_credentials: true`**. Strips the API key from the request before forwarding upstream. The provider never sees the Consumer's API key. This is a 3.14 default but the recipe sets it explicitly for clarity and to remain portable to older Gateway versions.

Expand Down Expand Up @@ -333,7 +332,7 @@
- **Additional route types.** A single Plugin instance can have multiple targets for different route types, each with their own model and auth configuration. Beyond `llm/v1/chat`, the Plugin supports additional route types for embeddings, completions, responses, realtime, and multimodal traffic. See the [ai-proxy-advanced reference](/plugins/ai-proxy-advanced/) for the current list.

{:.info}
> **Production credentials.** This recipe stores the Consumer API key directly in Plugin config and the LLM provider credentials in environment variables for simplicity. In production, use [Kong Vaults](/gateway/secrets-management/) to reference both from your preferred secret manager (AWS Secrets Manager, HashiCorp Vault, GCP Secret Manager, Azure Key Vault) instead.
> **Production credentials.** This recipe stores the Consumer API key directly in Plugin config and the LLM provider credentials in environment variables for simplicity. In production, use [Kong Vaults](/gateway/latest/kong-enterprise/secrets-management/) to reference both from your preferred secret manager (AWS Secrets Manager, HashiCorp Vault, GCP Secret Manager, Azure Key Vault) instead.

### Example response

Expand Down Expand Up @@ -992,7 +991,7 @@
The demo script makes three calls. The first two send the same prompt with different `model` values (`fast` then `smart`) and print the `X-Kong-LLM-Model` header so you can confirm Kong routed each request to a different upstream model. The third call presents an invalid API key and shows Kong rejecting it with `401` before any upstream call.

{:.info}
> The demo passes the API key via `default_headers` because the OpenAI SDK reserves `api_key` for the `Authorization: Bearer` header. To let clients pass the key through `api_key` directly, attach a [pre-function](/plugins/pre-function/) Plugin that copies the Bearer token to the `apikey` header server-side. See [Authenticate OpenAI SDK clients with Key Auth](/how-to/authenticate-openai-sdk-clients-with-key-auth/) for the pattern.
> The demo passes the API key via `default_headers` because the OpenAI SDK reserves `api_key` for the `Authorization: Bearer` header. To let clients pass the key through `api_key` directly, attach a [pre-function](/plugins/pre-function/) Plugin that copies the Bearer token to the `apikey` header server-side. See [Authenticate OpenAI SDK clients with Key Auth](https://developer.konghq.com/how-to/authenticate-openai-sdk-clients-with-key-auth/) for the pattern.

Create the demo script:

Expand Down
12 changes: 6 additions & 6 deletions app/_cookbooks/claude-code-sso.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ min_version:
categories:
- access-control
- llm
featured: true
featured: false
popular: false

# Machine-readable fields for AI agent setup
Expand All @@ -41,7 +41,7 @@ prereqs:
skip_product: true
skip_tool: true
inline:
- title: "{{site.konnect_product_name}}"
- title: Kong Konnect
content: |
This tutorial uses {{site.konnect_product_name}}. The [quickstart script](https://get.konghq.com/quickstart) provisions a recipe-scoped Control Plane and local Data Plane.

Expand All @@ -68,7 +68,7 @@ prereqs:
content: |
This tutorial uses [kongctl](/kongctl/) and [decK](/deck/) to manage Kong configuration, plus [jq](https://jqlang.org/) for JSON processing in the apply and cleanup steps.

1. Install **kongctl** from [developer.konghq.com/kongctl](/kongctl/).
1. Install **kongctl** from [developer.konghq.com/kongctl](https://developer.konghq.com/kongctl/).
2. Install **decK** version 1.43 or later from [docs.konghq.com/deck](https://docs.konghq.com/deck/).
3. Install **jq** from [jqlang.org](https://jqlang.org/).

Expand Down Expand Up @@ -150,7 +150,7 @@ prereqs:
```

{:.warning}
> **`apiKeyHelper` is bypassed if `ANTHROPIC_API_KEY` or `ANTHROPIC_AUTH_TOKEN` is set in your environment.** Both env vars take precedence over the helper per Claude Code's credential precedence rules. If either is set in your shell profile, unset it before running Claude Code with this recipe: `unset ANTHROPIC_API_KEY ANTHROPIC_AUTH_TOKEN`.
> **`apiKeyHelper` is bypassed if `ANTHROPIC_API_KEY` or `ANTHROPIC_AUTH_TOKEN` is set in your environment.** Both env vars take precedence over the helper per Claude Code's [credential precedence rules](https://code.claude.com/docs/en/iam). If either is set in your shell profile, unset it before running Claude Code with this recipe: `unset ANTHROPIC_API_KEY ANTHROPIC_AUTH_TOKEN`.

**Helper script: okta-claude-auth.sh**

Expand Down Expand Up @@ -540,7 +540,7 @@ sequenceDiagram
participant CC as Claude Code
participant H as apiKeyHelper script
participant O as Okta
participant K as {{site.base_gateway}}
participant K as Kong Gateway
participant L as LLM Provider

CC->>H: Request bearer token
Expand Down Expand Up @@ -896,7 +896,7 @@ When the token limit is exceeded, Kong returns `429 Too Many Requests` with a `R
header.

{:.info}
> The 60-second windows here are intentionally aggressive for the demo so a few interactive prompts visibly exhaust the budget. Most teams enforce monthly or daily token budgets in production, for example {%raw%}`limits: [{limit: 5000000, window_size: 2592000}]`{%endraw%}. Combine with [Kong Vaults](/gateway/secrets-management/) using {%raw%}`{vault://backend/key}`{%endraw%} references for credentials in production rather than environment variables.
> The 60-second windows here are intentionally aggressive for the demo so a few interactive prompts visibly exhaust the budget. Most teams enforce monthly or daily token budgets in production, for example {%raw%}`limits: [{limit: 5000000, window_size: 2592000}]`{%endraw%}. Combine with [Kong Vaults](/gateway/latest/kong-enterprise/secrets-management/) using {%raw%}`{vault://backend/key}`{%endraw%} references for credentials in production rather than environment variables.

## Apply the Kong configuration

Expand Down
Loading
Loading