diff --git a/en/docs/ai-gateway/1.1.0/overview.md b/en/docs/ai-gateway/1.1.0/overview.md index 642e38f38..4f8627e69 100644 --- a/en/docs/ai-gateway/1.1.0/overview.md +++ b/en/docs/ai-gateway/1.1.0/overview.md @@ -69,6 +69,10 @@ An MCP Proxy routes Model Context Protocol traffic to MCP servers. MCP is a prot - Apply authentication and access control to MCP traffic - Manage multiple MCP servers from a single control plane +### Streaming + +When an upstream service streams its response, the gateway relays it to the client chunk by chunk instead of buffering the whole response. This holds for LLM providers, App LLM proxies, and MCP proxies, and needs no configuration. See [Streaming responses](streaming-responses.md). + ## Default Ports | Port | Service | Description | @@ -115,6 +119,7 @@ You can extend the AI Gateway with custom guardrail policies by building a custo |---------|-------------| | [LLM](llm-proxy/quick-start-guide.md) | LLM provider configuration, guardrails, prompt management, and semantic caching | | [MCP](mcp-proxy/quick-start-guide.md) | MCP proxy setup and policies | +| [Streaming](streaming-responses.md) | Streamed responses across providers and proxies, and how policies and analytics behave | | [Observability](observability/logging.md) | Logging and tracing configuration | | [Analytics](analytics/moesif-analytics.md) | Analytics integrations (Moesif) | | [Policies and Guardrails](https://github.com/wso2/gateway-controllers/blob/main/docs/README.md) | Gateway policies and guardrails for AI traffic control | diff --git a/en/docs/ai-gateway/1.1.0/streaming-responses.md b/en/docs/ai-gateway/1.1.0/streaming-responses.md new file mode 100644 index 000000000..4a7182852 --- /dev/null +++ b/en/docs/ai-gateway/1.1.0/streaming-responses.md @@ -0,0 +1,108 @@ +--- +title: "Streaming Responses" +description: "Stream responses through API Platform AI Gateway chunk by chunk across LLM providers, LLM proxies, and MCP proxies, and understand how policies, analytics, and token usage behave." +canonical_url: https://wso2.com/api-platform/docs/ai-gateway/streaming-responses/ +md_url: https://wso2.com/api-platform/docs/ai-gateway/streaming-responses.md +tags: + - ai-gateway + - llm + - mcp + - streaming +author: WSO2 API Platform Documentation Team +last_updated: 2026-08-04 +content_type: "concept" +--- + +# Streaming responses + +The AI Gateway forwards a streamed response to the client chunk by chunk, as each chunk arrives from the upstream service. The gateway doesn't hold the response until the upstream finishes generating it, so the first token reaches your application at about the same time it leaves the provider. Chat interfaces and agent loops keep their token-by-token behavior when they run through the gateway. + +Streaming applies across the gateway's artifact types: + +- **LLM providers** — a request sent straight to a provider endpoint, such as `/openai/latest/chat/completions`, streams when the upstream streams. +- **App LLM proxies** — a proxy inherits the streaming behavior of the provider it consumes. +- **MCP proxies** — request bodies stream, and responses are handled differently. See [MCP proxies](#mcp-proxies). + +This page is for AI developers building on the gateway, and for platform administrators deciding which policies to attach. + +## How the gateway detects a streaming response + +Streaming needs no configuration on the `LlmProvider` or `Mcp` resource. The gateway decides per response, based on what the upstream sends: + +- The response carries `Content-Type: text/event-stream`, which is how OpenAI-compatible providers return Server-Sent Events (SSE). +- The response carries `Transfer-Encoding: chunked`. + +When either holds, the gateway switches the response body to full-duplex streaming and relays each chunk downstream as it arrives. The gateway applies the same two checks to request bodies, so a chunked or SSE request body also streams upstream. + +To stream an LLM response, set `"stream": true` in the request body, exactly as you would when calling the provider directly. The following example calls an OpenAI provider deployed at `/openai/latest`: + +```bash +curl -N -X POST "https://localhost:8443/openai/latest/chat/completions" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "gpt-4o-mini", + "stream": true, + "messages": [ + { + "role": "user", + "content": "Write a haiku about API gateways." + } + ] + }' -k +``` + +The same request works against an App LLM proxy. Replace `/openai/latest` with the proxy context, such as `/assistant`. + +The `-N` flag turns off curl's own output buffering, so you see the SSE events as they arrive rather than all at once at the end. + +## How policies behave on a streamed response + +Whether a response streams depends on the policies attached to the route. A policy that reads the response body either supports chunk-by-chunk processing or requires the complete body. + +**Every response-body policy on the chain must support streaming.** The gateway evaluates this per route, and it's all or nothing: + +- If every response-body policy supports streaming, the gateway streams the response to the client. +- If one policy requires the complete body, the gateway buffers the entire response, runs the chain, and then sends the response in one piece. The result is still correct, but the client waits for the last token before it sees the first. + +The chain spans both levels. For a request through an App LLM proxy, it covers the enterprise-wide policies the platform administrator attached to the `LlmProvider` and the application-specific policies the developer attached to the proxy. A buffered-only policy at either level buffers the response. + +Policies that don't read the response body — authentication, request-side rate limiting, header policies, prompt management — never affect streaming. + +### Gating policies + +A streaming-capable policy can still hold bytes back when it has to. A guardrail that enforces a minimum, such as a minimum sentence count, can't rule on content it hasn't seen. Such a policy accumulates chunks silently until it has enough content to decide, releases what it has accumulated, and then processes each later chunk as it arrives. The client sees a pause at the start of the response rather than a wait for the whole response. + +### MCP proxies + +Response bodies on MCP proxies stay buffered, even when the MCP server replies with a chunked or SSE body. The gateway runs the response chain against the complete body and then sends it. Request bodies on MCP proxies stream under the same rules as any other route. + +## Analytics on a streamed response + +Analytics doesn't cost you the streaming behavior. As the gateway forwards each chunk to the client, it also keeps its own copy. At the end of the stream, it parses the accumulated SSE events and emits one analytics event for the request. The client receives every chunk at the time it arrives; the copy is only used after the stream closes. + +## Token usage on a streamed response + +Token counts drive analytics, cost tracking, and token-based rate limiting on LLM traffic. On a streamed response, the gateway reads them from the `usage` block that the provider sends in the stream, which arrives in the final events rather than in every chunk. + +Providers differ in when they send that block: + +- **OpenAI-compatible providers** omit `usage` unless the client asks for it. Add `stream_options` to the request: + + ```json + { + "model": "gpt-4o-mini", + "stream": true, + "stream_options": { "include_usage": true }, + "messages": [{ "role": "user", "content": "Write a haiku about API gateways." }] + } + ``` + +- **Anthropic** reports token counts in its `message_start` and `message_delta` events, so no extra request field is needed. + +If a streamed response carries no `usage` block, the gateway has no token counts to record for that request. Analytics, cost calculation, and token-based rate limiting skip it. Set `stream_options` on OpenAI-compatible requests whenever you rely on any of those, including when the budget controls on the `LlmProvider` use token-based rate limiting. + +## Related documentation + +- [LLM Proxy Quick Start Guide](llm-proxy/quick-start-guide.md) — deploy a provider and a proxy, then send your first request +- [MCP Proxy Quick Start Guide](mcp-proxy/quick-start-guide.md) — deploy an MCP proxy +- [Sentence Count Guardrail](llm-proxy/guardrails/sentence-count.md) — a guardrail that gates a stream until it can evaluate the content diff --git a/en/docs/ai-gateway/next/llm-proxy/quick-start-guide.md b/en/docs/ai-gateway/next/llm-proxy/quick-start-guide.md index 3b0e0d9d5..6c434d1ed 100644 --- a/en/docs/ai-gateway/next/llm-proxy/quick-start-guide.md +++ b/en/docs/ai-gateway/next/llm-proxy/quick-start-guide.md @@ -67,7 +67,7 @@ export ADMIN_USERNAME=admin export ADMIN_PASSWORD='' # Start the complete stack -docker compose up -d +docker compose up # Verify gateway controller admin endpoint is running curl http://localhost:9094/api/admin/v1/health @@ -182,6 +182,10 @@ curl -X POST "https://localhost:8443/assistant/chat/completions" \ }' -k ``` +## View the LLM provider and proxy in AI Workspace + +The gateway syncs the artifacts you deploy on it up to [AI Workspace](../../../next/ai-workspace/overview.md), the control plane for AI traffic across your organization. The OpenAI provider and the `openai-assistant` proxy you deployed above appear there without being re-declared. See [Manage Gateway-deployed AI artifacts in AI Workspace](../../../next/ai-workspace/sync-gateway-created-artifacts.md). + ## Stopping the Gateway When stopping the gateway, you have two options: diff --git a/en/docs/ai-gateway/next/mcp-proxy/quick-start-guide.md b/en/docs/ai-gateway/next/mcp-proxy/quick-start-guide.md index fc0c38223..8d8186c65 100644 --- a/en/docs/ai-gateway/next/mcp-proxy/quick-start-guide.md +++ b/en/docs/ai-gateway/next/mcp-proxy/quick-start-guide.md @@ -109,6 +109,8 @@ apiVersion: gateway.api-platform.wso2.com/v1 kind: Mcp metadata: name: everything-mcp-v1.0 + annotations: + "gateway.api-platform.wso2.com/project-id": "default" spec: displayName: Everything version: v1.0 @@ -127,6 +129,10 @@ To test MCP traffic routing through the gateway, add the following URL to your M http://localhost:8080/everything/mcp ``` +## View the MCP proxy in AI Workspace + +The gateway syncs the artifacts you deploy on it up to [AI Workspace](../../../next/ai-workspace/overview.md), the control plane for AI traffic across your organization. The `everything-mcp-v1.0` proxy you deployed above appears there without being re-declared, in the `default` project named in its `project-id` annotation. See [Manage Gateway-deployed AI artifacts in AI Workspace](../../../next/ai-workspace/sync-gateway-created-artifacts.md). + ## Stopping the Gateway Stop and remove the MCP backend first. diff --git a/en/docs/ai-gateway/next/observability/logging.md b/en/docs/ai-gateway/next/observability/logging.md index b7a77e683..8565c1a76 100644 --- a/en/docs/ai-gateway/next/observability/logging.md +++ b/en/docs/ai-gateway/next/observability/logging.md @@ -75,7 +75,7 @@ This starts: To run only the core gateway services without the demonstration logging stack: ```bash -docker compose up -d +docker compose up ``` **Note:** The gateway components still log to stdout/stderr. You just won't have the centralized collection and visualization services running. You can still view logs using: diff --git a/en/docs/ai-gateway/next/observability/tracing.md b/en/docs/ai-gateway/next/observability/tracing.md index ed7548451..9353ff60a 100644 --- a/en/docs/ai-gateway/next/observability/tracing.md +++ b/en/docs/ai-gateway/next/observability/tracing.md @@ -88,7 +88,7 @@ This starts: To run only the core gateway services without the demonstration tracing stack: ```bash -docker compose up -d +docker compose up ``` **Note:** If tracing is enabled in the configuration but the OTLP collector is not running, components will log warnings about failed trace exports. To completely disable tracing, set `enabled = false` in the configuration. diff --git a/en/docs/ai-gateway/next/overview.md b/en/docs/ai-gateway/next/overview.md index 78bdb2bdb..757ac992c 100644 --- a/en/docs/ai-gateway/next/overview.md +++ b/en/docs/ai-gateway/next/overview.md @@ -66,6 +66,10 @@ An MCP Proxy routes Model Context Protocol traffic to MCP servers. MCP is a prot - Apply authentication and access control to MCP traffic - Manage multiple MCP servers from a single control plane +### Streaming + +When an upstream service streams its response, the gateway relays it to the client chunk by chunk instead of buffering the whole response. This holds for LLM providers, LLM proxies, and MCP proxies, and needs no configuration. See [Streaming responses](streaming-responses.md). + ## Default Ports | Port | Service | Description | @@ -112,6 +116,7 @@ You can extend the AI Gateway with custom guardrail policies by building a custo |---------|-------------| | [LLM](llm-proxy/quick-start-guide.md) | LLM provider configuration, guardrails, prompt management, and semantic caching | | [MCP](mcp-proxy/quick-start-guide.md) | MCP proxy setup and policies | +| [Streaming](streaming-responses.md) | Streamed responses across providers and proxies, and how policies and analytics behave | | [Observability](observability/logging.md) | Logging and tracing configuration | | [Analytics](analytics/moesif-analytics.md) | Analytics integrations (Moesif) | | [Policies and Guardrails](https://github.com/wso2/gateway-controllers/blob/main/docs/README.md) | Gateway policies and guardrails for AI traffic control | diff --git a/en/docs/ai-gateway/next/quick-start-guide.md b/en/docs/ai-gateway/next/quick-start-guide.md new file mode 100644 index 000000000..edb288f7a --- /dev/null +++ b/en/docs/ai-gateway/next/quick-start-guide.md @@ -0,0 +1,230 @@ +--- +title: "AI Gateway Quick Start Guide" +description: "Run API Platform AI Gateway with Docker Compose, deploy an LLM provider and an LLM proxy, route your first LLM request, and govern the gateway from AI Workspace." +canonical_url: https://wso2.com/api-platform/docs/ai-gateway/quick-start-guide/ +md_url: https://wso2.com/api-platform/docs/ai-gateway/quick-start-guide.md +tags: + - ai-gateway + - llm + - mcp + - quickstart + - docker +author: WSO2 API Platform Documentation Team +last_updated: 2026-08-04 +content_type: "quickstart" +--- + +# Quick Start Guide + +This guide takes you from a downloaded distribution to an LLM request routed through the API Platform AI Gateway, then shows you how to govern that gateway from [AI Workspace](../../next/ai-workspace/overview.md), the control plane for AI traffic. It's written for platform administrators and AI developers. + +!!! info "Watch the video walkthrough" + [Check out this quick start on YouTube](https://youtu.be/p5xBXZWt5GU?rel=0) or watch below. + + + +## Prerequisites + +A Docker-compatible container runtime such as: + +- Docker Desktop (Windows / macOS) +- Rancher Desktop (Windows / macOS) +- Colima (macOS) +- Docker Engine + Compose plugin (Linux) + +Ensure `docker` and `docker compose` commands are available. + +```bash +docker --version +docker compose version +``` + +To call an LLM through the gateway, you also need an OpenAI API key. + +## Start the gateway + +The commands below use version `1.2.0-rc`. Substitute the API Platform AI Gateway release version you want to run in the download URL, the archive name, and the directory name. + +```bash +# Download distribution. +wget https://github.com/wso2/api-platform/releases/download/ai-gateway/v1.2.0-rc/wso2apip-ai-gateway-1.2.0-rc.zip + +# Unzip the downloaded distribution. +unzip wso2apip-ai-gateway-1.2.0-rc.zip + +cd wso2apip-ai-gateway-1.2.0-rc/ + +# Run the one-time setup. This provisions the AES-256 at-rest encryption key, the router HTTPS +# listener certificate, api-platform.env, and the gateway-controller admin credentials. It prints +# the admin password once — copy it. +./scripts/setup.sh + +# Export the admin credentials so the management-API calls below can authenticate. +# The username defaults to "admin"; use the password setup.sh just printed. +export ADMIN_USERNAME=admin +export ADMIN_PASSWORD='' + +# Start the complete stack +docker compose up + +# Verify gateway controller admin endpoint is running +curl http://localhost:9094/api/admin/v1/health +``` + +!!! note "Running on Windows" + The commands above assume a Linux/macOS shell. On Windows, run the one-time setup with the PowerShell script instead — it takes the same flags and provisions the same files: + + ```powershell + powershell -ExecutionPolicy Bypass -File .\scripts\setup.ps1 + ``` + + Then set the admin credentials with `$env:ADMIN_USERNAME='admin'` and `$env:ADMIN_PASSWORD=''` in place of the `export` lines. + + The remaining `curl` commands on this page pipe their YAML payload in through a shell heredoc (`--data-binary @- <<'EOF'`), which PowerShell does not support. Either run them from Git Bash or WSL, or save the YAML between `EOF` markers to a file and post that file explicitly — note the `.exe`, since `curl` is an alias for `Invoke-WebRequest` in Windows PowerShell: + + ```powershell + curl.exe -X POST http://localhost:9090/api/management/v1/llm-providers ` + -H "Content-Type: application/yaml" ` + -u "${env:ADMIN_USERNAME}:${env:ADMIN_PASSWORD}" ` + --data-binary "@openai-provider.yaml" + ``` + +!!! tip "Customizing configuration" + The setup script (`setup.sh`, or `setup.ps1` on Windows) writes `api-platform.env`, which is loaded into the containers via Docker Compose `env_file`. To change the storage backend, connect to a control plane, or tune other settings, edit that file (or the `config.toml` interpolation tokens directly). See [Gateway Configuration and Environment Interpolation](./setup/configuration.md). + +## Deploy an OpenAI LLM provider configuration + +The API Platform Gateway includes first-class support for the OpenAI LLM provider. As a platform administrator, replace `` with your OpenAI API key and run the following command to deploy a sample OpenAI LLM provider. + +```bash +curl -X POST http://localhost:9090/api/management/v1/llm-providers \ + -H "Content-Type: application/yaml" \ + -u "$ADMIN_USERNAME:$ADMIN_PASSWORD" \ + --data-binary @- <<'EOF' +apiVersion: gateway.api-platform.wso2.com/v1 +kind: LlmProvider +metadata: + name: openai-provider +spec: + displayName: OpenAI Provider + version: v1.0 + template: openai + context: /openai/latest + upstream: + url: https://api.openai.com/v1 + auth: + type: api-key + header: Authorization + value: + accessControl: + mode: deny_all + exceptions: + - path: /chat/completions + methods: [POST] + - path: /models + methods: [GET] + - path: /models/{modelId} + methods: [GET] +EOF +``` + +To test LLM provider traffic routing through the gateway, invoke the following request. + +```bash +curl -X POST https://localhost:8443/openai/latest/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "gpt-4o-mini", + "messages": [ + { + "role": "user", + "content": "Hi" + } + ] + }' -k +``` + +## Deploy an LLM proxy configuration to consume an LLM provider + +The API Platform Gateway provides first-class support for configuring and deploying LLM proxies. As an AI developer, run the following command to deploy a sample LLM proxy that consumes the OpenAI LLM provider the platform administrator deployed above. + +```bash +curl -X POST http://localhost:9090/api/management/v1/llm-proxies \ + -H "Content-Type: application/yaml" \ + -u "$ADMIN_USERNAME:$ADMIN_PASSWORD" \ + --data-binary @- <<'EOF' +apiVersion: gateway.api-platform.wso2.com/v1 +kind: LlmProxy +metadata: + name: openai-assistant +spec: + displayName: OpenAI Assistant + version: v1.0 + context: /assistant + provider: + id: openai-provider + policies: [] +EOF +``` + +To test LLM proxy traffic routing through the gateway and consume the LLM provider, invoke the following request. + +```bash +curl -X POST "https://localhost:8443/assistant/chat/completions" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "gpt-4o-mini", + "messages": [ + { + "role": "user", + "content": "Hi" + } + ] + }' -k +``` + +## Govern this gateway from AI Workspace + +The gateway you just started serves traffic on its own, and it doesn't have to run alone. [AI Workspace](../../next/ai-workspace/overview.md) is the control plane for AI traffic across your organization: one console for LLM providers, App LLM proxies, MCP proxies, policies such as guardrails and token-based rate limits, and the credentials behind them. Register this gateway with AI Workspace to govern every AI gateway you run from a single place, across every environment. + +Both directions work, and you can use them together: + +- **Top-down.** Configure an artifact in AI Workspace, apply policies to it, then deploy it to one or more gateways. +- **Bottom-up.** Keep deploying through the management API, the way this guide does. Every artifact you create on the gateway syncs up to AI Workspace automatically and appears there as a copy the gateway owns, so the OpenAI provider and the `openai-assistant` proxy you deployed above show up without being re-declared. To see what a synced artifact looks like, and what stays editable, see [Manage Gateway-deployed AI artifacts in AI Workspace](../../next/ai-workspace/sync-gateway-created-artifacts.md). + +The gateway keeps serving traffic either way. If AI Workspace is unreachable, the gateway carries on and the sync catches up once the connection is restored. + +## Stopping the gateway + +When stopping the gateway, you have two options: + +**Option 1: Stop runtime, keep data (persisted proxies and configuration)** + +```bash +docker compose down +``` + +This stops the containers but preserves the `controller-data` volume. When you restart with `docker compose up`, all your configurations are restored. + +**Option 2: Complete shutdown with data cleanup (fresh start)** + +```bash +docker compose down -v +``` + +This stops the containers and removes the `controller-data` volume. The next startup is a clean slate with no persisted templates or provider configuration. + +## Next steps + +- Route to more than one provider, with failover: [Multi-provider routing](./llm-proxy/multi-provider-routing.md) +- Add guardrails to a proxy, such as [PII masking](./llm-proxy/guardrails/pii-masking-regex.md) or a [JSON schema guardrail](./llm-proxy/guardrails/json-schema.md) +- Expose an MCP server through the gateway: [MCP proxy quick start guide](./mcp-proxy/quick-start-guide.md) +- Govern AI traffic across all your gateways from the control plane: [AI Workspace overview](../../next/ai-workspace/overview.md) diff --git a/en/docs/ai-gateway/next/setup/configuration.md b/en/docs/ai-gateway/next/setup/configuration.md index bea19e181..d74c0d490 100644 --- a/en/docs/ai-gateway/next/setup/configuration.md +++ b/en/docs/ai-gateway/next/setup/configuration.md @@ -113,14 +113,14 @@ The distribution ships `scripts/setup.sh` (and `scripts/setup.ps1`, its Windows ```bash ./scripts/setup.sh - docker compose up -d + docker compose up ``` === "Windows (PowerShell)" ```powershell powershell -ExecutionPolicy Bypass -File .\scripts\setup.ps1 - docker compose up -d + docker compose up ``` The setup script provisions, idempotently: diff --git a/en/docs/ai-gateway/next/setup/database-setup.md b/en/docs/ai-gateway/next/setup/database-setup.md index f23d387bf..e9002a692 100644 --- a/en/docs/ai-gateway/next/setup/database-setup.md +++ b/en/docs/ai-gateway/next/setup/database-setup.md @@ -284,7 +284,7 @@ For SQL Server, the shipped Compose files supply the whole connection string thr Start the gateway: ```bash -docker compose up -d +docker compose up ``` On startup the controller logs that it connected to the external database and that schema auto-apply was skipped. That message is expected — it confirms the controller is relying on the schema you provisioned. diff --git a/en/docs/ai-gateway/next/streaming-responses.md b/en/docs/ai-gateway/next/streaming-responses.md new file mode 100644 index 000000000..1a82f393d --- /dev/null +++ b/en/docs/ai-gateway/next/streaming-responses.md @@ -0,0 +1,108 @@ +--- +title: "Streaming Responses" +description: "Stream responses through API Platform AI Gateway chunk by chunk across LLM providers, LLM proxies, and MCP proxies, and understand how policies, analytics, and token usage behave." +canonical_url: https://wso2.com/api-platform/docs/ai-gateway/streaming-responses/ +md_url: https://wso2.com/api-platform/docs/ai-gateway/streaming-responses.md +tags: + - ai-gateway + - llm + - mcp + - streaming +author: WSO2 API Platform Documentation Team +last_updated: 2026-08-04 +content_type: "concept" +--- + +# Streaming responses + +The AI Gateway forwards a streamed response to the client chunk by chunk, as each chunk arrives from the upstream service. The gateway doesn't hold the response until the upstream finishes generating it, so the first token reaches your application at about the same time it leaves the provider. Chat interfaces and agent loops keep their token-by-token behavior when they run through the gateway. + +Streaming applies across the gateway's artifact types: + +- **LLM providers** — a request sent straight to a provider endpoint, such as `/openai/latest/chat/completions`, streams when the upstream streams. +- **LLM proxies** — a proxy inherits the streaming behavior of the provider it consumes. +- **MCP proxies** — request bodies stream, and responses are handled differently. See [MCP proxies](#mcp-proxies). + +This page is for AI developers building on the gateway, and for platform administrators deciding which policies to attach. + +## How the gateway detects a streaming response + +Streaming needs no configuration on the `LlmProvider`, `LlmProxy`, or `McpProxy`. The gateway decides per response, based on what the upstream sends: + +- The response carries `Content-Type: text/event-stream`, which is how OpenAI-compatible providers return Server-Sent Events (SSE). +- The response carries `Transfer-Encoding: chunked`. + +When either holds, the gateway switches the response body to full-duplex streaming and relays each chunk downstream as it arrives. The gateway applies the same two checks to request bodies, so a chunked or SSE request body also streams upstream. + +To stream an LLM response, set `"stream": true` in the request body, exactly as you would when calling the provider directly. The following example calls an LLM proxy deployed at `/assistant`: + +```bash +curl -N -X POST "https://localhost:8443/assistant/chat/completions" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "gpt-4o-mini", + "stream": true, + "messages": [ + { + "role": "user", + "content": "Write a haiku about API gateways." + } + ] + }' -k +``` + +The same request works against the provider endpoint directly. Replace `/assistant` with the provider context, such as `/openai/latest`. + +The `-N` flag turns off curl's own output buffering, so you see the SSE events as they arrive rather than all at once at the end. + +## How policies behave on a streamed response + +Whether a response streams depends on the policies attached to the route. A policy that reads the response body either supports chunk-by-chunk processing or requires the complete body. + +**Every response-body policy on the chain must support streaming.** The gateway evaluates this per route, and it's all or nothing: + +- If every response-body policy supports streaming, the gateway streams the response to the client. +- If one policy requires the complete body, the gateway buffers the entire response, runs the chain, and then sends the response in one piece. The result is still correct, but the client waits for the last token before it sees the first. + +The chain spans both levels. For a request through an LLM proxy, it covers the organization-wide policies the platform administrator attached to the `LlmProvider` and the per-application policies the developer attached to the `LlmProxy`. A buffered-only policy at either level buffers the response. + +Policies that don't read the response body — authentication, request-side rate limiting, header policies, prompt management — never affect streaming. + +### Gating policies + +A streaming-capable policy can still hold bytes back when it has to. A guardrail that enforces a minimum, such as a minimum sentence count, can't rule on content it hasn't seen. Such a policy accumulates chunks silently until it has enough content to decide, releases what it has accumulated, and then processes each later chunk as it arrives. The client sees a pause at the start of the response rather than a wait for the whole response. + +### MCP proxies + +Response bodies on MCP proxies stay buffered, even when the MCP server replies with a chunked or SSE body. The gateway runs the response chain against the complete body and then sends it. Request bodies on MCP proxies stream under the same rules as any other route. + +## Analytics on a streamed response + +Analytics doesn't cost you the streaming behavior. As the gateway forwards each chunk to the client, it also keeps its own copy. At the end of the stream, it parses the accumulated SSE events and emits one analytics event for the request. The client receives every chunk at the time it arrives; the copy is only used after the stream closes. + +## Token usage on a streamed response + +Token counts drive analytics, cost tracking, and token-based rate limiting on LLM traffic. On a streamed response, the gateway reads them from the `usage` block that the provider sends in the stream, which arrives in the final events rather than in every chunk. + +Providers differ in when they send that block: + +- **OpenAI-compatible providers** omit `usage` unless the client asks for it. Add `stream_options` to the request: + + ```json + { + "model": "gpt-4o-mini", + "stream": true, + "stream_options": { "include_usage": true }, + "messages": [{ "role": "user", "content": "Write a haiku about API gateways." }] + } + ``` + +- **Anthropic** reports token counts in its `message_start` and `message_delta` events, so no extra request field is needed. + +If a streamed response carries no `usage` block, the gateway has no token counts to record for that request. Analytics, cost calculation, and token-based rate limiting skip it. Set `stream_options` on OpenAI-compatible requests whenever you rely on any of those, including when the budget controls on the `LlmProvider` use token-based rate limiting. + +## Related documentation + +- [LLM Proxy Quick Start Guide](llm-proxy/quick-start-guide.md) — deploy a provider and a proxy, then send your first request +- [MCP Proxy Quick Start Guide](mcp-proxy/quick-start-guide.md) — deploy an MCP proxy +- [Sentence Count Guardrail](llm-proxy/guardrails/sentence-count.md) — a guardrail that gates a stream until it can evaluate the content diff --git a/en/docs/api-gateway/next/observability/logging.md b/en/docs/api-gateway/next/observability/logging.md index b0a6260db..7b7ccace1 100644 --- a/en/docs/api-gateway/next/observability/logging.md +++ b/en/docs/api-gateway/next/observability/logging.md @@ -75,7 +75,7 @@ This starts: To run only the core gateway services without the demonstration logging stack: ```bash -docker compose up -d +docker compose up ``` **Note:** The gateway components still log to stdout/stderr. You just won't have the centralized collection and visualization services running. You can still view logs using: diff --git a/en/docs/api-gateway/next/observability/metrics/enabling-metrics.md b/en/docs/api-gateway/next/observability/metrics/enabling-metrics.md index 04438a08a..b11978574 100644 --- a/en/docs/api-gateway/next/observability/metrics/enabling-metrics.md +++ b/en/docs/api-gateway/next/observability/metrics/enabling-metrics.md @@ -87,7 +87,7 @@ This starts: To run only the core gateway services without the demonstration metrics stack: ```bash -docker compose up -d +docker compose up ``` **Note:** The gateway components still expose metrics if enabled in the configuration. You can still access metrics directly at: diff --git a/en/docs/api-gateway/next/observability/tracing/enabling-tracing.md b/en/docs/api-gateway/next/observability/tracing/enabling-tracing.md index 327ae76fe..77b6d146e 100644 --- a/en/docs/api-gateway/next/observability/tracing/enabling-tracing.md +++ b/en/docs/api-gateway/next/observability/tracing/enabling-tracing.md @@ -58,7 +58,7 @@ This starts: To run only the core gateway services without the demonstration tracing stack: ```bash -docker compose up -d +docker compose up ``` **Note:** If tracing is enabled in the configuration but the OTLP collector is not running, components will log warnings about failed trace exports. To completely disable tracing, set `enabled = false` in the configuration. diff --git a/en/docs/api-gateway/next/policies/custom-policies/building-gateway-with-custom-policies.md b/en/docs/api-gateway/next/policies/custom-policies/building-gateway-with-custom-policies.md index ffefb924a..d25e23b11 100644 --- a/en/docs/api-gateway/next/policies/custom-policies/building-gateway-with-custom-policies.md +++ b/en/docs/api-gateway/next/policies/custom-policies/building-gateway-with-custom-policies.md @@ -235,7 +235,7 @@ services: Once updated, start the gateway as usual: ```bash -docker compose up -d +docker compose up ``` ## Deploy the API diff --git a/en/docs/api-gateway/next/quick-start-guide.md b/en/docs/api-gateway/next/quick-start-guide.md index 381a2e995..f44c006ce 100644 --- a/en/docs/api-gateway/next/quick-start-guide.md +++ b/en/docs/api-gateway/next/quick-start-guide.md @@ -54,7 +54,7 @@ export ADMIN_USERNAME=admin export ADMIN_PASSWORD='' # Start the complete stack -docker compose up -d +docker compose up # Verify gateway controller admin endpoint is running curl http://localhost:9094/api/admin/v1/health diff --git a/en/docs/api-gateway/next/setup/configuration.md b/en/docs/api-gateway/next/setup/configuration.md index 85e52a3e2..13725def0 100644 --- a/en/docs/api-gateway/next/setup/configuration.md +++ b/en/docs/api-gateway/next/setup/configuration.md @@ -114,14 +114,14 @@ The distribution ships `scripts/setup.sh` (and `scripts/setup.ps1`, its Windows ```bash ./scripts/setup.sh - docker compose up -d + docker compose up ``` === "Windows (PowerShell)" ```powershell powershell -ExecutionPolicy Bypass -File .\scripts\setup.ps1 - docker compose up -d + docker compose up ``` The setup script provisions, idempotently: diff --git a/en/docs/api-gateway/next/setup/database-setup.md b/en/docs/api-gateway/next/setup/database-setup.md index c3e04adb0..ba123f1d0 100644 --- a/en/docs/api-gateway/next/setup/database-setup.md +++ b/en/docs/api-gateway/next/setup/database-setup.md @@ -284,7 +284,7 @@ For SQL Server, the shipped Compose files supply the whole connection string thr Start the gateway: ```bash -docker compose up -d +docker compose up ``` On startup the controller logs that it connected to the external database and that schema auto-apply was skipped. That message is expected — it confirms the controller is relying on the schema you provisioned. diff --git a/en/docs/next/ai-workspace/getting-started.md b/en/docs/next/ai-workspace/getting-started.md index eb9b4ecf2..f99c54561 100644 --- a/en/docs/next/ai-workspace/getting-started.md +++ b/en/docs/next/ai-workspace/getting-started.md @@ -50,7 +50,7 @@ The script prompts for the admin username and password. Press Enter a | API Portal encryption key | `resources/keys/api-portal-encryption.key` | Encrypts the API Portal's subscription and webhook secrets at rest. Retain it for the same reason. | | API Portal session secret | `resources/keys/api-portal-session-secret` | Signs API Portal session cookies. Rotating it only signs users out. | | Admin credentials | `api-platform.env` | The Platform API's basic-auth admin user: `APIP_CP_ADMIN_USERNAME` plus the bcrypt `APIP_CP_ADMIN_PASSWORD_HASH`. | -| Compose defaults | `.env` | `COMPOSE_PROFILES`, which decides the services a plain `docker compose up -d` starts, and `COMPOSE_PROJECT_NAME`, which namespaces this copy's containers, networks, and volumes. | +| Compose defaults | `.env` | `COMPOSE_PROFILES`, which decides the services a plain `docker compose up` starts, and `COMPOSE_PROJECT_NAME`, which namespaces this copy's containers, networks, and volumes. | !!! warning "Save the printed admin username and password" @@ -62,7 +62,7 @@ The script prompts for the admin username and password. Press Enter a ## Step 3: Start the stack ```bash -docker compose up -d +docker compose up ``` !!! tip "Port 9643 or 9243 already taken?" @@ -113,7 +113,7 @@ Rerunning `./scripts/setup.sh` is safe. By default it fills in only what's missi | `--force` | Regenerate the TLS certificate, the JWT keypair, and the API Portal session secret, and rotate the admin credentials. Never touches either encryption key. | | `--rotate-encryption-key` | Replace `resources/keys/encryption.key` and `resources/keys/api-portal-encryption.key`, even though they exist. Destructive — see the warning below. | | `--certs-only` | Generate only the TLS certificate. Skips the keys, the admin credentials, and `api-platform.env`. | -| `--profiles=` | Write a different `COMPOSE_PROFILES` value to `.env`, for example `--profiles=all` or `--profiles=platform-api`. | +| `--profiles=` | Write a different `COMPOSE_PROFILES` value to `.env`, for example `--profiles=platform-api` or `--profiles=platform-api,api-portal`. | To rotate a single value by hand, delete it from `api-platform.env` — or delete the file under `resources/certificates` or `resources/keys` — and rerun the script. diff --git a/en/docs/next/ai-workspace/setting-up/authentication/connect-an-identity-provider.md b/en/docs/next/ai-workspace/setting-up/authentication/connect-an-identity-provider.md index 928ad92c7..3a44b9af5 100644 --- a/en/docs/next/ai-workspace/setting-up/authentication/connect-an-identity-provider.md +++ b/en/docs/next/ai-workspace/setting-up/authentication/connect-an-identity-provider.md @@ -146,6 +146,11 @@ client_secret = '{{ env "APIP_AIW_AUTH_OIDC_CLIENT_SECRET" }}' redirect_url = "https:///api/auth/callback" post_logout_redirect_url = "https:///login" +# Must match [platform_api.auth.authorization] — see below. +[ai_workspace.auth.authorization] +mode = "role" # or "scope" +role_to_scope_mapping = "/etc/ai-workspace/role-to-scope-mapping.yaml" + # A sibling of [ai_workspace.auth.oidc], not nested in it — applies to both auth modes. [ai_workspace.auth.claim_mappings] organization = "org_id" # claim carrying the org ID @@ -159,8 +164,9 @@ roles = "roles" {% endraw %} -Three things to get right: +Four things to get right: +- **`[ai_workspace.auth.authorization]`** needs the same `mode` as `[platform_api.auth.authorization]`. In role mode, mount the same mapping file into the `ai-workspace` container. The UI gates every action on the scopes `/api/session` reports, and AI Workspace derives those scopes from the token. In role mode it expands the `roles` claim through the mapping file. Omit this table, and the UI blocks operations the Platform API would authorize. - **`authority`** is the issuer URL. Endpoints are discovered from it, so it must be the URL whose `/.well-known/openid-configuration` describes your IdP. - **`redirect_url`** must match the URL registered in Step 1 exactly, character for character. - **`[ai_workspace.auth.claim_mappings]`** must give every key it shares with `[platform_api.auth.claim_mappings]` the same value. Both services read the same token, so a mismatch means one of them reads the wrong claim. AI Workspace uses `username` and `email` to render the signed-in user, so they matter here as much as the organization claims. The table has no `user_id` key — only the Platform API maps that claim. @@ -184,7 +190,7 @@ In the Docker Compose distribution, set it in the git-ignored `api-platform.env` Restart both services so they reload the configuration: ```bash -docker compose up -d --force-recreate +docker compose up --force-recreate ``` Open AI Workspace. Instead of the username and password form, you're redirected to your IdP's hosted login page, and land back in the workspace after signing in. @@ -211,5 +217,6 @@ The roles claim is the one that most often differs. These paths are known to wor ## Next steps - [Set up Asgardeo as your identity provider](asgardeo-setup.md): an example of these steps applied to Asgardeo, including scope registration +- [Set up Microsoft Entra ID as your identity provider](entra-id-setup.md): these steps applied to Entra ID, where authorization runs on app roles rather than scope-based authorization, and an API access scope such as `api:///access` is still required to acquire a token - [Authentication in AI Workspace](overview.md): how identity provider authentication compares with file-based authentication - [AI Workspace configuration](../configuration.md): how interpolation tokens deliver values into `config.toml` diff --git a/en/docs/next/ai-workspace/setting-up/authentication/entra-id-setup.md b/en/docs/next/ai-workspace/setting-up/authentication/entra-id-setup.md new file mode 100644 index 000000000..3249b7b1c --- /dev/null +++ b/en/docs/next/ai-workspace/setting-up/authentication/entra-id-setup.md @@ -0,0 +1,324 @@ +--- +title: "Set up Microsoft Entra ID as your identity provider" +description: "Configure Microsoft Entra ID for a production AI Workspace deployment: application registration, app roles, and the config.toml settings both services read." +canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/authentication/entra-id-setup/ +md_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/authentication/entra-id-setup.md +tags: + - cloud + - ai-workspace + - authentication + - oidc + - entra-id +author: WSO2 API Platform Documentation Team +last_updated: 2026-08-03 +content_type: "how-to" +--- + +# Set up Microsoft Entra ID as your identity provider + +This guide walks you through registering a Microsoft Entra ID application and configuring AI Workspace and the Platform API to authenticate against it. + +For background on identity provider (IdP) authentication, see [Authentication in AI Workspace](overview.md). For the configuration common to all identity providers, see [Connect an identity provider to AI Workspace](connect-an-identity-provider.md). + +## Prerequisites + +Before you begin, make sure you have: + +- A Microsoft Entra ID tenant. +- Permission to register applications and grant admin consent. +- AI Workspace and the Platform API accessible over HTTPS. +- Access to the `configs/config.toml` file both services read. + +This guide uses the following placeholders: + +| Placeholder | Description | +|-------------|-------------| +| `` | The **Directory (tenant) ID** from the application's **Overview** page | +| `` | The **Application (client) ID** from the application's **Overview** page | +| `` | The AI Workspace host as the browser reaches it, including the port when it isn't `443`—for example, `localhost:9643` | + +## Configure Microsoft Entra ID + +### Step 1: Register the application + +Register the application with the following settings: + +| Setting | Value | +|---------|-------| +| **Name** | `AI Workspace` | +| **Supported account types** | Accounts in this organizational directory only | +| **Redirect URI platform** | Web | +| **Redirect URI** | `https:///api/auth/callback` | +| **Additional redirect URI** | `https:///login` | + +Choose **Web** as the platform, not **Single-page application**, and register both redirect URIs on that platform. The first receives the sign-in callback. The second is where Entra ID returns the browser after sign-out. Entra ID validates the `post_logout_redirect_uri` against the registered redirect URIs, so the `/login` destination you set as `post_logout_redirect_url` in [Step 10](#step-10-configure-oidc-authentication) has to appear here. + +To register the application, follow these steps: + +1. In the Azure portal, go to **Microsoft Entra ID > App registrations > New registration**. +2. Enter the settings from the preceding table. +3. Select **Register**. +4. Open the application's **Overview** page and record the **Application (client) ID** and the **Directory (tenant) ID**. + +### Step 2: Expose an API + +Go to **App registrations > AI Workspace > Expose an API**. + +#### 2.1 Configure the application ID URI + +1. Next to **Application ID URI**, select **Add**. +2. Keep the default value `api://`. +3. Select **Save**. + +#### 2.2 Add an API scope + +Add a scope with the following settings: + +| Setting | Value | +|---------|-------| +| **Scope name** | `access` | +| **Who can consent** | Admins and users | +| **Admin consent display name** | Access AI Workspace | +| **State** | Enabled | + +To add the scope, follow these steps: + +1. Select **Add a scope**. +2. Enter the settings from the preceding table. +3. Select **Add scope**. + +#### 2.3 Add API permissions + +1. Go to **API permissions > Add a permission > My APIs**. +2. Choose the **AI Workspace** application. +3. Under **Delegated permissions**, select the `access` scope. +4. Confirm with **Add permissions**. + +#### 2.4 Grant admin consent + +On the **API permissions** page, select **Grant admin consent** and confirm the permission shows as **Granted**. + +### Step 3: Create a client secret + +1. Go to **Certificates & secrets > New client secret**. +2. Enter a description and select an expiration period. +3. Select **Add**. +4. Copy the **Value** of the client secret. + + !!! warning + Copy the **Value**, not the **Secret ID**. Entra ID shows the secret value only when the secret is created. + +Store the secret securely. You configure it in AI Workspace in [Step 10](#step-10-configure-oidc-authentication). + +### Step 4: Configure version 2.0 access tokens + +1. Go to **App registrations > AI Workspace > Manifest**. +2. Find the `api` section and set `requestedAccessTokenVersion` to `2`: + + ```json + "api": { + "requestedAccessTokenVersion": 2 + } + ``` + +3. Save the manifest. + +### Step 5: Create application roles + +Each app role takes the following settings, shown here for `ap_admin`: + +| Setting | Value | +|---------|-------| +| **Display name** | `ap_admin` | +| **Allowed member types** | Users/Groups | +| **Value** | `ap_admin` | +| **Do you want to enable this app role?** | Enabled | + +**Value** must match the corresponding role name in `role-to-scope-mapping.yaml`. + +To create a role, follow these steps: + +1. Go to **App registrations > AI Workspace > App roles**. +2. Select **Create app role**. +3. Enter the settings from the preceding table. +4. Repeat for each role you need. + +The default roles are: + +| Role | Grants | +|------|--------| +| `ap_admin` | Full access to every resource and operation | +| `ap_operator` | Gateway and deployment operations | +| `ap_publisher` | Creating and publishing APIs and proxies | +| `ap_subscriber` | Applications and subscriptions | +| `ap_viewer` | Read-only access | + +### Step 6: Assign roles to users or groups + +1. Go to **Microsoft Entra ID > Enterprise applications** and select the **AI Workspace** application. +2. Open **Users and groups > Add user/group**. +3. Choose the user or group, then choose the application role to assign. +4. Select **Assign**. + +### Step 7: Add optional claims + +1. Go to **App registrations > AI Workspace > Token configuration**. +2. Select **Add optional claim**. +3. Select **Access** as the token type. +4. Add the `preferred_username`, `tid`, `oid`, and `email` claims. + +### Step 8: Get the OpenID Connect (OIDC) endpoints + +Go to **App registrations > AI Workspace > Overview > Endpoints** and use the version 2.0 endpoints: + +| Endpoint | URL | +|----------|-----| +| OIDC metadata | `https://login.microsoftonline.com//v2.0/.well-known/openid-configuration` | +| Issuer | `https://login.microsoftonline.com//v2.0` | +| JSON Web Key Set (JWKS) | `https://login.microsoftonline.com//discovery/v2.0/keys` | + +These values go into the Platform API configuration in the next step. + +## Configure the Platform API + +### Step 9: Configure Platform API authentication + +AI Workspace and the Platform API read the same `configs/config.toml` file. Update the `[platform_api.auth]` tables: + +```toml +# Delegate authentication to the external identity provider. +[platform_api.auth] +mode = "idp" + +# JWKS-based validation against Microsoft Entra ID. +[platform_api.auth.idp] +name = "entra" +jwks_url = "https://login.microsoftonline.com//discovery/v2.0/keys" +issuer = ["https://login.microsoftonline.com//v2.0"] +audience = [""] + +# Use application roles for authorization. +[platform_api.auth.authorization] +enabled = true +mode = "role" +role_to_scope_mapping = "/etc/platform-api/role-to-scope-mapping.yaml" + +# Microsoft Entra ID claim mappings. +[platform_api.auth.claim_mappings] +organization = "tid" +org_handle = "tid" +org_name = "tid" +user_id = "sub" +username = "preferred_username" +email = "email" +roles = "roles" +``` + +All three organization keys map to `tid`, the directory (tenant) ID. `tid` is the only tenant-level identifier a version 2.0 access token carries by default, so every user in the tenant resolves to the same organization. For a readable organization name or slug, add a custom claim that carries the same value for every user in the tenant, then map `org_name` and `org_handle` to it. + +## Configure AI Workspace + +### Step 10: Configure OIDC authentication + +In the same `configs/config.toml` file, update the `[ai_workspace.auth]` tables: + +{% raw %} + +```toml +[ai_workspace.auth] +mode = "oidc" + +[ai_workspace.auth.oidc] +authority = "https://login.microsoftonline.com//v2.0" +client_id = "" +client_secret = '{{ file "/secrets/ai-workspace/oidc_client_secret" }}' +redirect_url = "https:///api/auth/callback" +post_logout_redirect_url = "https:///login" + +# Microsoft Entra ID scope configuration. +scope = "openid profile email offline_access api:///access" + +# Use application roles for authorization. +[ai_workspace.auth.authorization] +mode = "role" +role_to_scope_mapping = "/etc/ai-workspace/role-to-scope-mapping.yaml" + +# Microsoft Entra ID claim mappings. +[ai_workspace.auth.claim_mappings] +organization = "tid" +org_name = "tid" +org_handle = "tid" +username = "preferred_username" +email = "email" +roles = "roles" +``` + +{% endraw %} + +#### Supply the client secret + +Never write the client secret as a literal in `config.toml`. For production deployments, read it from a mounted secret file: + +{% raw %} + +```toml +client_secret = '{{ file "/secrets/ai-workspace/oidc_client_secret" }}' +``` + +{% endraw %} + +For local development, read it from an environment variable instead: + +{% raw %} + +```toml +client_secret = '{{ env "APIP_AIW_AUTH_OIDC_CLIENT_SECRET" }}' +``` + +{% endraw %} + +For more information, see [Sensitive values in `config.toml`](../configuration.md#sensitive-values-in-configtoml). + +## Restart and verify + +### Step 11: Restart the services + +Restart AI Workspace and the Platform API so they load the updated configuration. For Docker Compose: + +```bash +docker compose up --force-recreate +``` + +### Step 12: Sign in to AI Workspace + +1. Open AI Workspace in your browser. You're redirected to the Microsoft sign-in page. +2. Sign in as a user who has an application role assigned. After authentication, you land back in AI Workspace. + +### Step 13: Verify the access token + +Inspect the access token using [jwt.ms](https://jwt.ms). A correctly configured token contains values similar to these: + +```json +{ + "aud": "", + "iss": "https://login.microsoftonline.com//v2.0", + "ver": "2.0", + "tid": "", + "preferred_username": "user1@example.onmicrosoft.com", + "roles": [ + "ap_admin" + ], + "scp": "access" +} +``` + +Check the following fields: + +| Field | Expected value | +|-------|----------------| +| `aud` | `` | +| `iss` | `https://login.microsoftonline.com//v2.0` | +| `ver` | `2.0` | +| `tid` | `` | +| `roles` | The assigned application role | +| `scp` | `access` | diff --git a/en/docs/next/ai-workspace/setting-up/authentication/overview.md b/en/docs/next/ai-workspace/setting-up/authentication/overview.md index 19a9b51cf..2b1587df8 100644 --- a/en/docs/next/ai-workspace/setting-up/authentication/overview.md +++ b/en/docs/next/ai-workspace/setting-up/authentication/overview.md @@ -106,7 +106,7 @@ For production, configure AI Workspace to delegate login to an identity provider | Custom claims | Tokens carry organization identity as custom claims (claim names are configurable) | | Confidential client | AI Workspace is registered as a confidential client with a client secret, not a public or single-page application client | -[Connect an identity provider to AI Workspace](connect-an-identity-provider.md) covers the configuration for any such IdP. It walks through client registration, claim mappings, and the choice between scope and role authorization. [Set up Asgardeo as your identity provider](asgardeo-setup.md) applies those steps to one specific IdP. +[Connect an identity provider to AI Workspace](connect-an-identity-provider.md) covers the configuration for any such IdP. It walks through client registration, claim mappings, and the choice between scope and role authorization. Two guides apply those steps to a specific IdP: [Set up Asgardeo as your identity provider](asgardeo-setup.md) and [Set up Microsoft Entra ID as your identity provider](entra-id-setup.md). ## Choosing a mode diff --git a/en/docs/next/ai-workspace/setting-up/database.md b/en/docs/next/ai-workspace/setting-up/database.md index b7e97ef02..3bb36c122 100644 --- a/en/docs/next/ai-workspace/setting-up/database.md +++ b/en/docs/next/ai-workspace/setting-up/database.md @@ -188,7 +188,7 @@ conn_max_lifetime = 300 # seconds before a connection is recycled Recreate the container so it reloads the configuration: ```bash -docker compose up -d --force-recreate platform-api +docker compose up --force-recreate platform-api ``` Confirm the service is healthy: diff --git a/en/docs/next/ai-workspace/setting-up/ports.md b/en/docs/next/ai-workspace/setting-up/ports.md index 9a9107200..130ea6f4a 100644 --- a/en/docs/next/ai-workspace/setting-up/ports.md +++ b/en/docs/next/ai-workspace/setting-up/ports.md @@ -119,7 +119,7 @@ So a wrong value leaves AI Workspace working normally and breaks the gateway ins Recreate the containers so they pick up the new values: ```bash -docker compose up -d --force-recreate +docker compose up --force-recreate ``` !!! note "Ports in an OpenID Connect (OIDC) setup" diff --git a/en/docs/next/ai-workspace/sync-gateway-created-artifacts.md b/en/docs/next/ai-workspace/sync-gateway-created-artifacts.md index baf3150dd..af1b13099 100644 --- a/en/docs/next/ai-workspace/sync-gateway-created-artifacts.md +++ b/en/docs/next/ai-workspace/sync-gateway-created-artifacts.md @@ -1,5 +1,5 @@ --- -title: "Sync gateway-created AI artifacts to AI Workspace" +title: "Manage Gateway-deployed AI artifacts in AI Workspace" description: "Create an LLM provider template, LLM provider, LLM proxy, or MCP proxy on the AI Gateway and view the read-only copy that syncs up to AI Workspace." canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/sync-gateway-created-artifacts/ md_url: https://wso2.com/api-platform/docs/next/ai-workspace/sync-gateway-created-artifacts.md @@ -12,7 +12,7 @@ last_updated: 2026-07-31 content_type: "how-to" --- -# Sync gateway-created AI artifacts to AI Workspace +# Manage Gateway-deployed AI artifacts in AI Workspace You can create four kinds of AI artifact directly on the AI Gateway: diff --git a/en/docs/next/api-portal/getting-started.md b/en/docs/next/api-portal/getting-started.md index dfdea8566..2214a948a 100644 --- a/en/docs/next/api-portal/getting-started.md +++ b/en/docs/next/api-portal/getting-started.md @@ -56,7 +56,7 @@ It also prompts you for an **admin username and password**. Press Enter at the p ## Step 3: Start the Portal ```bash -docker compose up -d +docker compose up ``` This starts the API Portal & MCP Hub backed by SQLite by default. On first boot, the database schema and a default organization (`default`) with a `default` view are created automatically. diff --git a/en/mkdocs.yml b/en/mkdocs.yml index 2c1406def..eea233cf4 100644 --- a/en/mkdocs.yml +++ b/en/mkdocs.yml @@ -541,6 +541,8 @@ nav: - AI Gateway: - "next": - Overview: ai-gateway/next/overview.md + - Quick Start Guide: ai-gateway/next/quick-start-guide.md + - Streaming Responses: ai-gateway/next/streaming-responses.md - Setup: - Configuration & Interpolation: ai-gateway/next/setup/configuration.md - Setting Up the Database: ai-gateway/next/setup/database-setup.md @@ -593,6 +595,7 @@ nav: - AI Gateway runtime with four CPUs: ai-gateway/next/performance/ai-gateway-runtime-with-four-cpus.md - "1.1.0": - Overview: ai-gateway/1.1.0/overview.md + - Streaming Responses: ai-gateway/1.1.0/streaming-responses.md - LLM Proxy: - Quick Start Guide: ai-gateway/1.1.0/llm-proxy/quick-start-guide.md - LLM Provider Templates: ai-gateway/1.1.0/llm-proxy/llm-templates.md @@ -760,6 +763,7 @@ nav: - Overview: next/ai-workspace/setting-up/authentication/overview.md - Connect an Identity Provider: next/ai-workspace/setting-up/authentication/connect-an-identity-provider.md - Set up Asgardeo: next/ai-workspace/setting-up/authentication/asgardeo-setup.md + - Set up Microsoft Entra ID: next/ai-workspace/setting-up/authentication/entra-id-setup.md # Build the artifacts, in the order you create them. - AI Gateways: - Setting Up: next/ai-workspace/ai-gateways/setting-up.md @@ -782,7 +786,7 @@ nav: # Credentials the artifacts above reference. - Secrets Management: next/ai-workspace/secrets-management.md # The other two ways artifacts get created. - - Sync Gateway-Created Artifacts: next/ai-workspace/sync-gateway-created-artifacts.md + - Manage Gateway-deployed AI artifacts: next/ai-workspace/sync-gateway-created-artifacts.md - CI/CD: - Overview: next/ai-workspace/ci-cd/overview.md - Configure CI/CD Workflow: next/ai-workspace/ci-cd/configure-ci-cd-workflow.md @@ -921,6 +925,7 @@ plugins: ai-gateway/observability/logging.md: ai-gateway/1.1.0/observability/logging.md ai-gateway/observability/tracing.md: ai-gateway/1.1.0/observability/tracing.md ai-gateway/overview.md: ai-gateway/1.1.0/overview.md + ai-gateway/streaming-responses.md: ai-gateway/1.1.0/streaming-responses.md api-gateway/analytics/analytics-header-filter.md: api-gateway/1.1.0/analytics/analytics-header-filter.md api-gateway/analytics/moesif-analytics.md: api-gateway/1.1.0/analytics/moesif-analytics.md api-gateway/deployment/deploying-apis/bottom-up-api-deployment.md: api-gateway/1.1.0/deployment/deploying-apis/bottom-up-api-deployment.md