diff --git a/en/docs/ai-gateway/next/llm-proxy/multi-provider-routing.md b/en/docs/ai-gateway/next/llm-proxy/multi-provider-routing.md index 78c0d19ec..2a3a386c8 100644 --- a/en/docs/ai-gateway/next/llm-proxy/multi-provider-routing.md +++ b/en/docs/ai-gateway/next/llm-proxy/multi-provider-routing.md @@ -551,4 +551,4 @@ For automatic traffic distribution across models and providers, see: - [Model Round Robin](load-balancing/model-round-robin.md) - [Model Weighted Round Robin](load-balancing/model-weighted-round-robin.md) -AWS Bedrock usage can also be evaluated by the [LLM Cost policy](../../../next/ai-workspace/policies/overview.md#llm-cost). +AWS Bedrock usage can also be evaluated by the [LLM Cost policy](../../../ai-workspace/next/policies/overview.md#llm-cost). diff --git a/en/docs/ai-gateway/next/llm-proxy/quick-start-guide.md b/en/docs/ai-gateway/next/llm-proxy/quick-start-guide.md index b2a992946..3f411b474 100644 --- a/en/docs/ai-gateway/next/llm-proxy/quick-start-guide.md +++ b/en/docs/ai-gateway/next/llm-proxy/quick-start-guide.md @@ -207,7 +207,7 @@ curl -X POST "https://localhost:8443/assistant/chat/completions" \ ## View the LLM provider and proxy in AI Workspace -The gateway syncs the artifacts you deploy on it up to [AI Workspace](../../../next/ai-workspace/overview.md), the control plane for AI traffic across your organization. The OpenAI provider and the `openai-assistant` proxy you deployed above appear there without being re-declared. See [Manage Gateway-deployed AI artifacts in AI Workspace](../../../next/ai-workspace/sync-gateway-created-artifacts.md). +The gateway syncs the artifacts you deploy on it up to [AI Workspace](../../../ai-workspace/next/overview.md), the control plane for AI traffic across your organization. The OpenAI provider and the `openai-assistant` proxy you deployed above appear there without being re-declared. See [Manage Gateway-deployed AI artifacts in AI Workspace](../../../ai-workspace/next/sync-gateway-created-artifacts.md). ## Stopping the Gateway diff --git a/en/docs/ai-gateway/next/mcp-proxy/quick-start-guide.md b/en/docs/ai-gateway/next/mcp-proxy/quick-start-guide.md index 0f8742155..cad522253 100644 --- a/en/docs/ai-gateway/next/mcp-proxy/quick-start-guide.md +++ b/en/docs/ai-gateway/next/mcp-proxy/quick-start-guide.md @@ -154,7 +154,7 @@ http://localhost:8080/everything/mcp ## View the MCP proxy in AI Workspace -The gateway syncs the artifacts you deploy on it up to [AI Workspace](../../../next/ai-workspace/overview.md), the control plane for AI traffic across your organization. The `everything-mcp-v1.0` proxy you deployed above appears there without being re-declared, in the `default` project named in its `project-id` annotation. See [Manage Gateway-deployed AI artifacts in AI Workspace](../../../next/ai-workspace/sync-gateway-created-artifacts.md). +The gateway syncs the artifacts you deploy on it up to [AI Workspace](../../../ai-workspace/next/overview.md), the control plane for AI traffic across your organization. The `everything-mcp-v1.0` proxy you deployed above appears there without being re-declared, in the `default` project named in its `project-id` annotation. See [Manage Gateway-deployed AI artifacts in AI Workspace](../../../ai-workspace/next/sync-gateway-created-artifacts.md). ## Stopping the Gateway diff --git a/en/docs/ai-gateway/next/quick-start-guide.md b/en/docs/ai-gateway/next/quick-start-guide.md index edb288f7a..4fc0f72ff 100644 --- a/en/docs/ai-gateway/next/quick-start-guide.md +++ b/en/docs/ai-gateway/next/quick-start-guide.md @@ -16,7 +16,7 @@ content_type: "quickstart" # Quick Start Guide -This guide takes you from a downloaded distribution to an LLM request routed through the API Platform AI Gateway, then shows you how to govern that gateway from [AI Workspace](../../next/ai-workspace/overview.md), the control plane for AI traffic. It's written for platform administrators and AI developers. +This guide takes you from a downloaded distribution to a large language model (LLM) request routed through the API Platform AI Gateway. It then shows you how to govern that gateway from [AI Workspace](../../ai-workspace/next/overview.md), the control plane for AI traffic. !!! info "Watch the video walkthrough" [Check out this quick start on YouTube](https://youtu.be/p5xBXZWt5GU?rel=0) or watch below. @@ -193,12 +193,12 @@ curl -X POST "https://localhost:8443/assistant/chat/completions" \ ## Govern this gateway from AI Workspace -The gateway you just started serves traffic on its own, and it doesn't have to run alone. [AI Workspace](../../next/ai-workspace/overview.md) is the control plane for AI traffic across your organization: one console for LLM providers, App LLM proxies, MCP proxies, policies such as guardrails and token-based rate limits, and the credentials behind them. Register this gateway with AI Workspace to govern every AI gateway you run from a single place, across every environment. +The gateway you just started serves traffic on its own. [AI Workspace](../../ai-workspace/next/overview.md) is the control plane for AI traffic across your organization. One console manages LLM providers, App LLM proxies, MCP proxies, policies such as guardrails and token-based rate limits, and the credentials behind them. Register this gateway with AI Workspace to govern every AI gateway you run from one place, across every environment. Both directions work, and you can use them together: - **Top-down.** Configure an artifact in AI Workspace, apply policies to it, then deploy it to one or more gateways. -- **Bottom-up.** Keep deploying through the management API, the way this guide does. Every artifact you create on the gateway syncs up to AI Workspace automatically and appears there as a copy the gateway owns, so the OpenAI provider and the `openai-assistant` proxy you deployed above show up without being re-declared. To see what a synced artifact looks like, and what stays editable, see [Manage Gateway-deployed AI artifacts in AI Workspace](../../next/ai-workspace/sync-gateway-created-artifacts.md). +- **Bottom-up.** Keep deploying through the management API, as this guide does. The gateway syncs every artifact you create to AI Workspace automatically, where each one appears as a copy the gateway owns. The OpenAI provider and the `openai-assistant` proxy you deployed above appear there without being re-declared. To see what a synced artifact looks like, and what stays editable, see [Manage Gateway-deployed AI artifacts in AI Workspace](../../ai-workspace/next/sync-gateway-created-artifacts.md). The gateway keeps serving traffic either way. If AI Workspace is unreachable, the gateway carries on and the sync catches up once the connection is restored. @@ -227,4 +227,4 @@ This stops the containers and removes the `controller-data` volume. The next sta - Route to more than one provider, with failover: [Multi-provider routing](./llm-proxy/multi-provider-routing.md) - Add guardrails to a proxy, such as [PII masking](./llm-proxy/guardrails/pii-masking-regex.md) or a [JSON schema guardrail](./llm-proxy/guardrails/json-schema.md) - Expose an MCP server through the gateway: [MCP proxy quick start guide](./mcp-proxy/quick-start-guide.md) -- Govern AI traffic across all your gateways from the control plane: [AI Workspace overview](../../next/ai-workspace/overview.md) +- Govern AI traffic across all your gateways from the control plane: [AI Workspace overview](../../ai-workspace/next/overview.md) diff --git a/en/docs/ai-workspace/1.0.0/about-this-release.md b/en/docs/ai-workspace/1.0.0/about-this-release.md new file mode 100644 index 000000000..ac372b84c --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/about-this-release.md @@ -0,0 +1,230 @@ +--- +title: "About this release" +description: "Features, compatible product versions, key considerations, and known limitations of WSO2 AI Workspace 1.0.0." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/about-this-release/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/about-this-release.md +tags: + - cloud + - ai-workspace + - release-notes +author: WSO2 API Platform Documentation Team +last_updated: 2026-08-05 +content_type: "reference" +--- + +# About this release + +AI Workspace is the control plane for managing how applications access artificial intelligence (AI) services. Platform teams register [AI Gateway runtimes](ai-gateways/setting-up.md) in it, configure large language model (LLM) [providers](llm-providers/overview.md) and [proxies](llm-proxies/overview.md), attach [AI policies](policies/overview.md), and manage [credentials](secrets-management.md). Developers then point their applications and agents at the deployed endpoints. It runs as a distribution you deploy yourself, keeps its own database, and reaches gateways through explicit deployment rather than automatic propagation. + +**WSO2 AI Workspace 1.0.0** is the first AI Workspace release. Every capability listed below is available for the first time, so there is no predecessor to upgrade from. + +For more information on AI Workspace, see the [AI Workspace overview](overview.md). + +## Downloads + +Download the AI Workspace distribution from the [WSO2 API Platform releases page](https://github.com/wso2/api-platform/releases). To run it locally with Docker Compose, follow [Get started with AI Workspace](getting-started.md). + +## New features + +??? note "AI Workspace control plane" + + AI Workspace separates AI configuration from AI traffic. You manage artifacts and policies in the workspace, and the AI Gateway enforces them at request time. + + - **Central configuration**: Manage LLM providers, App LLM proxies, Model Context Protocol (MCP) proxies, policies, and secrets from one console instead of configuring each gateway separately. + - **Explicit deployment**: Changes take effect on live traffic only when you deploy them to a gateway. + - **Deployment tracking**: See which artifacts are deployed to which gateways, deploy one artifact to several gateways, and serve several artifacts from one gateway. + + **[AI Workspace overview](overview.md)** + +??? note "AI Gateway registration and management" + + Register the gateway runtimes that process your AI traffic, then deploy artifacts to them from the workspace. + + - **Token-based registration**: Register a gateway with a registration token that the workspace issues once. + - **Status monitoring**: Track whether each registered gateway is active. + - **Multi-gateway deployment**: Target one or more gateways when you deploy an artifact. + + **[Set up an AI Gateway](ai-gateways/setting-up.md)** + +??? note "LLM providers for seven AI services" + + An LLM provider holds the endpoint and authentication configuration for an upstream AI service, and any number of proxies can reuse it. + + - **Built-in provider support**: Connect OpenAI, Azure OpenAI, Azure AI Foundry, Anthropic, Google Gemini, Mistral AI, and AWS Bedrock. + - **Centralized credentials**: Store upstream API keys as secrets rather than in artifact configuration. + - **Reusable configuration**: Support multiple proxies with a single provider without duplicating credentials. + - **Direct invocation**: If you don't need application-specific controls, call a provider endpoint directly. + - **Inbound authentication with API keys**: The gateway checks an API key on every incoming client request to a deployed provider. The workspace generates each key, shows it once, and sets a 90-day validity period. Send the key in `X-API-Key` by default, or in a header name that suits your software development kit (SDK). Inbound keys are separate from the upstream API key the gateway uses to call the AI service. See [Configure inbound authentication](configure-inbound-auth.md). + - **SDK invocation**: Applications call a deployed provider through its Invoke URL using the OpenAI, Anthropic, Google Gemini, Mistral, Azure OpenAI, and LangChain SDKs. See [Invoke providers and proxies with AI SDKs](using-sdks.md). + + **[LLM providers overview](llm-providers/overview.md)** + +??? note "LLM provider templates for custom services" + + A template is a reusable blueprint that captures the endpoint URL, inbound authentication settings, OpenAPI definition, and token and model mappings for an upstream service. + + - **Built-in templates**: Use the read-only templates shipped for the seven supported services, and enable or disable each one. + - **Custom templates**: Define a template for any AI service that has no built-in template, from scratch or as a new version of a built-in template. + - **Versioning**: Keep multiple versions of a custom template, and see the highest-numbered version on each template card. + - **Provider type selector integration**: Custom templates appear alongside built-in providers when you add a provider. + + **[LLM provider templates overview](llm-provider-templates/overview.md)** + +??? note "App LLM proxies" + + If a specific generative AI (GenAI) application or agent needs its own controls, an App LLM proxy adds an application-facing endpoint on top of a provider. + + - **Isolated configuration**: Give each application, agent, team, or environment its own guardrails, access keys, and exposed resources. + - **Resource control**: Choose which API paths the proxy exposes, and enable or disable them without changing the upstream provider. + - **Provider switching**: If the replacement provider preserves the client-facing contract, swap the underlying provider without client changes. + - **Inbound authentication with API keys**: Require an API key that the workspace generates for that proxy, independently of the keys on the underlying provider. The same header name and 90-day validity rules apply. See [Configure inbound authentication](configure-inbound-auth.md). + - **SDK invocation**: Applications call a deployed proxy with the same AI SDKs and the same code path as a provider. The Invoke URL is the only difference. See [Invoke providers and proxies with AI SDKs](using-sdks.md). + + **[App LLM proxies overview](llm-proxies/overview.md)** + +??? note "MCP proxies" + + An MCP proxy routes requests through the gateway to an upstream MCP server, so MCP clients call a managed endpoint instead of the server directly. + + - **Managed MCP endpoints**: Expose an upstream MCP server through a gateway endpoint over streamable HTTP. + - **Security**: Authenticate and authorize the callers of MCP traffic. + - **Policy enforcement**: Attach policies that control the MCP traffic passing through the gateway. + - **Observability**: See which tools and servers are called, and which calls fail. + + **[MCP proxies overview](mcp-proxies/overview.md)** + +??? note "AI policies for content, traffic, and cost" + + Policies run on the gateway at request time. Attach a policy to a provider as a baseline, or to a proxy for one application or agent. + + - **Guardrails** inspect and act on request and response content: + + - **Content safety**: Azure content safety moderation, NVIDIA NeMo Guard content safety classification, and AWS Bedrock guardrails. + - **Prompt protection**: Semantic prompt guard for similarity-based allow and block lists, and IBM Granite Guardian for prompt injection and jailbreak detection. + - **PII protection**: Regex-based masking of personally identifiable information (PII), with restoration in the response. + - **Validation**: Word count, sentence count, content length, JSON schema, regex, and URL guardrails. + - **Tool filtering**: Semantic tool filtering, which limits the tools exposed to a model by relevance to the user query. + - See [Guardrail policies](policies/overview.md#guardrails). + + - **Rate limiting** caps several different measures of traffic, because many AI services bill per token: + + - **Rate limit: basic**: Caps request count within a time window. + - **Rate limit: advanced**: Caps request count with multi-dimensional and weighted quotas. Offers a choice of the generic cell rate algorithm (GCRA) or fixed window, and in-memory or Redis counters. + - **Token-based rate limit**: Caps prompt, completion, or total tokens, independently or in combination. + - **LLM cost and LLM cost-based rate limit**: Calculate the monetary cost of each call, and cap spend in US dollars (USD). + - **Built-in provider limits**: Cap requests and tokens from the **Rate Limiting** tab of a provider without attaching a policy. + - See [Rate limiting policies](policies/overview.md#rate-limiting). + + - **Traffic, prompt, and provider transformations** shape how requests are routed, composed, and translated: + + - **Model routing**: Model round robin and model weighted round robin distribute requests across models. + - **Header-based routing**: The LLM header router selects the target provider from a request header, so one OpenAI-shaped endpoint routes requests to several providers. + - **Prompt handling**: Prompt decorator, prompt template, and prompt compressor. + - **Response handling**: Semantic caching for semantically similar requests, and the respond policy for mocking and short-circuit logic. + - **Provider transformation**: Translate an OpenAI Chat Completions request into the Anthropic, Azure OpenAI, AWS Bedrock Converse, Gemini, or Mistral API shape, and translate the response back. + - See [Traffic management and prompt policies](policies/overview.md#traffic-management-and-prompt-policies). + + **[AI policies overview](policies/overview.md)** + +??? note "Custom AI policies" + + When no built-in policy covers a requirement, write your own and run it on the gateway. + + - **Policy authoring**: Define a policy with its own version and configuration schema. + - **Gateway packaging**: Build a gateway image that includes your policies. + - **Attachment**: Attach a custom policy to a provider or proxy the same way as a built-in policy. + + **[Write an AI policy](policies/writing-an-ai-policy.md)** + +??? note "Secrets management" + + Secrets keep raw API keys, tokens, and passwords out of artifact configuration. + + - **Encryption at rest**: Secrets are encrypted with AES-GCM-256. Plaintext values are never written to the database and never returned in an API response, including the creation response. + - **Placeholder references**: Reference a secret from LLM provider configurations, MCP proxy configurations, and API backend settings, and the gateway resolves it at request time. + - **Automatic secret creation**: Upstream API keys entered in the AI Workspace user interface (UI) become secrets. AI Workspace replaces each key with a placeholder before it saves the artifact. + - **Rotation without redeployment**: Update the secret value by handle, and referencing artifacts need no change. + + **[Secrets management](secrets-management.md)** + +??? note "Management of gateway-deployed AI artifacts" + + Artifacts created directly on a gateway sync up to AI Workspace, which reverses the usual flow from AI Workspace to the gateway. + + - **Automatic sync**: Automatic sync is enabled by default. LLM provider templates, LLM providers, LLM proxies, and MCP proxies created on a gateway appear in the workspace. + - **Gateway ownership**: Deployment fields stay read-only in the workspace, because the gateway owns them. + - **Editable metadata**: Descriptions, documentation, OpenAPI definitions, and template connection details remain editable. + - **Independent operation**: If AI Workspace is unavailable, these artifacts keep serving traffic. + + **[Sync gateway-created artifacts](sync-gateway-created-artifacts.md)** + +??? note "Git-based CI/CD with the `ap` CLI" + + Git-based continuous integration and continuous delivery (CI/CD) lets you manage AI Workspace artifacts as version-controlled project files. You run each step with the `ap` command-line interface (CLI) instead of making changes in the UI. + + - **Declarative project files**: Describe an artifact in `metadata.yaml`, `runtime.yaml`, and `definition.yaml`, and commit them to source control. + - **Supported artifact types**: LLM providers, App LLM proxies, and MCP proxies. + - **Validate and apply**: Validate an artifact with `ap ai-workspace build`, apply it with `ap ai-workspace apply`, and deploy the runtime artifact with `ap gateway apply -f runtime.yaml`. + - **Synchronous operations**: Each step runs from the project files, so the control plane and the gateway runtime don't depend on each other during artifact application. + + **[Git-based CI/CD overview](ci-cd/overview.md)** + +??? note "Insights through Moesif" + + The gateway runtime publishes AI traffic telemetry to [Moesif](https://www.moesif.com/), an API analytics platform. + + - **Published telemetry**: Requests, token usage, latency, cost, and guardrail events. + - **Single configuration step**: Set the `MOESIF_KEY` environment variable on the gateway runtime, and no workspace change is required. + - **Insights page**: Select **Insights** in the AI Workspace left navigation menu to open your Moesif workspace. + + **[Insights through Moesif](insights.md)** + +??? note "Deployment configuration" + + AI Workspace and the Platform API read their settings from a single `config.toml` file. + + - **Interpolation tokens**: Pull values in from environment variables and mounted files, so sensitive values stay out of configuration files. + - **Setup script**: Provision the Transport Layer Security (TLS) certificate, JSON Web Token (JWT) signing keypair, encryption keys, session secret, and admin credentials with `./scripts/setup.sh`. The script stops without generating weaker values. + - **Configurable ports**: Remap the published host port, or change the port each service listens on. + - **Database options**: Store artifacts in SQLite, which is the default, PostgreSQL, or Microsoft SQL Server, with TLS and connection pool settings. + + **[Deployment configuration reference](setting-up/configuration.md)** + +??? note "User authentication modes" + + AI Workspace supports two sign-in modes, and a running instance uses one at a time. + + - **File-based authentication**: Validate credentials against a hashed user list in configuration, with no identity provider required, for local use and demos. + - **Identity provider authentication**: Delegate login to an OpenID Connect (OIDC) identity provider for production. + - **Role assignment**: Assign roles per user to control what each person can do. + + **[User authentication overview](setting-up/authentication/overview.md)** + +## Compatible product versions + +AI Workspace deploys artifacts to the AI Gateway and shares a control plane with the API Portal. The following table lists the product versions tested with this release: + +| Product | Compatible version | +|---------|--------------------| +| WSO2 AI Gateway | 1.2.0 | +| WSO2 API Portal | 1.0.0 | + +## Key changes + +None. There is no earlier release to migrate a deployment from. + +## Improvements + +None. This is the first release, so there is no earlier behavior to improve on. + +## Deprecations + +None. + +## Fixed issues + +None recorded against a released version, since this is the first release. + +## Known issues + +- [Open AI Workspace issues in GitHub](https://github.com/wso2/api-platform/issues?q=is%3Aissue%20state%3Aopen%20label%3AArea%2FAIWorkspace) diff --git a/en/docs/ai-workspace/1.0.0/ai-gateways/setting-up.md b/en/docs/ai-workspace/1.0.0/ai-gateways/setting-up.md new file mode 100644 index 000000000..6ec9efcfa --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/ai-gateways/setting-up.md @@ -0,0 +1,320 @@ +--- +title: "Set up an AI Gateway in AI Workspace" +description: "Register an AI Gateway in AI Workspace, connect the runtime with a registration token, and manage existing gateways." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/ai-gateways/setting-up/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/ai-gateways/setting-up.md +tags: + - cloud + - ai-workspace + - ai-gateways +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-23 +content_type: "how-to" +--- + +# Set up an AI Gateway + +An AI Gateway is the runtime component that processes and routes requests between your applications and large language model (LLM) providers. Create and manage AI gateways in AI Workspace, then deploy your LLM providers and proxies to them. + +## Prerequisites + +- A user whose token carries the scopes these steps need: + + - `ap:gateway:read` to view gateways. + - `ap:gateway:manage` to add, edit, or delete a gateway. + - `ap:gateway:token:manage` to issue a registration token. `ap:gateway:manage` covers the token operations on its own. + + The built-in [role-to-scope mapping](../setting-up/authentication/overview.md) assigns these scopes as follows: + + - `ap_admin` and `ap_operator` grant all three scopes. + - `ap_viewer` and `ap_publisher` grant `ap:gateway:read` only, so they can view gateways but not change them. + +## View AI gateways + +1. Navigate to **AI Gateways** in the left navigation menu. + +The AI Gateways page displays a list of all configured gateways with the following details: + +| Column | Description | +|--------|-------------| +| **Name** | The gateway name | +| **Description** | A brief description of the gateway | +| **Status** | The gateway status, either Active or Not Active | +| **Last Updated** | Timestamp of the most recent change | +| **Actions** | Edit and delete options | + +## Add an AI gateway + +1. Navigate to **AI Gateways** in the left navigation menu. + +2. Click **+ Add AI Gateway**. + +3. Fill in the gateway details: + + - **Name** (required): a unique name for the gateway, for example `production-gateway` or `dev-gateway`. + - **Description** (optional): a brief description that identifies the gateway's purpose. + - **URL** (required): the gateway URL, for example `https://localhost:8443`. This is the endpoint where the gateway runtime is accessible. + + The Add AI Gateway form collects the name, description, and URL: + + ![Add AI Gateway form showing Name, Description, URL pre-filled with localhost:8443](../../../assets/img/ai-gateway/standalone-ai-workspace/ai-gateway/ai-gateway-form.png) + +4. Click **Add Gateway** to create the gateway. + +5. The gateway detail page opens showing the gateway name, status (**Not Active** initially), vhost, and creation timestamp. + +## Set up the gateway runtime + +After creating an AI gateway, you need to set up the gateway runtime. The detail page provides a **Get Started** section with setup instructions. + +### Gateway registration token + +A **Gateway Registration Token** is displayed at the top of the Get Started section. This token is required to connect your gateway runtime to the control plane. + +!!! danger "Important" + The token is shown only once. Save it securely before you leave this page. + +!!! tip "Lost your token?" + The registration token is single-use. If you need to reconfigure the gateway, click the **Reconfigure** button to generate a replacement token. The replacement token revokes the old one and disconnects the gateway from the control plane. + +### Installation methods + +The Get Started section provides setup instructions for multiple deployment options. + +!!! note "Where the control plane address comes from" + Every method below needs two values from the **Get Started** section: the control plane address the gateway connects to, and this gateway's registration token. + + Self-hosted AI Workspace fills that address into the commands it shows from `[ai_workspace.gateway] controlplane_host` in its own `config.toml`. The key is display-only, and AI Workspace never connects to it. The address has to be reachable from the machine running the gateway, not from the workspace. If the commands carry an address your gateway can't reach, correct that key and reload the page rather than editing the value into the gateway. For the Helm method, put the hostname in `controlPlane.host` and any port in `controlPlane.port`. See [Two keys that aren't interchangeable](../setting-up/ports.md#two-keys-that-arent-interchangeable). + +!!! note "Gateway version" + AI Workspace works with **gateway v1.2 and above**. Those gateways provision their keys and certificates with `./scripts/setup.sh` and take their configuration from `api-platform.env`, which Compose loads through the `env_file:` directive. The **Get Started** section offers the gateway versions AI Workspace supports and shows the commands for the version you register. + +=== "Quick start" + **Prerequisites:** + + - cURL installed + - unzip installed + - Docker installed and running + + **Step 1: Download the gateway** + + Run this command in your terminal to download the gateway: + + ```bash + curl -sLO https://github.com/wso2/api-platform/releases/download/ai-gateway/v1.2.0/wso2apip-ai-gateway-1.2.0.zip && \ + unzip wso2apip-ai-gateway-1.2.0.zip + ``` + + **Step 2: Set up the gateway** + + Run the one-time setup script. It provisions the AES-256 at-rest encryption key, the router HTTPS listener certificate, the gateway-controller admin credentials, and `api-platform.env` — all required before the first start (the gateway has no demo mode and fails closed if a required key, certificate, or credential is missing). The admin password is printed once — copy it: + + ```bash + cd wso2apip-ai-gateway-1.2.0 && ./scripts/setup.sh + ``` + + On Windows, use the PowerShell setup script instead — it takes the same flags and provisions the same files: + + ```powershell + cd wso2apip-ai-gateway-1.2.0 + powershell -ExecutionPolicy Bypass -File .\scripts\setup.ps1 + ``` + + **Step 3: Configure the gateway** + + Append the control plane host and your registration token to `api-platform.env`: + + ```bash + cat >> api-platform.env << 'ENVFILE' + APIP_GW_CONTROLLER_CONTROLPLANE_HOST= + APIP_GW_CONTROLLER_CONTROLPLANE_TOKEN= + ENVFILE + ``` + + Replace both placeholders with the values the Get Started section shows: the control plane address, as `host:port` with no scheme, and this gateway's registration token. + + **Step 4: Start the gateway** + + Start the gateway. `api-platform.env` is loaded automatically via the Compose `env_file:` directive: + + ```bash + docker compose up + ``` + +=== "Virtual machine" + **Prerequisites:** + + - cURL installed + - unzip installed + - A Docker-compatible container runtime such as: + - Docker Desktop (Windows / macOS) + - Rancher Desktop (Windows / macOS) + - Colima (macOS) + - Docker Engine + Compose plugin (Linux) + + Ensure docker and docker compose commands are available: + + ```bash + docker --version + docker compose version + ``` + + **Step 1: Download the gateway** + + Run this command in your terminal to download the gateway: + + ```bash + curl -sLO https://github.com/wso2/api-platform/releases/download/ai-gateway/v1.2.0/wso2apip-ai-gateway-1.2.0.zip && \ + unzip wso2apip-ai-gateway-1.2.0.zip + ``` + + **Step 2: Set up the gateway** + + Run the one-time setup script. It provisions the AES-256 at-rest encryption key, the router HTTPS listener certificate, the gateway-controller admin credentials, and `api-platform.env` — all required before the first start (the gateway has no demo mode and fails closed if a required key, certificate, or credential is missing). The admin password is printed once — copy it: + + ```bash + cd wso2apip-ai-gateway-1.2.0 && ./scripts/setup.sh + ``` + + On Windows, use the PowerShell setup script instead — it takes the same flags and provisions the same files: + + ```powershell + cd wso2apip-ai-gateway-1.2.0 + powershell -ExecutionPolicy Bypass -File .\scripts\setup.ps1 + ``` + + **Step 3: Configure the gateway** + + Append the control plane host and your registration token to `api-platform.env`: + + ```bash + cat >> api-platform.env << 'ENVFILE' + APIP_GW_CONTROLLER_CONTROLPLANE_HOST= + APIP_GW_CONTROLLER_CONTROLPLANE_TOKEN= + ENVFILE + ``` + + Replace both placeholders with the values the Get Started section shows: the control plane address, as `host:port` with no scheme, and this gateway's registration token. The token is single-use — if you need to reconfigure, click **Reconfigure** to generate a replacement registration token (this revokes the old token and disconnects the gateway). + + **Step 4: Start the gateway** + + Start the gateway. `api-platform.env` is loaded automatically via the Compose `env_file:` directive: + + ```bash + docker compose up + ``` + +=== "Docker" + **Prerequisites:** + + - cURL installed + - unzip installed + + **Step 1: Download the gateway** + + Run this command in your terminal to download the gateway: + + ```bash + curl -sLO https://github.com/wso2/api-platform/releases/download/ai-gateway/v1.2.0-rc2/wso2apip-ai-gateway-1.2.0-rc2.zip && \ + unzip wso2apip-ai-gateway-1.2.0-rc2.zip + ``` + + **Step 2: Set up the gateway** + + Run the one-time setup script. It provisions the AES-256 at-rest encryption key, the router HTTPS listener certificate, the gateway-controller admin credentials, and `api-platform.env` — all required before the first start (the gateway has no demo mode and fails closed if a required key, certificate, or credential is missing). The admin password is printed once — copy it: + + ```bash + cd wso2apip-ai-gateway-1.2.0 && ./scripts/setup.sh + ``` + + On Windows, use the PowerShell setup script instead — it takes the same flags and provisions the same files: + + ```powershell + cd wso2apip-ai-gateway-1.2.0 + powershell -ExecutionPolicy Bypass -File .\scripts\setup.ps1 + ``` + + **Step 3: Configure the gateway** + + Append the control plane host and your registration token to `api-platform.env`: + + ```bash + cat >> api-platform.env << 'ENVFILE' + APIP_GW_CONTROLLER_CONTROLPLANE_HOST= + APIP_GW_CONTROLLER_CONTROLPLANE_TOKEN= + ENVFILE + ``` + + Replace both placeholders with the values the Get Started section shows: the control plane address, as `host:port` with no scheme, and this gateway's registration token. The token is single-use — if you need to reconfigure, click **Reconfigure** to generate a replacement registration token (this revokes the old token and disconnects the gateway). + + **Step 4: Start the gateway** + + Start the gateway. `api-platform.env` is loaded automatically via the Compose `env_file:` directive: + + ```bash + docker compose up + ``` + +=== "Kubernetes" + **Prerequisites:** + + - cURL installed + - unzip installed + - Kubernetes 1.32+ + - Helm 3.18+ + + The registration token is a one-time generated token for this gateway. If you need to install or update the gateway chart again, first reconfigure this gateway to generate a replacement registration token. Reconfiguring revokes the previous token. + + **Create the encryption key secret** + + At-rest encryption is mandatory and fail-closed — nothing is auto-generated, and the chart doesn't render without an AES-256 key Secret. Create it in the install namespace before installing the chart: + + ```bash + openssl rand 32 > default-aesgcm256-v1.bin + kubectl create secret generic gateway-encryption-keys \ + --from-file=default-aesgcm256-v1.bin=default-aesgcm256-v1.bin + rm default-aesgcm256-v1.bin # don't leave the plaintext key on disk + ``` + + **Install the chart** + + Run this command to install the gateway chart with the encryption key and control plane configurations: + + ```bash + helm install gateway oci://ghcr.io/wso2/api-platform/helm-charts/gateway --version 1.2.0 \ + --set gateway.controller.encryptionKeys.enabled=true \ + --set gateway.controller.encryptionKeys.secretName=gateway-encryption-keys \ + --set gateway.controller.controlPlane.host="" \ + --set gateway.controller.controlPlane.port= \ + --set gateway.controller.controlPlane.token.value="" + ``` + + Use the Helm chart version that matches the gateway version shown on this page's **Get Started** section, not necessarily `1.2.0`. + + Replace the placeholders with the values the Get Started section shows. Split the control plane address across the two flags: the hostname in `controlPlane.host`, its port in `controlPlane.port`. `` is this gateway's registration token. + +Once the gateway runtime is running and connected, the gateway status changes from **Not Active** to **Active**. + +## Manage AI gateways + +### Edit a gateway + +1. In the AI Gateways list, click the edit icon next to the gateway you want to modify. + +2. Update the gateway details as needed. + +3. Click **Save** to apply the changes. + +### Delete a gateway + +1. In the AI Gateways list, click the delete icon next to the gateway you want to remove. + +2. Confirm the deletion when prompted. + +!!! danger "Irreversible action" + Deleting a gateway is permanent, and it undeploys every provider and proxy on that gateway immediately. + +## Next steps + +- [Configure an LLM provider](../llm-providers/configure-provider.md): set up an LLM provider and deploy it to your gateway +- [Configure an App LLM proxy](../llm-proxies/configure-proxy.md): create a specialized proxy for a GenAI application or agent and deploy it to your gateway diff --git a/en/docs/ai-workspace/1.0.0/ci-cd/configure-ci-cd-workflow.md b/en/docs/ai-workspace/1.0.0/ci-cd/configure-ci-cd-workflow.md new file mode 100644 index 000000000..cffc8e3cc --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/ci-cd/configure-ci-cd-workflow.md @@ -0,0 +1,432 @@ +--- +title: "Configure an AI Workspace CI/CD workflow" +description: "Use the ap CLI to validate and apply AI Workspace artifacts such as LLM providers, App LLM proxies, and MCP proxies from project files." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/ci-cd/configure-ci-cd-workflow/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/ci-cd/configure-ci-cd-workflow.md +tags: + - cloud + - ai-workspace + - ci-cd +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-08 +content_type: "how-to" +--- + +# Configure an AI Workspace CI/CD workflow + +This guide shows how to validate and apply AI Workspace artifacts from project files using the `ap` CLI. Use this workflow when you want **LLM providers**, **App LLM proxies**, or **MCP proxies** to be reviewed, versioned, and promoted through a Git-based release process. + +## Prerequisites + +!!! info "Before you begin" + - Install the `ap` CLI and make it available on your `PATH`. + - Configure access to the target AI Workspace. + - Configure access to the target gateway. + - Start the target AI Gateway with deployment synchronization disabled. + - For App LLM proxy and MCP proxy artifacts, get the API Platform project ID that the artifact belongs to. + - Keep secrets out of project files. Use platform-managed secrets or environment variables supported by the CLI. + +## Configure the AI Workspace connection + +Add the AI Workspace connection once, then set it as the active workspace for later commands. + +=== "Template" + ```shell + ap ai-workspace add \ + --display-name \ + --server \ + --auth + + ap ai-workspace use --display-name + ``` + +=== "Example" + ```shell + ap ai-workspace add \ + --display-name dev-aiws \ + --server https://ai-workspace.example.com \ + --auth oauth + + ap ai-workspace use --display-name dev-aiws + ``` + +The AI Workspace CLI supports `basic`, `oauth`, and `api-key` authentication. Environment variables override credentials stored in the CLI configuration. + +| Auth type | Environment variables | +|-----------|-----------------------| +| `basic` | `WSO2AP_AIWORKSPACE_USERNAME`, `WSO2AP_AIWORKSPACE_PASSWORD` | +| `oauth` | `WSO2AP_AIWORKSPACE_TOKEN` | +| `api-key` | `WSO2AP_AIWORKSPACE_API_KEY` | + +## Configure the gateway connection + +Configure and select the gateway connection. + +=== "Template" + ```shell + ap gateway add \ + --display-name \ + --server + + ap gateway use --display-name + ``` + +=== "Example" + ```shell + ap gateway add \ + --display-name dev-gw \ + --server https://gateway.example.com + + ap gateway use --display-name dev-gw + ``` + +## Start the gateway for declarative CI/CD + +When using this CI/CD flow, start the AI Gateway with control-plane deployment synchronization disabled. + +```toml +[controller.controlplane] +deployment_sync_enabled = false +``` + +In **declarative CI/CD**, the AI Workspace artifact operation and the gateway runtime operation run **independently of each other**. Both read from the same project files. Each operation runs **synchronously**. Neither one relies on output from another service or on **asynchronous gateway-to-control-plane synchronization**. Disabling deployment synchronization keeps the gateway from synchronizing deployed artifacts back to AI Workspace while CI/CD owns the artifact lifecycle. + +## Create or update the project + +Create a project if you don't already have one. + +=== "Template" + ```shell + ap project init \ + --display-name \ + --type \ + --version \ + --context + + cd + ``` + +=== "Example" + ```shell + ap project init \ + --display-name wso2-claude-proxy \ + --type LLM-Proxy \ + --version v1.0 \ + --context /customer-support + + cd wso2-claude-proxy + ``` + +The project contains the files used by the AI Workspace build and apply commands. + +```text +wso2-claude-proxy/ +|-- metadata.yaml +|-- runtime.yaml +|-- definition.yaml +|-- docs/ +|-- tests/ +`-- .api-platform/ + `-- config.yaml +``` + +## Author the AI Workspace artifact + +Update `metadata.yaml`, `runtime.yaml`, and `definition.yaml` for the AI Workspace artifact you want to publish. + +The artifact kind in `metadata.yaml` must match the artifact kind in `runtime.yaml` after the `Metadata` suffix is removed. + +| Artifact | `metadata.yaml` kind | `runtime.yaml` kind | +|----------|----------------------|---------------------| +| LLM provider | `LlmProviderMetadata` | `LlmProvider` | +| App LLM proxy | `LlmProxyMetadata` | `LlmProxy` | +| MCP proxy | `McpMetadata` | `Mcp` | + +For example, an App LLM proxy can define its metadata and intended gateways in `metadata.yaml`. + +```yaml +kind: LlmProxyMetadata +metadata: + name: wso2-claude-proxy +spec: + displayName: Customer Support Proxy + version: v1.0 + associatedGateways: + - id: gw-dev + configurations: + host: dev-gateway.example.com +``` + +The corresponding `runtime.yaml` uses the same artifact name and the matching runtime kind. + +```yaml +kind: LlmProxy +metadata: + name: wso2-claude-proxy +spec: + context: /wso2-claude-proxy + description: Proxy endpoint for the customer support assistant. + provider: + id: openai-provider +``` + +Add the OpenAPI or MCP capability definition to `definition.yaml`. The AI Workspace build requires this file for all supported artifact types. + +## Validate the artifact + +Run `ap ai-workspace build` to validate the project files. If you run the command from the project root, you can omit `-f`. + +=== "Template" + ```shell + # From the project root + ap ai-workspace build + + # From another directory + ap ai-workspace build -f + ``` + +=== "Example" + ```shell + # From the project root + ap ai-workspace build + + # From another directory + ap ai-workspace build -f /path/to/wso2-claude-proxy + ``` + +The build command validates that: + +- `metadata.yaml`, `runtime.yaml`, and `definition.yaml` are available +- the metadata and runtime kinds match +- the metadata names match +- all configured paths remain inside the project directory + +## Deploy the runtime artifact to a gateway + +Apply `runtime.yaml` to the selected gateway so the runtime changes described in the project files are propagated to the gateway in a declarative manner. + +=== "Template" + ```shell + ap gateway apply -f /runtime.yaml + ``` + +=== "Example" + ```shell + ap gateway apply -f /path/to/wso2-claude-proxy/runtime.yaml + ``` + +This deploys or updates the runtime artifact on the active gateway. The AI Workspace apply step is separate and can be run before or after the gateway apply step in the same pipeline, depending on how you structure promotion for the environment. + +## Apply the artifact to AI Workspace + +Run `ap ai-workspace apply` from the project root, or pass the project directory with `-f`. + +For LLM providers: + +=== "Template" + ```shell + # From the project root + ap ai-workspace apply + + # From another directory + ap ai-workspace apply -f + ``` + +=== "Example" + ```shell + # From the project root + ap ai-workspace apply + + # From another directory + ap ai-workspace apply -f /path/to/openai-provider + ``` + +For App LLM proxies and MCP proxies, include the project ID. + +=== "Template" + ```shell + # From the project root + ap ai-workspace apply --project-id + + # From another directory + ap ai-workspace apply \ + -f \ + --project-id + ``` + +=== "Example" + ```shell + # From the project root + ap ai-workspace apply --project-id customer-support-project + + # From another directory + ap ai-workspace apply \ + -f /path/to/wso2-claude-proxy \ + --project-id customer-support-project + ``` + +The apply command validates the project, builds the request payload in memory, and creates or updates the artifact in AI Workspace. + +| Runtime kind | AI Workspace endpoint | +|--------------|-----------------------| +| `LlmProvider` | `/llm-providers` | +| `LlmProxy` | `/llm-proxies` | +| `Mcp` | `/mcp-proxies` | + +Create and update use the same command. If an artifact with the same `metadata.name` already exists, `apply` updates it. Otherwise, it creates a new artifact. + +## Use environment-specific values + +Use `ENV_CLI_` placeholders for values that differ between environments, such as upstream URLs, hosts, model names, or project IDs. + +```yaml +spec: + upstream: + url: ${ENV_CLI_UPSTREAM_URL} +``` + +Provide the values with an env file during apply. + +=== "Template" + ```shell + ap ai-workspace apply \ + --env-file \ + --project-id + ``` + +=== "Example" + ```shell + ap ai-workspace apply \ + --env-file ./values.env \ + --project-id customer-support-project + ``` + +For `ap ai-workspace apply`, the CLI resolves placeholders from the file passed with `--env-file`, the project's `.env` file, or the process environment. Apply fails if a referenced placeholder has no value. + +`ap gateway apply` takes a single `runtime.yaml` rather than a project directory, and the examples in this guide pass no env file to it. Run `ap gateway apply --help` for the placeholder options your CLI version supports. If it offers none, resolve the placeholders in `runtime.yaml` in your pipeline before the apply step, so the gateway receives a file with no unresolved `ENV_CLI_` references. + +!!! warning "Don't store secrets in project files" + `ENV_CLI_` placeholders are intended for environment-specific configuration values that are sent to AI Workspace. Don't use them for API keys, tokens, or other secrets. Use platform-managed secrets for sensitive values. + +## Example pipeline + +The following example validates an App LLM proxy or MCP proxy project, applies the runtime artifact to the selected gateway, and applies the AI Workspace artifact to the selected workspace. + +=== "Template" + ```shell + ap gateway use --display-name "" + ap ai-workspace use --display-name "" + + ap ai-workspace build -f + + ap gateway apply -f /runtime.yaml + + ap ai-workspace apply \ + -f \ + --project-id \ + --env-file + ``` + +=== "Example" + ```shell + ap gateway use --display-name "dev-gw" + ap ai-workspace use --display-name "dev-aiws" + + ap ai-workspace build -f /path/to/wso2-claude-proxy + + ap gateway apply -f /path/to/wso2-claude-proxy/runtime.yaml + + ap ai-workspace apply \ + -f /path/to/wso2-claude-proxy \ + --project-id customer-support-project \ + --env-file /path/to/wso2-claude-proxy/values.env + ``` + +For an LLM provider project, omit `--project-id`. + +=== "Template" + ```shell + ap ai-workspace apply \ + -f \ + --env-file + ``` + +=== "Example" + ```shell + ap ai-workspace apply \ + -f /path/to/openai-provider \ + --env-file /path/to/openai-provider/values.env + ``` + +## Verify the applied artifact + +Use the AI Workspace get or list commands to verify the result. + +=== "Template" + ```shell + # List LLM providers + ap ai-workspace llm-provider list + + # List App LLM proxies in a project + ap ai-workspace app-llm-proxy list --project-id + + # List MCP proxies in a project + ap ai-workspace mcp-proxy list --project-id + ``` + +=== "Example" + ```shell + # List LLM providers + ap ai-workspace llm-provider list + + # List App LLM proxies in a project + ap ai-workspace app-llm-proxy list --project-id customer-support-project + + # List MCP proxies in a project + ap ai-workspace mcp-proxy list --project-id customer-support-project + ``` + +## AI Workspace CLI commands + +The following table summarizes the AI Workspace commands available in the `ap` CLI. + +| Command | Description | +|---------|-------------| +| `ap ai-workspace add` | Add an AI Workspace connection to the CLI configuration. | +| `ap ai-workspace list` | List configured AI Workspace connections. | +| `ap ai-workspace remove` | Remove an AI Workspace connection from the CLI configuration. | +| `ap ai-workspace use` | Set the active AI Workspace connection. | +| `ap ai-workspace current` | Show the active AI Workspace connection. | +| `ap ai-workspace build` | Validate the AI Workspace artifact files in a project. | +| `ap ai-workspace apply` | Create or update an AI Workspace artifact from project files. | +| `ap ai-workspace llm-provider list` | List LLM providers in the active organization. | +| `ap ai-workspace llm-provider get` | Get a single LLM provider by ID, or list providers when no ID is provided. | +| `ap ai-workspace llm-provider delete` | Delete an LLM provider by ID. | +| `ap ai-workspace app-llm-proxy list` | List App LLM proxies in a project. | +| `ap ai-workspace app-llm-proxy get` | Get a single App LLM proxy by ID, or list proxies when no ID is provided. | +| `ap ai-workspace app-llm-proxy delete` | Delete an App LLM proxy by ID. | +| `ap ai-workspace mcp-proxy list` | List MCP proxies in a project. | +| `ap ai-workspace mcp-proxy get` | Get a single MCP proxy by ID, or list proxies when no ID is provided. | +| `ap ai-workspace mcp-proxy delete` | Delete an MCP proxy by ID. | + +Use `--help` with any command to view the supported flags and examples. For example: + +```shell +ap ai-workspace apply --help +ap ai-workspace app-llm-proxy list --help +``` + +## Recommended practices + +- Store the project files in source control. +- Review changes through pull requests before applying them to shared environments. +- Use pipeline variables for target workspace names, gateway names, project IDs, and environment-specific values. +- Keep `metadata.yaml`, `runtime.yaml`, and `definition.yaml` aligned to avoid build failures. +- Avoid changing the same artifact manually in the UI and through CI/CD at the same time. +- Avoid running another synchronization path that updates the same AI Workspace artifact while CI/CD owns it. + +## Next steps + +- [AI Workspace CI/CD overview](overview.md): how the Git-based flow works +- [Configure an LLM provider](../llm-providers/configure-provider.md): create and deploy providers from the UI +- [Manage an App LLM proxy](../llm-proxies/manage-proxy.md): manage an App LLM proxy after you create it diff --git a/en/docs/ai-workspace/1.0.0/ci-cd/overview.md b/en/docs/ai-workspace/1.0.0/ci-cd/overview.md new file mode 100644 index 000000000..c74fc1fc1 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/ci-cd/overview.md @@ -0,0 +1,105 @@ +--- +title: "AI Workspace CI/CD overview" +description: "Manage AI Workspace artifacts such as LLM providers, App LLM proxies, and MCP proxies through a Git-based CI/CD workflow." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/ci-cd/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/ci-cd/overview.md +tags: + - cloud + - ai-workspace + - ci-cd +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-08 +content_type: "overview" +--- + +# AI Workspace CI/CD overview + +AI Workspace CI/CD lets you manage AI Workspace artifacts as version-controlled project files. You apply those files to the AI Workspace control plane with the `ap` CLI. This gives platform teams a repeatable way to create, update, review, and promote AI artifacts without relying only on manual UI changes. + +You can use this flow for: + +- **Large language model (LLM) providers**—reusable connections to model providers such as OpenAI, Anthropic, Azure OpenAI, Gemini, and Mistral AI +- **App LLM proxies**—application-facing proxy endpoints with their own security, guardrails, policies, and provider reference +- **Model Context Protocol (MCP) proxies**—managed proxy endpoints for upstream MCP servers + +## How the CI/CD flow works + +In a CI/CD workflow, a project in source control is the source of truth. The project contains declarative files that describe the AI Workspace artifact and the runtime behavior expected by the gateway. + +The high-level flow is: + +1. Start the AI Gateway with control-plane deployment synchronization disabled. +2. Configure the gateway and AI Workspace connections in the `ap` CLI. +3. Create or update an API Platform project. +4. Commit `metadata.yaml`, `runtime.yaml`, and `definition.yaml` to source control. +5. Validate the AI Workspace artifact with `ap ai-workspace build`. +6. Apply the artifact to AI Workspace with `ap ai-workspace apply`. +7. Deploy the runtime artifact to a gateway with `ap gateway apply -f runtime.yaml` so the declarative runtime changes are propagated to the gateway. + +The AI Workspace control plane and the gateway runtime don't depend on each other during artifact application. Each operation in the flow runs synchronously from the project files, without relying on output from another service or on asynchronous synchronization. AI Workspace stores the artifact and its intended gateway associations, while the gateway runtime receives runtime artifacts through `ap gateway apply`. + +For this declarative flow, start the AI Gateway with deployment synchronization disabled, so the gateway doesn't synchronize deployed artifacts back to AI Workspace: + +```toml +[controller.controlplane] +deployment_sync_enabled = false +``` + +## Project files + +An AI Workspace CI/CD project uses the same project file model as other API Platform CLI workflows. + +| File | Purpose | +|------|---------| +| `metadata.yaml` | Defines the artifact identity and display metadata. For AI Workspace artifacts, it can also include `spec.associatedGateways`. | +| `runtime.yaml` | Defines the runtime behavior, such as context, provider reference, upstream configuration, security, and policies. Running `ap gateway apply -f runtime.yaml` applies this runtime configuration and the artifact to the gateway. | +| `definition.yaml` | Defines the OpenAPI or MCP capability definition used when building the AI Workspace payload. This file is required for all supported AI Workspace artifact types. | +| `.api-platform/config.yaml` | Stores project-level CLI configuration, including the AI Workspace file paths used during build and apply. | + +## Supported artifact types + +The CLI validates that the artifact kind in `metadata.yaml` matches the kind in `runtime.yaml`. + +| Artifact | `metadata.yaml` kind | `runtime.yaml` kind | +|----------|----------------------|---------------------| +| LLM provider | `LlmProviderMetadata` | `LlmProvider` | +| App LLM proxy | `LlmProxyMetadata` | `LlmProxy` | +| MCP proxy | `McpMetadata` | `Mcp` | + +## Gateway associations + +Use `spec.associatedGateways` in `metadata.yaml` to record the gateways an AI Workspace artifact is intended to run on. + +```yaml +kind: LlmProxyMetadata +metadata: + name: customer-support-proxy +spec: + displayName: Customer Support Proxy + version: v1.0 + associatedGateways: + - id: gw-dev + configurations: + host: dev-gateway.example.com + - id: gw-prod + configurations: + host: prod-gateway.example.com +``` + +AI Workspace persists these associations during create and update operations and returns them in artifact list and detail responses. The association records the intended gateway targets; it doesn't create gateway deployment records by itself. + +AI Workspace validates gateway associations within the organization context of the authenticated user. You can't associate a gateway from another organization with the artifact. + +## Benefits of the CI/CD flow + +- **Git-based lifecycle**—Review, version, and promote AI Workspace artifacts through pull requests. +- **Repeatable deployments**—Apply the same project files across environments using pipeline variables. +- **Reduced drift**—Keep UI-managed artifacts aligned with the project files used by CI/CD. +- **One model across artifact types**—Use the same build and apply model for every supported API Platform artifact type. + +## Next steps + +- [Configure an AI Workspace CI/CD workflow](configure-ci-cd-workflow.md): build and apply an AI Workspace artifact with the `ap` CLI +- [Configure an LLM provider](../llm-providers/configure-provider.md): create a provider from the AI Workspace UI +- [Configure an App LLM proxy](../llm-proxies/configure-proxy.md): create an App LLM proxy from the AI Workspace UI +- [Configure an MCP proxy](../mcp-proxies/configure-proxy.md): create an MCP proxy from the AI Workspace UI diff --git a/en/docs/ai-workspace/1.0.0/configure-inbound-auth.md b/en/docs/ai-workspace/1.0.0/configure-inbound-auth.md new file mode 100644 index 000000000..64b6a20a2 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/configure-inbound-auth.md @@ -0,0 +1,63 @@ +--- +title: "Configure inbound authentication" +description: "Control the header name client applications use to send their API key when calling a deployed LLM provider or App LLM proxy." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/configure-inbound-auth/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/configure-inbound-auth.md +tags: + - cloud + - ai-workspace + - authentication +author: WSO2 API Platform Documentation Team +last_updated: 2026-06-22 +content_type: "how-to" +--- + +# Configure inbound authentication + +The **Security** tab on both LLM providers and App LLM proxies controls how client applications authenticate when calling the deployed gateway endpoint. + +*Inbound authentication* is the check the gateway runs on every incoming client request. The credential for that check is an API key your application sends to the gateway. It isn't the upstream API key the gateway uses to call the LLM provider, which you configure in the **Connection** tab. + +## How it works + +When a provider or proxy is deployed, the gateway enforces inbound authentication on every incoming request. Clients must include a valid API key in the request header, under the name you configure in the Security tab. + +The API key itself is generated by the AI Workspace (shown once at creation, valid for 90 days). The Security tab controls **which header name** the client must use to send it. + +## Security tab fields + +| Field | Description | +|-------|-------------| +| **Authentication** | The authentication type. `apiKey` is the only supported option. | +| **API Key**, or **Key name** | The name of the request header that must carry the API key. Defaults to `X-API-Key`. | +| **Key Location**, or **Sent in** | Where clients send the key. `header` is the only supported option. | + +## Default behavior: the X-API-Key header + +Out of the box, the gateway expects the API key in the `X-API-Key` request header: + +```http +X-API-Key: +``` + +Most AI SDKs don't send this header automatically, so add it explicitly when you initialize the client. See [Invoke via SDKs](using-sdks.md) for code examples. + +## Use a custom header name + +Change the key name to match what your SDK sends natively. For example, setting the key name to `Authorization` lets clients pass the key as a bearer token: + +```http +Authorization: Bearer +``` + +SDKs such as the OpenAI SDK and the Mistral SDK send an `Authorization: Bearer` header by default, so with this configuration they need no additional header setup. The Azure OpenAI SDK sends `Authorization: Bearer` only when it authenticates with a managed identity or another token credential. When it authenticates with an API key, it sends the `api-key` header instead, so set the key name to `api-key` for that configuration. + +!!! note + The gateway validates the key value regardless of prefix. It strips a leading `Bearer` prefix before comparing the value against the stored key. + +## Configure the header name + +1. Open your LLM provider or App LLM proxy and go to the **Security** tab. +2. Set the **API Key** or **Key name** field to the header name your application uses. +3. Click **Save**. +4. Click **Deploy to Gateway**. Security changes take effect only after redeployment. diff --git a/en/docs/ai-workspace/1.0.0/genai-applications.md b/en/docs/ai-workspace/1.0.0/genai-applications.md new file mode 100644 index 000000000..8ea53ef84 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/genai-applications.md @@ -0,0 +1,126 @@ +--- +title: "GenAI applications" +description: "Group API keys under a named GenAI application for application-level usage visibility, analytics, and governance." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/genai-applications/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/genai-applications.md +tags: + - cloud + - ai-workspace + - genai-applications +author: WSO2 API Platform Documentation Team +last_updated: 2026-06-22 +content_type: "concept" +--- + +# GenAI applications + +A GenAI application represents a real AI application inside AI Workspace, with the API keys it uses attached to it. This gives you application-level visibility and control instead of tracking usage only at the individual key level. + +Without GenAI applications, usage is tied only to the API keys generated for large language model (LLM) providers or App LLM proxies. AI Workspace then tracks three metrics—models, tokens, and cost—against each key rather than against an application. That makes attribution hard when several developers generate keys for the same project. + +With a GenAI application, you can: + +- Group API keys under a named application +- View usage and analytics at the application level +- Improve governance and accountability for shared GenAI workloads + +## When to use GenAI applications + +Use a GenAI application when: + +- Multiple developers work on the same AI application +- The application uses more than one API key +- You want analytics grouped by application rather than by key alone +- Platform teams need clearer visibility into which applications drive usage + +Examples include customer support copilots, internal knowledge assistants, document analysis apps, or workflow-specific AI agents. + +## How it works + +The typical flow is: + +1. Generate one or more API keys for an LLM provider or App LLM proxy. +2. Create a GenAI application in the AI Workspace. +3. Attach the generated API keys to that application. +4. Invoke the gateway using those mapped keys. +5. View analytics and usage for that application in Insights. + +The same application can have multiple API keys mapped to it. This is useful when different developers, environments, or services within the same project need separate keys while still rolling up usage to one application. + +## Prerequisites + +- At least one configured [LLM provider](llm-providers/configure-provider.md) or [App LLM proxy](llm-proxies/configure-proxy.md) +- At least one generated API key +- Access to the AI Workspace project that holds the GenAI application + +## Create a GenAI application + +1. Navigate to **AI Workspace** in your API Platform dashboard. +2. Open **GenAI Applications** from the left navigation menu. +3. Click **+ Create Application**. +4. Provide the application details: + + - **Name**: A human-readable name for the application + - **Description**: Optional details to identify the app's purpose + +5. Click **Create**. + +The application is created within the current project and becomes the shared representation of that GenAI workload. + +## Attach API keys to an application + +After creating the application, attach the API keys that the application already uses. + +1. Open the GenAI application. +2. Go to the **API Keys** section. +3. Click **Attach API Keys**. +4. Select one or more API keys generated for your LLM providers or App LLM proxies. +5. Save the mapping. + +Only existing keys are mapped. This workflow doesn't create new keys. It links previously generated keys to the application so usage can be attributed correctly. + +## View and manage attached keys + +The GenAI application details page shows the keys mapped to the application. + +For each mapped key, you can view details such as: + +- The associated provider or proxy +- The key status +- The user who created the key +- The creation and update timestamps +- The expiry details, when applicable + +You can remove mappings for keys that should no longer be associated with the application. + +Removing a mapping only detaches the key from that GenAI application. It doesn't delete the underlying key unless you remove it from the provider or proxy separately. + +## Insights for GenAI applications + +Once a mapped key is used to invoke the gateway, the gateway can identify the owning GenAI application and publish analytics accordingly. + +This lets you analyze usage by application, including: + +- Request volume +- Token consumption +- Latency trends +- Error patterns +- Cost and resource usage by application + +This is especially useful for teams that need to understand which GenAI applications are driving traffic and spend. + +See [Insights](insights.md) for more on the analytics experience. + +## Best practices + +- Create one GenAI application per real application or agent workload, not per developer. +- Map all keys used by the same application so analytics stay complete. +- Use clear names such as `Docs Assistant`, `Support Copilot`, or `Invoice Analyzer`. +- Review mappings periodically and remove keys that are no longer in use. +- Combine GenAI applications with App LLM proxies when an application also needs its own authentication, guardrails, or routing behavior. + +## Related + +- [App LLM proxies overview](llm-proxies/overview.md): create application-specific endpoints and controls +- [Insights](insights.md): analyze usage by application +- [Invoke providers and proxies via SDKs](using-sdks.md): use mapped keys when calling providers and proxies diff --git a/en/docs/ai-workspace/1.0.0/getting-started.md b/en/docs/ai-workspace/1.0.0/getting-started.md new file mode 100644 index 000000000..608bef6d8 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/getting-started.md @@ -0,0 +1,192 @@ +--- +title: "Get started with AI Workspace" +description: "Run AI Workspace locally with Docker Compose, create an AI Gateway, configure an LLM provider, and deploy it through the AI Workspace control plane." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/getting-started/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/getting-started.md +tags: + - cloud + - ai-workspace + - quickstart +author: WSO2 API Platform Documentation Team +last_updated: 2026-08-05 +content_type: "quickstart" +--- + +# Get started with AI Workspace + +The AI Workspace lets you manage AI gateways and large language model (LLM) providers. This guide gets AI Workspace running locally with Docker Compose, then walks you through creating your first AI gateway and LLM provider. + +## Prerequisites + +- Install [Docker](https://docs.docker.com/get-docker/) with the Compose plugin, or another Compose-compatible container runtime such as Podman. +- Free up ports **9643** and **9243** on your machine. If either one is taken, see [Change the ports AI Workspace uses](setting-up/ports.md). +- Install `curl` and `unzip`. + +This guide shows commands with `docker compose`. If you use Podman or another Compose-compatible runtime, run the equivalent compose command instead, such as `podman compose up -d`. + +## Step 1: Download AI Workspace + +Run this command in your terminal to download and unzip AI Workspace: + +```bash +curl -sLO https://github.com/wso2/api-platform/releases/download/portals/ai-workspace/v1.0.0/wso2apip-ai-workspace-1.0.0.zip && \ +unzip wso2apip-ai-workspace-1.0.0.zip +``` + +## Step 2: Run the setup script + +```bash +cd wso2apip-ai-workspace-1.0.0 +./scripts/setup.sh +``` + +Run the script once before the first start. The stack never auto-generates keys or certificates. If something a service needs is missing, that service fails closed with a descriptive error rather than starting with a weaker value. + +The script prompts for the admin username and password. Press Enter at each prompt to accept `admin` and a randomly generated password. The script provisions the following: + +| Artifact | Location | Purpose | +|----------|----------|---------| +| Transport Layer Security (TLS) certificate | `resources/certificates/cert.pem` and `key.pem` | Self-signed HTTPS pair shared by the services. | +| RS256 JSON Web Token (JWT) signing keypair | `resources/keys/jwt_private.pem` and `jwt_public.pem` | The Platform API signs login tokens with the private key; AI Workspace and the API Portal verify them with the public key. There's no shared hash-based message authentication code (HMAC) secret. | +| At-rest encryption key | `resources/keys/encryption.key` | The Platform API's 32-byte key for encrypting stored secrets, subscription tokens, and WebSub HMAC secrets. **Retain it** — losing or changing it makes previously-encrypted data unreadable. | +| API Portal encryption key | `resources/keys/api-portal-encryption.key` | Encrypts the API Portal's subscription and webhook secrets at rest. Retain it for the same reason. | +| API Portal session secret | `resources/keys/api-portal-session-secret` | Signs API Portal session cookies. Rotating it only signs users out. | +| Admin credentials | `api-platform.env` | The Platform API's basic-auth admin user: `APIP_CP_ADMIN_USERNAME` plus the bcrypt `APIP_CP_ADMIN_PASSWORD_HASH`. | +| Compose defaults | `.env` | `COMPOSE_PROFILES`, which decides the services a plain `docker compose up` starts, and `COMPOSE_PROJECT_NAME`, which namespaces this copy's containers, networks, and volumes. | + + +!!! warning "Save the printed admin username and password" + The admin password is shown only once, and `api-platform.env` holds only its bcrypt hash. To set a new one, delete both `APIP_CP_ADMIN_USERNAME` and `APIP_CP_ADMIN_PASSWORD_HASH` from `api-platform.env` and rerun `./scripts/setup.sh`. Deleting only one of the two makes the script stop with an error, because a username without its matching hash can never authenticate. + +!!! warning "Don't delete or edit `COMPOSE_PROJECT_NAME`" + The project name is pinned on the first run and never changes afterward — not on a rerun, not under any flag. The stack's data lives in volumes prefixed with it, so a different name starts the stack with an empty database. To choose the name yourself, set `COMPOSE_PROJECT_NAME` in the environment for the first run. + +## Step 3: Start the stack + +```bash +docker compose up +``` + +!!! tip "Port 9643 or 9243 already taken?" + If `docker compose up -d` fails with a port binding error, identify what is already listening on the default ports: + + On macOS or Linux, run: + + ```bash + lsof -nP -iTCP:9643 -sTCP:LISTEN + lsof -nP -iTCP:9243 -sTCP:LISTEN + ``` + + On Windows PowerShell, run: + + ```powershell + Get-NetTCPConnection -State Listen -LocalPort 9643,9243 | Select-Object LocalAddress, LocalPort, OwningProcess + ``` + + Stop the conflicting service if you don't need it. If you need to keep it running, change the host-side `ports:` mapping in `docker-compose.yaml` before you start. For example, use `"9743:9643"` for AI Workspace. Open AI Workspace on the remapped host port in the next step. For example, use `https://localhost:9743` instead of `https://localhost:9643`. See [Change the ports AI Workspace uses](setting-up/ports.md) for the two config keys that need to match. + +## Step 4: Open AI Workspace + +Open `https://localhost:9643` and sign in with the admin credentials that `setup.sh` printed: + +![AI Workspace file-based login window with Username and Password fields](../../assets/img/ai-gateway/standalone-ai-workspace/authentication/filebased-login.png) + +!!! tip "Browser trust warning?" + The generated TLS certificates are self-signed. Click **Advanced > Proceed** to continue, then return to the workspace. + +!!! note "About this login" + These credentials come from file-based authentication, generated by the setup script and stored in your local environment configuration. Use them to try AI Workspace locally. Before you move to a production or shared environment, connect an identity provider to manage user login. See [Authentication in AI Workspace](setting-up/authentication/overview.md). + +## Step 5: Create an AI Gateway + +An AI gateway is the runtime that processes and routes requests between your applications and LLM providers. You need at least one gateway before configuring providers or proxies. + +1. Navigate to **AI Gateways** in the left navigation menu. +2. Click **+ Add AI Gateway**. +3. Fill in the **Name** and **URL**, then click **Add Gateway**. +4. Copy the **Gateway Registration Token** and save it securely straight away—it's shown only once. Then follow the setup instructions to start the gateway runtime. +5. Once connected, the gateway status changes from **Not Active** to **Active**. + +For detailed instructions, see [Set up an AI Gateway](ai-gateways/setting-up.md). + +## Step 6: Configure an LLM provider + +An LLM provider connects AI Workspace to an AI service platform such as OpenAI, Anthropic, or Azure OpenAI. + +1. Navigate to **LLM** > **LLM Providers**. +2. Click **+ Add New Provider** and select your provider type. +3. Fill in the **Name**, **Version**, and **API Key**, then click **Add Provider**. +4. Configure how applications authenticate when they access this provider through the gateway. +5. Click **Deploy to Gateway** and select your active gateway. + +For detailed instructions, see [Configure an LLM provider](llm-providers/configure-provider.md). + +## Rerun the setup script + +Rerunning `./scripts/setup.sh` is safe. By default it fills in only what's missing and never overwrites a value that already exists. The flags change that: + +| Flag | Effect | +|------|--------| +| `--force` | Regenerate the TLS certificate, the JWT keypair, and the API Portal session secret, and rotate the admin credentials. Never touches either encryption key. | +| `--rotate-encryption-key` | Replace `resources/keys/encryption.key` and `resources/keys/api-portal-encryption.key`, even though they exist. Destructive — see the warning below. | +| `--certs-only` | Generate only the TLS certificate. Skips the keys, the admin credentials, and `api-platform.env`. | +| `--profiles=` | Write a different `COMPOSE_PROFILES` value to `.env`, for example `--profiles=platform-api` or `--profiles=platform-api,api-portal`. | + +To rotate a single value by hand, delete it from `api-platform.env`, or delete the file under `resources/certificates` or `resources/keys`, and rerun the script. Don't delete `resources/keys/encryption.key` or `resources/keys/api-portal-encryption.key` this way. Rerunning the script regenerates a missing encryption key without warning, which makes data encrypted under the old key unreadable. To rotate either encryption key, use `--rotate-encryption-key` and read the warning below first. + +!!! warning "Rotating an encryption key destroys encrypted data" + `--rotate-encryption-key` replaces both encryption keys, which makes everything encrypted under the old keys permanently unreadable. That covers stored [AI Workspace secrets](secrets-management.md), subscription tokens, and WebSub HMAC secrets held by the Platform API. It also covers the API Portal's subscription secrets and webhook secrets. At an interactive terminal the script asks you to type `rotate` to confirm; in a non-interactive run, passing the flag is itself the confirmation. Rotating the JWT keypair with `--force` is milder — it only invalidates issued login tokens, so everyone signs in again. + +## Provision the at-rest encryption key manually + +If you don't run `setup.sh`, provision the at-rest encryption key yourself before the first start. It protects [AI Workspace secrets](secrets-management.md), subscription tokens, and WebSub HMAC secrets, and the Platform API refuses to start if it's missing or malformed. Keep it stable across restarts and replicas. + +The key is a single 32-byte AES-256 value, supplied as 64 hex characters or base64. Generate it and write it to the file the container mounts at `/etc/platform-api/keys`. Create the file so that only its owner can read it: + +```sh +(umask 077 && openssl rand -hex 32 > resources/keys/encryption.key) +chmod 600 resources/keys/encryption.key +``` + +Keep the key out of source control, alongside `api-platform.env`. A trailing newline is trimmed on load. The Platform API doesn't read the key from an environment variable directly. It reads the `encryption_key` field in `config.toml`, which pulls the value in through an interpolation token: + +{% raw %} + +```toml +# config.toml - resolved from a mounted key file: +encryption_key = '{{ file "/etc/platform-api/keys/encryption.key" }}' + +# Alternatively, from an environment variable: +# encryption_key = '{{ env "APIP_CP_ENCRYPTION_KEY" }}' +``` + +{% endraw %} + +To use the environment variable form instead, switch the token to {% raw %}`{{ env "APIP_CP_ENCRYPTION_KEY" }}`{% endraw %} and set the variable in `api-platform.env`. For how these tokens work, see [AI Workspace configuration and environment interpolation](setting-up/configuration.md). + +## Change environment values after setup + +`api-platform.env` holds the values the containers read at startup. Those are the admin credentials the setup script wrote, plus anything else your `config.toml` pulls in through an {% raw %}`{{ env }}`{% endraw %} token. Edit that file to change a setting, for example to switch the AI Workspace login mode or point at a different control plane. Then restart the stack. + +The sample `docker-compose.yaml` loads the file with the `env_file:` directive. It sets `format: raw` so that the `$` characters in a bcrypt password hash aren't treated as Compose interpolation: + +```yaml +services: + platform-api: + env_file: + - path: api-platform.env + required: true + format: raw +``` + +Keep `api-platform.env` out of source control. It's git-ignored in the distribution. + +## Next steps + +- [Manage an LLM provider](llm-providers/manage-provider.md): configure connection, access control, security, rate limiting, guardrails, and models +- [Configure an App LLM proxy](llm-proxies/configure-proxy.md): create a specialized endpoint for one application or agent +- [Manage an App LLM proxy](llm-proxies/manage-proxy.md): configure provider settings, resources, security, and guardrails + +!!! note + Create an App LLM proxy only when a specific GenAI application or agent needs its own guardrails, authentication, exposed resources, or routing on top of a provider. + diff --git a/en/docs/ai-workspace/1.0.0/insights.md b/en/docs/ai-workspace/1.0.0/insights.md new file mode 100644 index 000000000..0d707c6fe --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/insights.md @@ -0,0 +1,79 @@ +--- +title: "Insights" +description: "Open your Moesif analytics workspace from the AI Workspace to view traffic, token usage, latency, and consumer behavior for your AI Gateways." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/insights/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/insights.md +tags: + - cloud + - ai-workspace + - insights +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-09 +content_type: "concept" +--- + +# Insights + +The **Insights** page in the AI Workspace links to your [Moesif](https://www.moesif.com/) analytics workspace, where you can view traffic, token usage, latency, and consumer behavior for the requests flowing through your AI Gateways. Moesif is an external analytics service that stores and displays this data. + +AI Workspace doesn't embed or proxy Moesif content. The Insights page provides a button that opens your Moesif workspace in a new tab, and all analytics are viewed in Moesif itself. + +## How it works + +1. **Configure your AI Gateway** with a Moesif application ID (see [Enable Moesif on a gateway](#enable-moesif-on-a-gateway)). The gateway runtime publishes telemetry — requests, tokens, latency, and guardrail events — directly to Moesif. +2. **Open Insights** in the AI Workspace left navigation menu, then click the button to open your Moesif workspace in a new tab. +3. **View analytics** in Moesif. All data resides in Moesif, not in the AI Workspace. + +## Enable Moesif on a gateway + +Analytics are published to Moesif only when your gateway runtime is configured with a Moesif application ID. Set the `MOESIF_KEY` environment variable on the gateway runtime. + +When you set up the gateway, add the key to `api-platform.env`, the environment file the gateway loads. `./scripts/setup.sh` generates that file, or `.\scripts\setup.ps1` on Windows: + +```bash +MOESIF_KEY= +``` + +Once the key is set and the gateway is running, the gateway publishes events to Moesif automatically. No changes to the AI Workspace are required. + +For the full gateway setup procedure, see [Set up an AI Gateway](ai-gateways/setting-up.md). + +!!! note + If `MOESIF_KEY` isn't configured, the gateway doesn't publish telemetry, and your Moesif workspace shows no data. + +## What Moesif tracks + +With the gateway Moesif integration active, your Moesif workspace shows: + +- **Request and response traffic** — Volume, latency, and error rates. +- **Token usage** — Prompt, completion, and total tokens by model and provider. +- **Estimated LLM cost** — Cost estimates based on token consumption. +- **Guardrail policy triggers** — How often guardrails intervene. +- **Per-application and per-consumer breakdowns** — Usage for mapped [GenAI applications](genai-applications.md) and consumers. + +Filtering, segmentation, and dashboarding are done entirely within Moesif. + +## Use Insights for cost and rate limit management + +Use Moesif analytics to tune your [token-based rate limits](policies/overview.md#token-based-rate-limit): + +1. Use token usage trends to understand actual consumption patterns before setting rate limits. +2. Check per-consumer traffic to determine appropriate per-consumer limits. +3. Monitor error rates to detect rate limit violations (`429 Too Many Requests` responses) and adjust limits accordingly. + +## Use Insights for guardrail monitoring + +When guardrails are active, you can use Moesif analytics to: + +- Track how often guardrails intervene (guardrail intervention responses appear as `422` errors). +- Identify which endpoints or consumers trigger the most interventions. +- Validate that newly configured guardrails behave as expected. + +See [Policies overview](policies/overview.md) for more on configuring guardrails. + +## Related resources + +- [Set up an AI Gateway](ai-gateways/setting-up.md)—configure `MOESIF_KEY` on the gateway runtime +- [Token-based rate limiting](policies/overview.md#token-based-rate-limit)—set token limits and use analytics to monitor usage +- [Policies overview](policies/overview.md)—monitor guardrail interventions through analytics +- [Integrate API Platform with Moesif](../../cloud/monitoring-and-insights/integrate-bijira-with-moesif.md)—advanced Moesif integration options for the broader API Platform diff --git a/en/docs/ai-workspace/1.0.0/llm-provider-templates/configure-template.md b/en/docs/ai-workspace/1.0.0/llm-provider-templates/configure-template.md new file mode 100644 index 000000000..862b3e733 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/llm-provider-templates/configure-template.md @@ -0,0 +1,101 @@ +--- +title: "Configure an LLM provider template" +description: "Create a custom LLM provider template in AI Workspace, configure its connection and token mappings, version it, and deploy it to a gateway." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-provider-templates/configure-template/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-provider-templates/configure-template.md +tags: + - cloud + - ai-workspace + - llm-provider-templates +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-10 +content_type: "how-to" +--- + +# Configure an LLM provider template + +When the built-in templates don't cover the upstream large language model (LLM) service you want to use, create a custom template for it. This guide shows how to create a custom template, complete its configuration, manage its versions, and deploy it to a gateway. + +## Prerequisites + +- A user whose token carries two scopes: `ap:llm_template:manage` to create, edit, and delete templates, and `ap:llm_template:read` to list them. In the [role-to-scope mapping](../setting-up/authentication/overview.md), `ap_admin` and `ap_operator` grant both scopes. `ap_publisher` and `ap_viewer` grant only `ap:llm_template:read`, which permits listing templates but not modifying them. +- At least one [AI Gateway created and set up](../ai-gateways/setting-up.md). +- The endpoint URL and OpenAPI specification of the upstream service. + +## Create a custom template + +1. Navigate to **AI Workspace** > **Settings** > **LLM Provider Templates**. +2. Click **Create**. +3. Enter the relevant details, including the template **name** and the upstream **endpoint URL** (for example, `https://api.example.com`). +4. Click **Create**. + +The new template starts at version **v1.0**. Open it to complete the rest of the configuration. + +![Create LLM provider template form with name and endpoint URL fields](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider-template/create-template-form.png) + +## Configure the template + +The template details page has an overview, a version selector, and the tabs described below. + +![Template details page showing the version selector and configuration tabs](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider-template/template-overview.png) + +### Overview + +Shows the template's logo, description, current version, and when it was last updated. For custom templates, you can edit details such as the logo URL and description here. From here you can also: + +- **Download YAML**: export the current version as a manifest you can apply to a gateway. +- **Enable** or **Disable** the current version (built-in templates only). +- **Delete** the current version (custom templates only). + +### Connection + +Configure how the gateway connects to the upstream service: + +- **Endpoint URL**: the base URL of the upstream service. +- **OpenAPI specification**: provide a **URL** and click **Fetch** to load it, or **upload** the file. +- **Authentication**: the inbound auth type, header or parameter name, and value prefix. + +![Connection tab showing endpoint URL, OpenAPI specification, and authentication settings](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider-template/template-connection.png) + +### Token mapping + +Define where token usage and model information are read from in requests and responses: + +- **Default (Global) mappings**: prompt, completion, total, and remaining tokens, plus the request and response model. +- **Per-resource overrides**: different mappings for individual API resources. + +![Token mapping tab showing default mappings and per-resource overrides](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider-template/template-token-mapping.png) + +## Versioning + +Built-in template versions are read-only. You can edit a custom template version in place at any time. You can also create a new version to introduce a different configuration while keeping the earlier version available. Either way, providers already created from a version aren't affected—a provider copies the template configuration at creation time. + +**To create a new version:** + +1. Open the template and click the **version selector** (for example, **v1.0**). +2. Click **Create new version**. +3. Enter the new version (for example, `v2.0`) and adjust the configuration as needed. +4. Click **Create**. + +![Version selector dropdown with the Create new version option](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider-template/create-new-version.png) + +!!! note + Creating a new version of a **built-in** template produces a **custom** version. + +## Deploy a custom template to the gateway + +Built-in templates are already available on the gateway, but a custom template has to be deployed manually: + +1. Open the template's **Overview** tab and click **Download YAML**. +2. Apply the downloaded manifest to the target gateway. + +!!! warning "Required step" + A provider created from a custom template only works after the template is deployed to the gateway that serves the provider. + +!!! info + You can apply the template manifest through the gateway's management API. See the [LLM provider template Management API reference](../../../api-gateway/next/gateway-controller-management-api/llm-provider-template-management.md) for details. + +## Next steps + +- [Manage an LLM provider template](manage-template.md): create providers from a template, and edit, enable, disable, or delete templates +- [Configure an LLM provider](../llm-providers/configure-provider.md): create a provider from your template diff --git a/en/docs/ai-workspace/1.0.0/llm-provider-templates/manage-template.md b/en/docs/ai-workspace/1.0.0/llm-provider-templates/manage-template.md new file mode 100644 index 000000000..8c72d6f8d --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/llm-provider-templates/manage-template.md @@ -0,0 +1,73 @@ +--- +title: "Manage an LLM provider template" +description: "Use an LLM provider template to create a provider, and edit, enable/disable, or delete template versions in AI Workspace." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-provider-templates/manage-template/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-provider-templates/manage-template.md +tags: + - cloud + - ai-workspace + - llm-provider-templates +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-10 +content_type: "how-to" +--- + +# Manage an LLM provider template + +This guide shows how to create a large language model (LLM) provider from a template, and how to edit, enable, disable, and delete templates. + +## Use a template to create a provider + +When [adding an LLM provider](../llm-providers/configure-provider.md): + +1. Pick a template from the template picker. Custom templates appear alongside the built-in ones. + + ![Template picker in the Add Provider flow showing built-in and custom template cards](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider-template/select-template-for-provider.png) + +2. If the template has more than one version, select the version you want and click **Continue**. If there is only one version, it is selected for you. + + ![Version selection step with available template versions and the Continue button](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider-template/select-template-version.png) + +3. Enter the provider name and credentials. The provider takes its endpoint, authentication, and token mappings from the selected template version. + +!!! warning + A provider created from a custom template only works after the template is [deployed to the gateway](configure-template.md#deploy-a-custom-template-to-the-gateway) that serves the provider. + +## Edit a template + +You can only edit **custom** templates; built-in templates are read-only. There are two ways to make a change: + +- Edit a version in place through the [**Overview**, **Connection**, and **Token Mapping** tabs](configure-template.md#configure-the-template) — for example, to update the logo URL, endpoint, or mappings. +- [Create a new version](configure-template.md#versioning) to introduce a different configuration while keeping the existing version available. + +Editing a template version doesn't affect the providers already created from it. A provider copies the template configuration at creation time, so template changes apply only to providers created afterward. + +## Enable or disable a template + +You can turn a **built-in** template version on or off from the template's **Overview** tab. + +![Built-in template Overview tab showing the Enable/Disable action](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider-template/enable-disable-template.png) + +- Only built-in templates can be enabled or disabled. To remove a custom template, delete it instead. +- A version can't be disabled while a provider is using it. +- Disabled templates appear dimmed in the listing, and you can't create providers from them. + +## Delete a template + +To remove a **custom** template version: + +1. Navigate to **Settings** > **LLM Provider Templates** and open the template. +2. Pick the version you want to remove from the **version selector**. +3. Click **Delete** and confirm. + +![Delete confirmation dialog for a custom template version](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider-template/delete-template.png) + +!!! warning "Deletion is blocked while in use" + You can't delete a template version while a provider created from it still exists. The console shows the error `Cannot delete: one or more providers were created from this template.` Delete those providers first, then try again. + +Deleting the last remaining version removes the whole template. + +## Next steps + +- [LLM providers overview](../llm-providers/overview.md): create a provider from a template +- [App LLM proxies overview](../llm-proxies/overview.md): add application-facing endpoints on top of a provider diff --git a/en/docs/ai-workspace/1.0.0/llm-provider-templates/overview.md b/en/docs/ai-workspace/1.0.0/llm-provider-templates/overview.md new file mode 100644 index 000000000..6f0aae443 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/llm-provider-templates/overview.md @@ -0,0 +1,44 @@ +--- +title: "LLM provider templates overview" +description: "Learn how LLM provider templates capture endpoint, authentication, OpenAPI, and token mapping configuration as reusable blueprints for creating LLM providers in AI Workspace." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-provider-templates/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-provider-templates/overview.md +tags: + - cloud + - ai-workspace + - llm-provider-templates +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-10 +content_type: "overview" +--- + +# LLM provider templates overview + +A large language model (LLM) provider template is a reusable blueprint that holds everything needed to connect to an upstream LLM service: + +- The upstream **endpoint URL** +- The **inbound authentication** settings (auth type, header or parameter name, and value prefix) +- The provider's **OpenAPI specification** +- The **token and model mappings** used for usage tracking + +Once a template holds this configuration, you can create any number of [LLM providers](../llm-providers/overview.md) from it without entering the same details again. + +## Template types + +| Type | Description | +|------|-------------| +| **Built-in** | Shipped with the product for well-known services: OpenAI, Azure OpenAI, Azure AI Foundry, AWS Bedrock, Anthropic, Mistral, and Gemini. These are read-only; you can only enable or disable them. | +| **Custom** | Created by you, either from scratch or as a new version of a built-in template. You can edit and delete these freely. | + +## View templates + +1. Navigate to **AI Workspace** in your API Platform dashboard. +2. Go to **Settings** > **LLM Provider Templates**. + +Custom and built-in templates appear in separate sections, and each template card shows its most recent version. Disabled templates appear dimmed. + +![LLM provider templates listing showing custom and built-in template sections](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider-template/template-listing.png) + +## Next step + +[Configure an LLM provider template](configure-template.md): create a custom template, configure it, and deploy it to a gateway. diff --git a/en/docs/ai-workspace/1.0.0/llm-providers/configure-provider.md b/en/docs/ai-workspace/1.0.0/llm-providers/configure-provider.md new file mode 100644 index 000000000..3f95024d3 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/llm-providers/configure-provider.md @@ -0,0 +1,175 @@ +--- +title: "Configure an LLM provider" +description: "Add an LLM provider in AI Workspace, configure authentication and guardrails, and deploy it to an AI Gateway." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-providers/configure-provider/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-providers/configure-provider.md +tags: + - cloud + - ai-workspace + - llm-providers +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-23 +content_type: "how-to" +--- + +# Configure an LLM provider + +An LLM provider connects an AI service platform such as OpenAI or Anthropic to AI Workspace. Once you configure a provider, deploy it and call it directly through the managed gateway. + +!!! note + App LLM proxies are optional. Use one when you want app-specific or agent-specific guardrails, authentication, or resource exposure on top of the same provider. + +## Prerequisites + +- A user whose token carries the scopes these steps need: + + - `ap:llm_provider:manage` to add and edit providers. + - `ap:llm_provider:deployment:manage` to deploy a provider. + - `ap:gateway:read` to choose the target gateway. + - `ap:secret:create` because AI Workspace stores the upstream API key as an encrypted [secret](../secrets-management.md) on your behalf. + + Of the roles the [role-to-scope mapping](../setting-up/authentication/overview.md) ships, `ap_admin` grants all four. + +- At least one [AI Gateway created and set up](../ai-gateways/setting-up.md). +- API credentials for your LLM provider, such as an API key or an access token. + +## Add a provider + +1. Navigate to **AI Workspace** in your API Platform dashboard. +2. Select **LLM Providers** from the menu. +3. Click **+ Add New Provider** and choose your provider type, for example **OpenAI** or **Anthropic**. Any [custom LLM provider templates](../llm-provider-templates/overview.md) you have created also appear in the picker. + + ![Add LLM Service Provider panel showing selectable tiles for OpenAI, Mistral, Gemini, Azure OpenAI, Azure AI Foundry, AWS Bedrock, and Anthropic](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider/select-llm-provider-type.png) + +4. If the selected template has more than one version, select a version and click **Continue**. A single version is selected automatically. + +## Configure provider details + +After selecting your provider type, fill in the provider configuration form: + + ![Add LLM Service Provider form with OpenAI selected, showing Name, Version, Description, Context, API Key, and Guardrails fields](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider/llm-provider-details.png) + +### Basic information + +1. **Name** (required): Enter a unique name for the provider (for example, `openai-production`, `anthropic-dev`). + +2. **Version** (required): The version is pre-filled (for example, `v1.0`). You can edit this if needed. + +3. **Description** (optional): Add a description to identify the provider's purpose. + +4. **Context** (optional): Enter the context path (default: `/`). This is the base context for the provider. + +### Authentication + +The authentication fields vary depending on the provider you selected: + +=== "OpenAI" + **API Key** (required): Enter your OpenAI API key (starts with `sk-proj-` or `sk-`). + + !!! info + OpenAI's endpoint URL is pre-configured automatically. + +=== "Anthropic" + **API Key** (required): Enter your Anthropic API key (starts with `sk-ant-`). + + !!! info + Anthropic's endpoint URL is pre-configured automatically. + +=== "Gemini" + **API Key** (required): Enter your Google AI API key. + + !!! info + Gemini's endpoint URL is pre-configured automatically. + +=== "Mistral AI" + **API Key** (required): Enter your Mistral AI API key. + + !!! info + Mistral AI's endpoint URL is pre-configured automatically. + +=== "Azure OpenAI" + 1. **Upstream URL** (required): Enter your Azure OpenAI resource endpoint (for example, `https://your-resource.openai.azure.com/`). + 2. **API Key** (required): Enter your Azure OpenAI API key. + +=== "Azure AI Foundry" + 1. **Upstream URL** (required): Enter your Azure AI Foundry endpoint URL. + 2. **API Key** (required): Enter your Azure AI Foundry API key. + +=== "AWS Bedrock" + 1. **Upstream URL** (required): Enter the Bedrock runtime endpoint for your region, in the form `https://bedrock-runtime..amazonaws.com` (for example, `https://bedrock-runtime.us-east-1.amazonaws.com`). + 2. **API Key** (required): Enter your Amazon Bedrock API key. Paste the raw key. AI Workspace adds the `Bearer` prefix followed by a space, and sends the credential as `Authorization: Bearer `. + + !!! info + Bedrock's endpoint isn't pre-configured, because the runtime host is region-specific. Use the region your model access lives in. If you use a short-term Bedrock API key, it's scoped to the region that issued it and doesn't work against another region's endpoint. A long-term key, which is backed by service-specific Identity and Access Management (IAM) credentials, isn't tied to a single region. + +!!! info "How the API key is stored" + AI Workspace stores the upstream API key as an encrypted secret and keeps only a {% raw %}`{{ secret "handle" }}`{% endraw %} reference in the provider configuration. The plaintext key never lands in the provider configuration or in an API response. See [Secrets management](../secrets-management.md). + +!!! note "Custom provider templates" + If you're adding a provider from a custom [LLM provider template](overview.md#connect-a-custom-provider) (**Settings > LLM Provider Templates**), the **Authentication Type** can also be set to **other** (no credentials stored — use a policy to handle upstream auth) or **none** (no upstream authentication sent), in addition to **api-key**. + +### Add guardrails (optional) + +Attach policies and guardrails that apply to every request this provider serves: + +1. In the **Guardrails** section of the form, click **+ Add Guardrail**. + +2. A sidebar opens showing the available guardrails and policies. + +3. Click a guardrail to select it and configure its settings. + +4. Click **Add** to attach it to the provider. + +!!! tip "Advanced settings" + Each guardrail includes advanced configuration options for fine-tuning its behavior. After selecting a guardrail, configure these settings before you attach it to the provider. + +!!! info + Learn more about available guardrails in the [Policies overview](../policies/overview.md). For the full list of policies and their specifications, visit the [Policy Hub](https://wso2.com/api-platform/policy-hub/). + +## Save provider + +1. After configuring all settings and adding guardrails (if needed), click **Add Provider**. + +2. A confirmation message reports that the provider was created. + +3. The provider appears in the providers list. + +## Deploy provider to gateway + +After creating your provider, you must deploy it to a gateway before it can be used. + +!!! warning "Required step" + The provider isn't functional until you deploy it to at least one gateway. + +1. Click the **Deploy to Gateway** button in the top right corner. + +2. Click **Deploy** on one or more gateways from the available list. + +3. Wait for the deployment to complete. The status changes to **Deployed**. + +## Get started + +Once the provider is deployed, the provider details page shows the Invoke URL on the left and a **Get Started** panel on the right. + +### Invoke URL + +Select a gateway from the **Gateways** dropdown to see the base URL for accessing this provider through that gateway. + +### API keys + +Generate an API key to authenticate requests to the deployed gateway. + +1. Click **Generate API Key** in the Get Started panel. +2. Copy and save your API key immediately. + +!!! danger "Important" + An API key is displayed only once. Store it in a secure location immediately, because you can't retrieve it again. + +### Deployed gateways + +The **Deployed Gateways** section lists all gateways this provider is deployed to, along with the host address and deployment status. + +## Next steps + +- [Configure an App LLM proxy](../llm-proxies/configure-proxy.md): configure and deploy a specialized proxy endpoint for a GenAI application or agent that uses your provider +- [Manage an LLM provider](manage-provider.md): configure access control, security, rate limiting, and more diff --git a/en/docs/ai-workspace/1.0.0/llm-providers/manage-provider.md b/en/docs/ai-workspace/1.0.0/llm-providers/manage-provider.md new file mode 100644 index 000000000..ac578b88d --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/llm-providers/manage-provider.md @@ -0,0 +1,293 @@ +--- +title: "Manage an LLM provider" +description: "Configure connection, access control, security, rate limiting, guardrails, and models for a deployed LLM provider, then redeploy or delete it." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-providers/manage-provider/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-providers/manage-provider.md +tags: + - cloud + - ai-workspace + - llm-providers +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-23 +content_type: "how-to" +--- + +# Manage an LLM provider + +After deploying your LLM provider, manage its configuration through the provider details page. This guide covers all management operations organized by tab. + +## Access provider details + +1. Navigate to **AI Workspace** > **LLM Providers**. + +2. Click a provider name to open its details page. + +The provider details page shows: + +- The provider name, version, and template type +- The creation and last-updated timestamps +- Six management tabs: Connection, Access Control, Security, Rate Limiting, Guardrails, and Models +- The **Deploy to Gateway** button in the top right + +## Connection + +Manage upstream endpoint configuration and authentication credentials for the LLM provider. + +### Provider endpoint + +The **Upstream URL** field sets where the gateway forwards API requests: + +| Provider type | URL configuration | +|---------------|-------------------| +| OpenAI, Anthropic, Gemini, Mistral AI | Pre-configured (read-only) | +| Azure OpenAI, Azure AI Foundry, AWS Bedrock | Custom URL (editable) | + +**To update:** + +1. Modify the **Provider Endpoint** field +2. Click **Save** + +### Authentication + +Configure how the gateway authenticates with the upstream provider: + +| Authentication type | Behavior | +|----------------------|----------| +| **api-key** | The gateway attaches an **Authentication Header** and **Credentials** value to every upstream request. For built-in providers, the template sets both. | +| **other** | The workspace stores no credentials for this provider. Use a policy to configure upstream authentication instead. | +| **none** | The gateway sends no upstream authentication. | + +**To configure `api-key`:** + +- **Authentication Header**: The header name the gateway sends upstream +- **Credentials**: Your provider API key + - Click the show icon to reveal or hide the value + - Update and click **Save** to change credentials + +Switching to **other** or **none** clears the stored authentication header and credentials for this provider. + +### OpenAPI specification + +The **Spec URL** field holds the OpenAPI specification the resource list is imported from: + +- Supply either a URL or a file upload +- The specification populates the Access Control resources list +- Re-import it to refresh the available endpoints + +## Access control + +Control which API endpoints are accessible through this provider. + +### Mode + +Select the access control mode at the top of the tab: + +| Mode | Behavior | +|------|----------| +| **Allow all** | All resources are accessible by default. Move specific resources to **Denied Resources** to block them. | +| **Deny all** | All resources are blocked by default. Move specific resources to **Allowed Resources** to permit them. | + +Use the arrow buttons between the two panels to move resources: + +- **`>>`** — Move all resources to the other panel +- **`>`** — Move the selected resources to the other panel +- **`<`** — Move the selected resources back +- **`<<`** — Move all resources back + +### Import resources + +Click **Import resources from specification** to load or refresh the resource list from the provider's OpenAPI specification. + +## Security + +Configure how applications authenticate when accessing this provider through the gateway. + +### API key authentication + +Set the API key that client applications must provide: + +| Field | Description | Example | +|-------|-------------|---------| +| **Authentication Type** | Authentication method | `apiKey` (default) | +| **API Key** | The name of the request header that must carry the API key | `x-api-key`, `apikey`, `Authorization` | +| **Key Location** | Where clients send the key. `header` is the only supported option. | `header` | +| **API Key Value Prefix** | An optional prefix prepended to the value clients must send, so that clients send `Bearer ` | `Bearer` | + +**To configure:** + +1. Select **Authentication Type**: `apiKey` +2. Enter the **API Key** header name your applications use. +3. Optionally set an **API Key Value Prefix**, for example `Bearer`, if clients send the key with a scheme prefix. +4. Click **Save**. + +## Rate limiting + +Control request and token consumption to prevent cost overruns and keep usage fair across all consumers. + +Configure rate limits in the **Backend** section, which controls requests from the gateway to the upstream LLM provider. Limiting that traffic protects your provider API credentials and caps total spend. + +The Backend section supports two configuration modes: + +- **Provider-wide** — A single limit applied across all API endpoints. The limit maintains **one shared counter**: traffic on any endpoint draws down the same allowance, so exhausting the limit via one endpoint rejects requests on all endpoints. +- **Per Resource** — Individual limits per API endpoint (for example, chat completions vs. embeddings). Each endpoint maintains its **own independent counter**. + +The Backend section of the Rate Limiting tab presents both the Provider-wide and Per Resource modes: + +![Rate Limiting tab showing Provider-wide and Per Resource configuration modes in the Backend section](../../../assets/img/ai-gateway/standalone-ai-workspace/llm-provider/rate-limiting-tab.png) + +!!! note + Provider-wide and Per Resource modes are mutually exclusive. Clear existing limits before switching modes. + +!!! info "Provider-wide limits are a hard ceiling" + A provider-wide limit is evaluated before any per-resource policy and counts **every request attempt** — including requests that a stricter per-resource policy later rejects. When the shared allowance is exhausted, requests to every endpoint receive `HTTP 429`. See [Policy scope: global or per resource](../policies/overview.md#policy-scope-global-or-per-resource) for details. + +### Limit criteria + +Configure either of these criteria, or both: + +| Criterion | Description | +|-----------|-------------| +| **Request Count** | Maximum number of requests within the reset duration | +| **Token Count** | Maximum number of tokens (prompt plus completion) within the reset duration | + +For each enabled criterion, set the **Quota** and **Reset Duration** (`second`, `minute`, or `hour`). + +### Provider-wide configuration + +1. Select **Provider-wide** in the Backend section. +2. Enable **Request Count**, **Token Count**, or both. +3. Enter the **Quota** and select the **Reset Duration** for each criterion. +4. Click **Save**. + +### Per-resource configuration + +1. Select **Per Resource** in the Backend section. +2. Expand **Limit per Resource** to set default limits for all endpoints: + - Enable the criteria and configure **Quota** and **Reset Duration**. +3. To override limits for a specific endpoint, expand that resource row and configure it separately. +4. Click **Save**. + +!!! tip "Cost control best practices" + Set conservative backend limits first to protect your provider credentials. Monitor actual usage via the Insights dashboard before increasing limits. Use Per Resource mode only when endpoints have significantly different usage patterns. + +**Learn more:** [Token-based rate limiting](../policies/overview.md#token-based-rate-limit) + +## Guardrails + +Attach guardrails to enforce content safety, compliance, and quality standards. Attach a guardrail on a provider globally to all endpoints or at the resource level to specific endpoints. Either way, the guardrail affects every proxy that uses this provider. + +### View attached guardrails + +The tab displays the guardrails attached to the provider: + +- **Guardrail name** and type +- **Configuration status** and parameters +- **Enable/disable toggles** to activate or deactivate a guardrail + +### Add a guardrail + +Attach a guardrail globally to all endpoints or at the resource level to one endpoint. A global guardrail runs on every request, whichever endpoint the request calls. A resource-level guardrail runs only on the endpoint you attach it to. If you configure both, the gateway evaluates the global guardrails first, then the resource-level ones. See [Policy scope: global or per resource](../policies/overview.md#policy-scope-global-or-per-resource). + +**To add a global guardrail:** + +1. In the **Global Guardrails** section, click **+ Add Guardrail**. +2. A sidebar opens showing available guardrail types. +3. Select a guardrail and configure its settings: + - Fill in required parameters + - Expand **Advanced Settings** for additional options +4. Click **Add** to attach it to the provider. + +**To add a resource-level guardrail:** + +1. Find the resource you want to protect and expand its card. +2. Click **+ Add Guardrail** within that resource. +3. Select and configure the guardrail (same process as global guardrails). +4. Click **Add** to attach it to the resource. + +### Configure guardrails + +You can't edit guardrail parameters in place. To change a guardrail's configuration, delete it and add it again with the updated settings. + +**To update a guardrail:** + +1. Delete the existing guardrail. +2. Click **+ Add Guardrail** and re-add it with the updated configuration. +3. Redeploy the provider to apply the changes. + +!!! tip "Advanced settings" + Each guardrail includes advanced configuration options such as custom thresholds, severity levels, and execution phases. Click **Advanced Settings** when adding a guardrail. + +!!! warning "Production impact" + Guardrail changes require a manual redeploy to take effect on deployed gateways. Test thoroughly in a non-production environment before enabling strict guardrails. + +**Learn more:** [Policies overview](../policies/overview.md). For the full policy catalog, visit the [Policy Hub](https://wso2.com/api-platform/policy-hub/). + +## Models + +Configure which AI models are accessible through this provider. + +### Add and remove models + +The **Models** tab displays a chip list of the models available through this provider. Each chip represents one model ID. + +**To add a model:** + +1. Click **Add model provider**, select a provider from the list, and click **Add** to import its model catalog. +2. Type or paste individual model IDs into the input field and press Enter to add them as chips. +3. Click **Save**, then **Deploy to Gateway** to apply the change. + +**To remove a model:** + +1. Click the remove icon on the model chip you want to remove. +2. Click **Save**, then **Deploy to Gateway** to apply the change. + +The gateway blocks any model that isn't in the chip list. An application that requests a removed model receives an error. + +## Lifecycle operations + +### Redeploy provider + +Push configuration changes to deployed gateways. + +**When to redeploy:** + +- After updating connection settings +- After modifying rate limits or guardrails +- After adding or removing models + +**To redeploy:** + +1. Click **Deploy to Gateway** (top right corner) +2. Select the gateways to deploy to +3. Review the changes summary +4. Click **Deploy** + +!!! info "Deployment status" + Monitor deployment progress in the notification panel. Changes take effect within seconds of successful deployment. + +### Delete provider + +Permanently remove the provider and all its configurations. + +!!! warning "Prerequisite" + You can't delete a provider while an App LLM proxy uses it. Delete or reassign all dependent proxies before proceeding. + +**To delete:** + +1. Navigate to **AI Workspace** > **LLM Providers** +2. Find the provider in the list +3. Click the **Delete** icon +4. Review the warning and confirm deletion + +!!! danger "Irreversible action" + Deleting a provider: + + - Removes it from all deployed gateways **immediately** + - Breaks applications consuming this provider + - Deletes all configuration, including guardrails, rate limits, and models + - **Cannot be undone** + +## Next steps + +- [Configure an App LLM proxy](../llm-proxies/configure-proxy.md): configure and deploy specialized proxy endpoints for GenAI applications or agents using your provider +- [Policies overview](../policies/overview.md): explore all available guardrails and policies diff --git a/en/docs/ai-workspace/1.0.0/llm-providers/overview.md b/en/docs/ai-workspace/1.0.0/llm-providers/overview.md new file mode 100644 index 000000000..6f6d60476 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/llm-providers/overview.md @@ -0,0 +1,44 @@ +--- +title: "LLM providers overview" +description: "Connect AI service platforms such as OpenAI, Anthropic, Azure OpenAI, Gemini, and Mistral AI as reusable LLM providers in AI Workspace." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-providers/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-providers/overview.md +tags: + - cloud + - ai-workspace + - llm-providers +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-23 +content_type: "overview" +--- + +# LLM providers overview + +An LLM provider is an integration with an AI service platform that offers language models. By configuring providers in AI Workspace, you can: + +- **Centralize credential management**: Store API keys and authentication details securely +- **Connect multiple providers**: Integrate with leading LLM services +- **Monitor provider status**: Track availability and health of connected services +- **Reuse configuration**: Use one provider across multiple proxies without duplicating credentials + +## Supported providers + +AI Workspace supports the following LLM providers: + +| Provider | Description | Learn more | +|----------|-------------|-----------| +| **OpenAI** | Access GPT-4, GPT-3.5, and other OpenAI models | [OpenAI documentation](https://developers.openai.com/api/docs) | +| **Anthropic** | Integrate Anthropic Claude models | [Anthropic documentation](https://docs.anthropic.com/) | +| **Azure OpenAI** | Use OpenAI models hosted on Microsoft Azure | [Azure OpenAI documentation](https://azure.microsoft.com/products/ai-services/openai-service) | +| **Azure AI Foundry** | Access models through Azure AI Foundry platform | [Azure AI Foundry documentation](https://azure.microsoft.com/products/ai-studio) | +| **Gemini** | Integrate Google's Gemini language models | [Gemini documentation](https://ai.google.dev/gemini-api) | +| **Mistral AI** | Access Mistral's open and commercial models | [Mistral AI documentation](https://mistral.ai/) | +| **AWS Bedrock** | Access Anthropic, Meta, Mistral, and Amazon models through Amazon Bedrock | [AWS Bedrock documentation](https://aws.amazon.com/bedrock/) | + +## Connect a custom provider + +If the LLM service you use isn't in the list above, a user with template-management access can define a reusable **LLM Provider Template** under **Settings > LLM Provider Templates**. A template captures the endpoint and authentication shape for a custom provider. It then appears in the provider type selector alongside the built-in providers whenever you add a provider. + +## Next step + +[Configure an LLM provider](configure-provider.md): set up your first provider. diff --git a/en/docs/ai-workspace/1.0.0/llm-proxies/configure-proxy.md b/en/docs/ai-workspace/1.0.0/llm-proxies/configure-proxy.md new file mode 100644 index 000000000..181a5221d --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/llm-proxies/configure-proxy.md @@ -0,0 +1,153 @@ +--- +title: "Configure an App LLM proxy" +description: "Create an App LLM proxy, optionally attach guardrails, and deploy it to a gateway." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-proxies/configure-proxy/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-proxies/configure-proxy.md +tags: + - cloud + - ai-workspace + - llm-proxies +author: WSO2 API Platform Documentation Team +last_updated: 2026-06-22 +content_type: "how-to" +--- + +# Configure an App LLM proxy + +An App LLM proxy exposes a managed endpoint that your GenAI applications or agents use to reach an LLM provider. Once deployed, the proxy handles authentication and enforces any guardrails you configure. + +You can also call the provider directly. Use a proxy when you need specialized endpoints for different applications, agents, or environments. Each proxy gets its own keys, guardrails, and access controls on top of the same provider. + +This guide walks you through creating an App LLM proxy, optionally attaching guardrails, and deploying it to a gateway. + +## Prerequisites + +!!! info "Before you begin" + - A user whose token carries the scopes these steps need: + + - `ap:llm_proxy:manage` to create and edit proxies. + - `ap:llm_proxy:deployment:manage` to deploy a proxy. + - `ap:llm_provider:read` to choose the provider behind it. + - `ap:gateway:read` to choose the target gateway. + - `ap:secret:create`, but only when the proxy carries its own upstream credential. + + Of the roles the [role-to-scope mapping](../setting-up/authentication/overview.md) ships, `ap_admin` grants all of these. `ap_publisher` grants every one except `ap:secret:create`. + + - At least one [configured and deployed LLM provider](../llm-providers/configure-provider.md). + +## Create an App LLM proxy + +1. Navigate to **LLM** > **Proxies** in the left navigation menu. + +2. Click **+ Create Proxy**. + +3. Fill in the required proxy details: + + 1. **Name*** (Required): Enter a unique name for the proxy (for example, `support-chat-api`, `sales-agent-proxy`). The Proxy ID is auto-generated from the name (lowercase, hyphen-separated). + + 2. **LLM Service Provider*** (Required): Select the LLM provider this proxy routes to from the dropdown. It defaults to the most recently updated provider. + + 3. **Version*** (Required): The version is pre-filled (for example, `v1.0`). You can edit this if needed. + + 4. **Description** (Optional): Add a brief description to identify the GenAI application or agent use case this proxy is created for. + + 5. **Context** (Optional): Enter the context path (default: `/`). This is the base path for proxy endpoints (normalized with "/" prefix). + +4. Click **Create Proxy** to save the proxy. + +## Add guardrails (optional) + +Attach policies and guardrails that apply to every request this proxy serves: + +1. In the **Guardrails** tab, click **+ Add Guardrail**. + +2. A sidebar opens showing the available guardrails and policies. + +3. Click a guardrail to select it and configure its settings. + +4. Click **Submit** to attach it to the proxy. + +!!! tip "Advanced settings" + Each guardrail includes advanced configuration options for fine-tuning its behavior. After selecting a guardrail, configure these settings before you attach it to the proxy. + +!!! info + Learn more about available guardrails in the [Policies overview](../policies/overview.md). For the full list of policies and their specifications, visit the [Policy Hub](https://wso2.com/api-platform/policy-hub/). + +## Deploy proxy to gateway + +After creating your proxy, you must deploy it to a gateway before it can be used. + +!!! warning "Required step" + The proxy isn't functional until you deploy it to at least one gateway. + +1. Click the **Deploy to Gateway** button in the top-right corner of the proxy details page. + +2. The **Deploy to Gateway** page opens, showing all available gateways. Use the search bar to find a specific gateway. + +3. Each gateway card displays: + - **Gateway name** and **status** (Active or Not Active) + - **Current Deployment** identifier (if previously deployed) + +4. Click **Deploy** next to the gateway you want to deploy to. + +5. Once deployed, expand the gateway card to view deployment details: + + | Field | Description | + |-------|-------------| + | **Deployment Status** | Either Active or Inactive | + | **Deployment ID** | Unique identifier for the deployment | + | **Deployed** | The time since deployment | + | **Stop** | Button to stop an active deployment | + +6. The **API Deployment History** panel on the right shows the deployment timeline: + - Deployment identifier and timestamp + - Status tags (**Latest**, **Deployed**) + +**To stop a deployment:** + +Click the **Stop** button next to an active deployment to undeploy the proxy from that gateway. + +Click **Back to App LLM Proxy** to return to the proxy details page. + +!!! info "Deployment status" + Monitor deployment progress on this page. Changes take effect within seconds of successful deployment. + +## Get started + +Once the proxy is deployed, the proxy details page shows the **Get Started** panel on the right. + +### Invoke URL + +Select a gateway from the **Gateways** dropdown to see the base URL for calling this proxy through that gateway. + +The URL follows the format: + +```text +https://{gateway-host}/{proxy-name} +``` + +To call a specific resource, append the resource path: + +```text +https://{gateway-host}/{proxy-name}/chat/completions +``` + +### App LLM proxy keys + +Generate an API key to authenticate requests to the deployed gateway. + +1. Click **Generate API Key** in the Get Started panel. +2. Copy and save your API key immediately. + +!!! danger "Important" + An API key is displayed only once. Store it in a secure location immediately, because you can't retrieve it again. + +### Deployed gateways + +The **Deployed Gateways** section lists all gateways this proxy is deployed to, along with the deployment status. + +## Next steps + +- [Invoke providers and proxies via SDKs](../using-sdks.md): connect to your proxy from Python using provider-native SDKs +- [Manage an App LLM proxy](manage-proxy.md): configure the provider, resources, security, and guardrails +- [Policies overview](../policies/overview.md): learn about the policies available for rate limiting and caching diff --git a/en/docs/ai-workspace/1.0.0/llm-proxies/manage-proxy.md b/en/docs/ai-workspace/1.0.0/llm-proxies/manage-proxy.md new file mode 100644 index 000000000..d58b733a0 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/llm-proxies/manage-proxy.md @@ -0,0 +1,196 @@ +--- +title: "Manage an App LLM proxy" +description: "Configure provider settings, resources, security, and guardrails for a deployed App LLM proxy, then save, redeploy, or delete it." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-proxies/manage-proxy/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-proxies/manage-proxy.md +tags: + - cloud + - ai-workspace + - llm-proxies +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-23 +content_type: "how-to" +--- + +# Manage an App LLM proxy + +Once you create an App LLM proxy, manage its configuration through the proxy details page. This guide covers every management operation available for your App LLM proxy. + +## Access proxy details + +1. Navigate to **LLM** > **Proxies** in the left navigation menu. + +2. Click a proxy name to open its details page. + +The proxy details page displays the following information at the top: + +- **Proxy name**, for example `Test-Openai` +- **Provider**: the linked LLM service provider +- **Version**: the proxy version +- **Last updated**: the timestamp of the most recent change + +The page is organized into four configuration tabs: **Provider**, **Resources**, **Security**, and **Guardrails**. On the right side, you'll find the **Get Started** panel with API key generation and deployed gateway information. + +The **Deploy to Gateway** button and a **delete** icon are located in the top-right corner of the page. + +## Provider settings + +Configure which LLM service provider the proxy connects to and how it authenticates with the provider. + +### LLM service provider + +1. Go to the **Provider** tab. + +2. Use the **Provider** dropdown to select or change the linked LLM service provider. + + The dropdown lists all service providers that have been configured in your workspace. + + !!! info + Changing the provider updates related settings such as authentication and available resources. + +### API key configuration + +If your selected provider uses API key authentication, you'll see the **API Key Configuration** section below the provider dropdown. + +| Field | Description | +|-------|-------------| +| **Header Name** | The authentication header required by the provider (for example, `X-API-Key`). This field is read-only and is defined by the provider template. | +| **API Key** | Enter your provider's API key in this field. | + +**To configure:** + +1. Enter the API key in the **API Key** field. +2. Click **Save API Key** to store the credential securely. + +!!! warning "Security best practices" + - The workspace encrypts API keys and stores them securely. + - Keys aren't displayed after saving. + - Store a backup copy in a secure location. + - Rotate keys regularly. + +## Resources + +Define which API endpoints (resources) are available through this proxy by importing an OpenAPI specification. + +### OpenAPI specification + +The OpenAPI specification defines the resources, or API endpoints, available through your proxy. For some providers, the workspace populates the specification for you. + +**To import manually:** + +1. Go to the **Resources** tab. + +2. Click the **Import from file** button. + +3. Upload a JSON or YAML file containing your OpenAPI specification. + +4. The proxy automatically parses the file and displays the extracted resources. + +!!! tip "OpenAPI format" + The proxy accepts OpenAPI 3.0 and 3.1 specifications in both JSON and YAML formats. + +### View resources + +After importing, the **Resources** section displays the count and list of parsed resources. Each resource shows: + +- **HTTP Method**: the HTTP method (GET, POST, PUT, DELETE, or PATCH) +- **Resource Path**: the endpoint path, for example `/v1/chat/completions` +- **Description**: the description extracted from the OpenAPI specification + +## Security + +Configure how client applications authenticate when accessing your proxy endpoints. + +### Authentication + +1. Go to the **Security** tab. + +2. Configure the following authentication settings: + + | Field | Description | + |-------|-------------| + | **Authentication type** | Select the authentication method from the dropdown. **API Key** is the only available method. | + | **Key name** | The name of the request header that clients must provide, for example `X-API-Key`. | + | **Sent in** | Where clients send the key. `header` is the only supported option. | + | **API Key Value Prefix** | An optional prefix prepended to the value clients must send, for example `Bearer`, so that clients send `Bearer `. | + +3. Click **Save** to apply the security configuration. + +## Guardrails + +Attach guardrails to enforce content safety, compliance, and quality standards on this proxy. + +### Global guardrails + +Global guardrails apply to **all resources** in the proxy. + +**To add a global guardrail:** + +1. Go to the **Guardrails** tab. + +2. In the **Global Guardrails** section, click **+ Add Guardrail**. + +3. A right-side panel opens displaying the available guardrails and policies. + +4. Select a guardrail from the list. + +5. Configure the guardrail settings including version, parameters, and any advanced settings. + +6. Click **Submit** to add the guardrail. + +Each guardrail you add appears as a pill showing its name and version. + +### Resource-wise guardrails + +Resource-wise guardrails apply to **specific endpoints** only. This section lists each resource from your OpenAPI specification, so you can attach guardrails to individual endpoints. + +**To add a resource-specific guardrail:** + +1. In the **Resource-wise Guardrails** section, find the resource you want to protect. + +2. Expand the resource card and click **+ Add Guardrail**. + +3. Select and configure the guardrail (same process as global guardrails). + +4. Click **Submit** to attach the guardrail to the resource. + +### Manage guardrails + +- **View**: Global guardrails appear as pills in the Global Guardrails section. Resource-specific guardrails appear under each resource card. +- **Remove**: Click the **×** icon on a guardrail pill to remove it. + +!!! warning + After adding or removing guardrails, you must redeploy the proxy for the changes to take effect. Click **Deploy to Gateway** to apply your changes. + +!!! info + Learn more about available guardrails in the [Policies overview](../policies/overview.md). For the full list of policies and their specifications, visit the [Policy Hub](https://wso2.com/api-platform/policy-hub/). + +## Save changes + +After making configuration changes across any tab (Provider, Resources, Security, Guardrails), click the **Save** button at the bottom-right of the page to persist your changes. + +Use the **Cancel** button to discard all unsaved changes and revert to the last saved state. + +!!! info + Configuration changes require a manual redeploy to take effect on deployed gateways. After saving, click **Deploy to Gateway** to apply your changes. + +## Delete proxy + +To delete a proxy, click the **delete** icon (trash icon) in the top-right corner of the proxy details page, next to the Deploy to Gateway button. + +!!! danger "Irreversible action" + Deleting a proxy is permanent and can't be undone. + + - The workspace removes the proxy from all gateways immediately. + - All generated API keys stop working. + - All configuration, including resources, security, and guardrails, is deleted. + +!!! warning "Before deleting" + - Ensure no applications are actively using the proxy. + - Back up any important configuration settings. + - Notify teams that depend on the proxy endpoints. + +## Next steps + +- [Policies overview](../policies/overview.md): learn about all available policies for proxies +- [Policy Hub](https://wso2.com/api-platform/policy-hub/): browse the full catalog of available guardrails and policies diff --git a/en/docs/ai-workspace/1.0.0/llm-proxies/overview.md b/en/docs/ai-workspace/1.0.0/llm-proxies/overview.md new file mode 100644 index 000000000..72ce7d2fd --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/llm-proxies/overview.md @@ -0,0 +1,40 @@ +--- +title: "App LLM proxies overview" +description: "Add an application-facing endpoint on top of an LLM provider for app- or agent-specific authentication, guardrails, and access controls." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-proxies/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-proxies/overview.md +tags: + - cloud + - ai-workspace + - llm-proxies +author: WSO2 API Platform Documentation Team +last_updated: 2026-06-22 +content_type: "overview" +--- + +# App LLM proxies overview + +## Why a proxy on top of a provider? + +An **LLM Provider** connects the gateway to an upstream LLM service and can be called directly. An **App LLM Proxy** adds an optional, application-facing endpoint on top when you need controls that are specific to a GenAI application or agent. + +The main benefit is specialization and isolation. You can call a single provider directly, or back multiple App LLM proxies with it. For example, use one proxy for a customer-facing chatbot with strict guardrails, a second for an internal agent with relaxed settings, and a third for a workflow-specific GenAI assistant. You configure each proxy independently. + +## What you can do with an App LLM proxy + +**Expose a controlled API endpoint.** The proxy gives you a stable URL your GenAI application or agent calls. You control which resources (API paths) are exposed, and can enable or disable them without touching the upstream provider. + +**Add app-specific controls only when needed.** If provider-level controls are sufficient, call the provider directly. Use an App LLM proxy only when a specific application or agent needs its own authentication, guardrails, exposed resources, or traffic controls. + +**Enforce authentication.** Require applications or agents to present an API key before the gateway forwards their requests to the LLM. The workspace generates keys per proxy, and each key expires after 90 days. + +**Apply guardrails.** Attach content safety, personally identifiable information (PII) masking, or semantic caching policies globally across all endpoints, or target them at specific resources only. + +**Create specialized endpoints for apps and agents.** Create separate proxies for different GenAI applications, agents, teams, or environments such as dev, staging, and production. Each proxy has its own guardrails, access keys, and exposed resources. Provider-level rate limits stay shared across every proxy that uses the same provider backend. + +**Switch between compatible providers without client changes.** Applications and agents call the proxy URL rather than the provider directly. You can swap the underlying LLM provider, for example from OpenAI to Azure OpenAI, and clients need no changes as long as the new provider preserves the client-facing contract. That contract covers the authentication clients send and the resources the proxy exposes. + +## Next steps + +- [Configure an App LLM proxy](configure-proxy.md): create and deploy your first specialized proxy +- [Manage an App LLM proxy](manage-proxy.md): update configuration, guardrails, and resources after deployment diff --git a/en/docs/ai-workspace/1.0.0/mcp-proxies/apply-policies.md b/en/docs/ai-workspace/1.0.0/mcp-proxies/apply-policies.md new file mode 100644 index 000000000..ce2a2d8d9 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/mcp-proxies/apply-policies.md @@ -0,0 +1,44 @@ +--- +title: "Apply policies to an MCP proxy" +description: "Apply access control, authorization, and rewrite policies to MCP proxies using the Policy Hub." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/mcp-proxies/apply-policies/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/mcp-proxies/apply-policies.md +tags: + - cloud + - ai-workspace + - mcp-proxies + - policies +author: WSO2 API Platform Documentation Team +last_updated: 2026-06-22 +content_type: "how-to" +--- + +# Apply policies to an MCP proxy + +Once you create a Model Context Protocol (MCP) proxy, open it from **MCP** > **MCP Proxies** and go to its **Policies** tab to apply policies. + +AI Workspace provides built-in policies that govern how traffic flows through your MCP proxies. A policy applies to the whole proxy by default. Inside an MCP-specific policy you can also define rules per tool or per prompt. Those rules apply the policy at each capability level. + +## Access control policies + +These policies enforce security for MCP proxies. + +| Policy | Description | +|-----------|-------------| +| [MCP authentication](https://wso2.com/api-platform/policy-hub/policies/mcp-auth) | Applies authentication as defined in the MCP specification. | +| [MCP authorization](https://wso2.com/api-platform/policy-hub/policies/mcp-authz) | Applies fine-grained authorization for MCP capabilities and JSON-RPC methods. | +| [MCP access control](https://wso2.com/api-platform/policy-hub/policies/mcp-acl-list) | Allows or denies access to MCP capabilities. | + +## Other policies + +| Policy | Description | +|-----------|-------------| +| [MCP rewrite](https://wso2.com/api-platform/policy-hub/policies/mcp-rewrite) | Rewrites the MCP capabilities returned through the proxy. When applied, the proxy returns only the modified capabilities. | + +You can apply the other standard policies to MCP proxies too. Not every policy supports MCP traffic. Check the policy's entry in the [Policy Hub](https://wso2.com/api-platform/policy-hub/) for its supported proxy types and behavior before you attach it. + +## Policy Hub + +The policies in AI Workspace are powered by the [Policy Hub](https://wso2.com/api-platform/policy-hub/), a central registry of the available policies and their versions. + +Visit the [Policy Hub](https://wso2.com/api-platform/policy-hub/) to explore all available policies, their documentation, and configuration schemas. diff --git a/en/docs/ai-workspace/1.0.0/mcp-proxies/configure-proxy.md b/en/docs/ai-workspace/1.0.0/mcp-proxies/configure-proxy.md new file mode 100644 index 000000000..292460790 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/mcp-proxies/configure-proxy.md @@ -0,0 +1,108 @@ +--- +title: "Configure an MCP proxy" +description: "Create an MCP proxy from an upstream MCP server URL and deploy it to a gateway." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/mcp-proxies/configure-proxy/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/mcp-proxies/configure-proxy.md +tags: + - cloud + - ai-workspace + - mcp-proxies +author: WSO2 API Platform Documentation Team +last_updated: 2026-06-22 +content_type: "how-to" +--- + +# Configure an MCP proxy + +A Model Context Protocol (MCP) proxy exposes a managed endpoint that your applications use to reach an MCP server. Once deployed, the proxy handles authentication and other access controls. + +This guide walks you through creating a proxy and deploying it to a gateway. + +## Prerequisites + +!!! info "Before you begin" + - Access to AI Workspace with the **Admin** or **Developer** role. + +## Create a new proxy + +1. Navigate to **MCP** > **MCP Proxies** in the left navigation menu. + +2. Click **+ Create MCP Proxy**. + +3. Provide the **MCP Server URL**. AI Workspace connects to that URL and fetches the server information. + + !!! warning "Protected servers" + If the MCP server is protected with static credentials, provide them under **Advanced Configurations**. AI Workspace uses those credentials when it fetches the server information. + +4. Click **Next** to proceed to the next step. + +5. Fill in the required proxy details: + + 1. **Name** (required): Enter a unique name for the proxy (for example, `mcp-tools-proxy`, `context-server-proxy`). The Proxy ID is auto-generated from the name (lowercase, hyphen-separated). + + 2. **Version** (required): The version is pre-filled (for example, `v1.0`). You can edit this if needed. + + 3. **Description** (optional): Add a brief description to identify the proxy's purpose. + + 4. **Context** (optional): Enter the context path (default: `/`). This is the base path for proxy endpoints (normalized with "/" prefix). + + 5. **Target** (required): The MCP server URL the gateway calls. AI Workspace fills this in from the URL you provided in the previous step. + +6. Click **Create** to create and save the proxy. + +## Deploy proxy to gateway + +After creating your proxy, you must deploy it to a gateway before it can be used. + +!!! warning "Required step" + The proxy isn't functional until you deploy it to at least one gateway. + +1. Click the **Deploy to Gateway** button in the top-right corner of the proxy details page. + +2. The **Deploy to Gateway** page opens, showing all available gateways. Use the search bar to find a specific gateway. + +3. Each gateway card displays: + - **Gateway name** and **status** (Active or Not Active) + - **Current Deployment** identifier (if previously deployed) + +4. Click **Deploy** next to the gateway you want to deploy to. + +5. Once deployed, expand the gateway card to view deployment details: + + | Field | Description | + |-------|-------------| + | **Deployment Status** | Either Active or Inactive | + | **Deployment ID** | Unique identifier for the deployment | + | **Deployed** | The time since deployment | + | **Stop** | Button to stop an active deployment | + +6. The **API Deployment History** panel on the right shows the deployment timeline: + - Deployment identifier and timestamp + - Status tags (**Latest**, **Deployed**) + +**To stop a deployment:** + +Click the **Stop** button next to an active deployment to undeploy the proxy from that gateway. + +Click **Back to MCP Proxy** to return to the proxy details page. + +!!! info "Deployment status" + Monitor deployment progress on this page. Changes take effect within seconds of successful deployment. + +## Get started + +Once the proxy is deployed, the overview page shows the MCP proxy URL. + +### Connect with an MCP client + +Select a gateway from the **Gateways** dropdown to see the base URL for calling this proxy through that gateway. + +The URL follows the format: + +```text +https://{gateway-host}/{proxy-context}/mcp +``` + +## Next steps + +- [Apply policies to an MCP proxy](apply-policies.md): govern MCP traffic with access control, authorization, and rewrite policies. diff --git a/en/docs/ai-workspace/1.0.0/mcp-proxies/overview.md b/en/docs/ai-workspace/1.0.0/mcp-proxies/overview.md new file mode 100644 index 000000000..03df731a4 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/mcp-proxies/overview.md @@ -0,0 +1,40 @@ +--- +title: "MCP proxies overview" +description: "Connect the AI Gateway to upstream MCP servers and apply security, policy, and observability controls through an MCP proxy." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/mcp-proxies/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/mcp-proxies/overview.md +tags: + - cloud + - ai-workspace + - mcp-proxies +author: WSO2 API Platform Documentation Team +last_updated: 2026-06-22 +content_type: "overview" +--- + +# MCP proxies overview + +## What is Model Context Protocol? + +Model Context Protocol (MCP) is a JSON-RPC-based protocol that standardizes how applications interact with large language models (LLMs). It shares contextual information with LLMs, such as local files, databases, or APIs. It also lets applications expose tools and capabilities for AI-driven workflows and integrations. + +MCP follows a host-client-server architecture and supports two transport mechanisms: stdio and streamable HTTP. Use stdio for local communication between a client and a server on the same machine. In most other cases, deploy the server in a remote environment with authorization controls, so LLM applications access the data securely. + +For more information, refer to the official [specification](https://modelcontextprotocol.io/introduction). + +## What is an MCP proxy? + +An MCP proxy connects the gateway to an upstream MCP server. MCP clients call the dedicated endpoint the gateway provides, and the gateway forwards their requests to your upstream MCP server. On that proxy you apply policies that control the MCP traffic passing through the gateway. + +## What does an MCP proxy offer? + +MCP defines an RPC-based communication model between agents and tools, but it leaves the demands of an enterprise environment to you. An MCP proxy covers them with built-in security, governance, and observability for MCP communication: + +- **Security**: authenticates and authorizes callers. +- **Policies**: enforces the policies that control MCP traffic. +- **Observability**: shows which tools and servers are called, and which calls fail. + +## Next steps + +- [Configure an MCP proxy](configure-proxy.md): create and deploy your first proxy +- [Apply policies to an MCP proxy](apply-policies.md): apply policies after deployment diff --git a/en/docs/ai-workspace/1.0.0/overview.md b/en/docs/ai-workspace/1.0.0/overview.md new file mode 100644 index 000000000..c035a4fcb --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/overview.md @@ -0,0 +1,168 @@ +--- +title: "AI Workspace overview" +description: "Centrally manage AI gateways, LLM providers, proxies, policies, and secrets from a single control plane." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/overview.md +tags: + - cloud + - ai-workspace + - overview +author: WSO2 API Platform Documentation Team +last_updated: 2026-08-01 +content_type: "overview" +--- + +# AI Workspace overview + +AI Workspace is the control plane for managing how your applications access AI services. It's intended for platform teams and developers who govern large language model (LLM) traffic across an organization. + +From one console, you can do the following: + +- Connect AI Gateway runtimes. +- Configure LLM providers and proxies. +- Apply AI policies. +- Manage credentials. +- Deploy these configurations to your gateways. + +Instead of configuring each gateway or application separately, AI Workspace gives you a central place to manage and govern AI traffic across your organization. + +## How AI Workspace works + +AI Workspace manages your AI configuration, and the AI Gateway processes requests. The following diagram shows how the two relate: + +```text + AI Workspace + Control plane + │ + │ Deploy configuration + ▼ +Applications ────────► AI Gateway ────────► AI services + Data plane + │ + ┌─────────┼─────────┐ + │ │ │ + LLM App LLM MCP + Provider Proxy Proxy +``` + +A typical workflow has these steps: + +1. Connect an AI Gateway to AI Workspace. +2. Configure AI artifacts, such as LLM providers, App LLM proxies, and Model Context Protocol (MCP) proxies. +3. Apply policies such as guardrails and rate limits. +4. Deploy the configuration to one or more gateways. +5. Send application traffic through the deployed gateway. + +Changes you make in AI Workspace don't affect live traffic until you deploy them to a gateway. + +## What you can manage + +### AI gateways + +Connect and manage the AI Gateway runtimes that process your AI traffic. In AI Workspace, you can do the following: + +- Register gateways with AI Workspace. +- Deploy artifacts to one or more gateways. +- Monitor gateway status. +- View where artifacts are deployed. + +To register your first gateway, see [Set up an AI Gateway](ai-gateways/setting-up.md). + +### LLM providers + +Configure connections to upstream AI services such as OpenAI, Anthropic, Azure OpenAI, Gemini, Mistral AI, and AWS Bedrock. + +An LLM provider holds the information required to connect to an upstream AI service, including its endpoint and authentication configuration. + +To configure a connection, see [LLM providers](llm-providers/overview.md). To connect a service that has no built-in template, see [LLM provider templates](llm-provider-templates/overview.md). + +### App LLM proxies + +Create application-facing endpoints when an application or agent needs configuration or policies that differ from the underlying LLM provider. + +To decide whether you need one, see [App LLM proxies](llm-proxies/overview.md). + +### MCP proxies + +Create managed endpoints for upstream MCP servers. MCP standardizes how applications share context and tools with LLMs. + +To put a gateway in front of an MCP server, see [MCP proxies](mcp-proxies/overview.md). + +### AI policies + +Apply policies to LLM providers, App LLM proxies, and MCP proxies to control how the gateway handles AI traffic. Policies fall into three groups: + +- Guardrails for content safety, personally identifiable information (PII) protection, validation, and prompt injection detection. +- Rate limits for requests, tokens, and monetary usage. +- Traffic and prompt controls for routing, prompt templates, semantic caching, and provider transformations. + +To see what you can attach and where it takes effect, see [Policies](policies/overview.md). + +### Secrets + +Store credentials securely and reference them from your AI artifacts, so credential values never appear in application or gateway configuration. + +To store and reference a credential, see [Secrets management](secrets-management.md). + +## Control plane and data plane + +AI Workspace and AI Gateway work together as the control plane and the data plane. The following table compares their roles: + +| | AI Workspace | AI Gateway | +|---|---|---| +| **Role** | Control plane | Data plane | +| **Purpose** | Manage AI configurations and policies | Process AI traffic | +| **Handles** | Artifacts, policies, secrets, and deployments | Requests and responses | +| **When it acts** | When you configure or deploy | When an application sends a request | + +The workspace tracks which artifacts are deployed to which gateways. For example, you can deploy a single LLM provider to multiple gateways, and each gateway can serve multiple providers and proxies. + +To configure and operate the gateway runtime directly, see the [AI Gateway documentation](../../ai-gateway/next/overview.md). + +## Configure AI Workspace + +Before you create AI artifacts, you can configure how AI Workspace runs. The following table lists the available configuration topics: + +| Topic | Description | +|---|---| +| [Configuration and interpolation](setting-up/configuration.md) | Configure services using `config.toml`, environment variables, and mounted files | +| [Ports](setting-up/ports.md) | Configure the ports that AI Workspace uses | +| [Database](setting-up/database.md) | Configure PostgreSQL or SQL Server for artifact storage | +| [Authentication](setting-up/authentication/overview.md) | Configure local authentication or an OpenID Connect (OIDC) identity provider | + +## Create and deploy artifacts + +AI Workspace supports several ways to create and manage artifacts. The following table lists the available methods: + +| Method | Description | +|---|---| +| AI Workspace | Create and manage artifacts from the AI Workspace console | +| AI Gateway | Create artifacts through the gateway management API or configuration files | +| Continuous integration and continuous delivery (CI/CD) | Manage artifacts as files in source control and deploy them using the `ap` CLI | + +To set up a Git-based workflow, see [AI Workspace CI/CD](ci-cd/overview.md). To bring artifacts that a gateway created back into the workspace, see [Sync gateway-created artifacts](sync-gateway-created-artifacts.md). + +## Monitor AI traffic + +AI Gateway publishes traffic and usage information that shows how your applications use AI services. The published information covers the following: + +- Request and token usage +- Latency +- Cost +- Guardrail events + +To view this information and understand AI usage across applications and consumers, see [Insights](insights.md). + +## Where to start + +If you haven't used AI Workspace before, follow [Get started with AI Workspace](getting-started.md). It walks you through running AI Workspace locally, connecting your first AI Gateway, and configuring an LLM provider. + +Otherwise, choose the path that matches what you want to do: + +- Connect a gateway: [Set up an AI Gateway](ai-gateways/setting-up.md) +- Connect an AI service: [LLM providers](llm-providers/overview.md) +- Create an application-specific endpoint: [App LLM proxies](llm-proxies/overview.md) +- Connect an MCP server: [MCP proxies](mcp-proxies/overview.md) +- Control AI traffic: [Policies](policies/overview.md) +- Manage credentials: [Secrets management](secrets-management.md) +- Call AI services from an application: [Invoke providers and proxies via SDKs](using-sdks.md) diff --git a/en/docs/ai-workspace/1.0.0/policies/apply-ai-policies-to-proxies.md b/en/docs/ai-workspace/1.0.0/policies/apply-ai-policies-to-proxies.md new file mode 100644 index 000000000..5ecb34bbf --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/policies/apply-ai-policies-to-proxies.md @@ -0,0 +1,106 @@ +--- +title: "Apply AI policies to LLM providers, App LLM proxies, and MCP proxies" +description: "Sync a custom AI policy from your AI Gateway to your organization and apply it to LLM providers, App LLM proxies, and MCP proxies." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/apply-ai-policies-to-proxies/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/apply-ai-policies-to-proxies.md +tags: + - cloud + - ai-workspace + - custom-policy +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-24 +content_type: "how-to" +--- + +# Apply AI policies to proxies + +After [building the AI Gateway with your custom AI policy](build-gateway-with-ai-policies.md) and starting it, the gateway automatically sends its policy manifest to the control plane on connection. You can then sync your custom AI policy to the organization and apply it to your large language model (LLM) providers, App LLM proxies, and Model Context Protocol (MCP) proxies. + +## Step 1: View the gateway policies + +1. Go to your AI Workspace. +2. From the left navigation, select **AI Gateways**. +3. Select your gateway to open the gateway detail view. +4. Click the **Policies** tab. + +!!! note + Each time the gateway connects to the control plane, it sends an updated manifest with its policy details. The Policies tab always reflects the most recent manifest received. + +The console fetches the manifest from the gateway and displays a table of all policies installed on it, with the following columns: + +| Column | Description | +|---|---| +| **Name** | Policy name | +| **Version** | Installed version | +| **Description** | Policy description | +| **Policy Type** | `Policy Hub` for policy hub managed policies, `Custom` for your own AI policies | +| **Sync Status** | Whether the policy is synced to the organization — shows a **Sync** button when the policy is unsynced or a later version is available, or **Latest Version Available**/**N/A** if already up to date | + +![AI Gateway detail page showing the Policies tab with a table of installed policies, their Policy Type, and Sync Status](../../../assets/img/custom-policies/ViewGatewayPolicies.svg) + +## Step 2: Sync the custom AI policy to the organization + +Sync a custom AI policy to the organization before you apply it to an LLM provider, App LLM proxy, or MCP proxy. + +In the **Sync Status** column, each custom AI policy shows one of the following: + +- **Sync button** — the policy is unsynced, or a later version is available. Click **Sync** to register it in the organization. +- **Latest Version Available** — the policy is already synced and up to date. No action needed. + +!!! note + Policy Hub policies, which WSO2 manages, show **N/A** in the Sync Status column and can't be synced manually. + +Once synced, the custom AI policy is available organization-wide and can be applied to LLM providers, App LLM proxies, and MCP proxies. + +![Policies tab after syncing, showing the Sensitive Data Mask custom policy with Policy Type Custom and Sync Status Latest Version Available](../../../assets/img/custom-policies/SyncCustomPolicy.png) + +!!! note + - Each major version of a custom AI policy is maintained as a separate policy entry with the same name. + - A minor version update re-enables the Sync button, so you can sync the later version to the organization. + - Patch version updates aren't supported. + - Version downgrades aren't allowed. + +## Step 3: View organization-level custom policies + +After syncing, the custom AI policy appears in **Settings > Custom Policies**. To view it: + +1. From the left navigation, select **Settings**. +2. Select **Custom Policies**. + +This section lists all custom AI policies available in the organization with the following details: + +| Column | Description | +|---|---| +| **Name** | Policy name | +| **Version** | Synced version | +| **Description** | Policy description | +| **Created At** | Date and time the policy was first synced | +| **Updated At** | Date and time the policy was last updated | + +![Settings > Custom Policies page listing the synced sensitive-data-mask policy with its version, description, and timestamps](../../../assets/img/custom-policies/ShowinCustomPolicySection.png) + +!!! note + To delete a synced custom AI policy, none of the LLM providers, App LLM proxies, or MCP proxies in the organization should be using it. + +## Step 4: Apply the custom AI policy + +Once synced, a custom AI policy is attached the same way for LLM providers, App LLM proxies, and MCP proxies — it appears alongside the built-in guardrails and policies wherever policies are configured for that resource: + +1. Navigate to **AI Workspace** and open the **LLM Providers**, **App LLM Proxies**, or **MCP Proxies** list, then click on the resource you want to configure. +2. Go to the **Guardrails & Policies** tab (MCP proxies use a **Policies** tab). +3. Click **+ Add Guardrail** / **+ Add Policy** in the section for the scope you want: + - **Global Guardrails** or **Global Policies**—applies to every endpoint or capability of the resource. + - A specific endpoint or capability card (expand it first)—applies only there. +4. Select your custom AI policy from the sidebar — it's listed alongside the built-in policies — and configure its parameters. +5. Click **Add** or **Submit** to attach it, then deploy or redeploy the resource to the gateway to apply the changes. + +![Guardrails & Policies sidebar showing the synced custom policy listed alongside built-in policies](../../../assets/img/custom-policies/SyncedCustomPolicyinPoliciesSection.svg) + +For more detail on the global and per-resource scopes, see [Policy scope: global or per resource](overview.md#policy-scope-global-or-per-resource). + +## Next steps + +- [LLM providers overview](../llm-providers/overview.md): configure LLM providers +- [App LLM proxies overview](../llm-proxies/overview.md): configure App LLM proxies for GenAI applications and agents +- [MCP proxies overview](../mcp-proxies/overview.md): configure MCP proxies +- [Policies overview](overview.md): explore the built-in guardrails available alongside your custom AI policies diff --git a/en/docs/ai-workspace/1.0.0/policies/build-gateway-with-ai-policies.md b/en/docs/ai-workspace/1.0.0/policies/build-gateway-with-ai-policies.md new file mode 100644 index 000000000..86ce1e5e2 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/policies/build-gateway-with-ai-policies.md @@ -0,0 +1,194 @@ +--- +title: "Build the AI Gateway with custom AI policies" +description: "Install the AP CLI, configure build.yaml, and build a custom AI Gateway image that includes your custom AI policies." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/build-gateway-with-ai-policies/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/build-gateway-with-ai-policies.md +tags: + - cloud + - ai-workspace + - custom-policy +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-24 +content_type: "how-to" +--- + +# Build the gateway with AI policies + +After [writing a custom AI policy](writing-an-ai-policy.md), build it into the AI Gateway image so it can run alongside the built-in guardrails and rate-limiting policies. + +## Install the ap CLI + +The `ap` CLI builds a custom gateway image that carries your own policies. It's the same `ap` CLI that [AI Workspace CI/CD](../ci-cd/overview.md) uses for commands such as `ap gateway apply` and `ap ai-workspace build`. If you already have it installed for that workflow, skip ahead to [Configure the build file](#configure-the-build-file). Otherwise, download the binary for your platform from the [AP CLI releases page](https://github.com/wso2/api-platform/releases/tag/ap%2Fv0.7.0) and follow the steps below to install it. + +=== "macOS / Linux" + + **Step 1: Extract the binary** + + After downloading the zip file for your platform, extract it: + + ```bash + unzip ap-darwin-amd64-v0.7.0.zip # replace with your downloaded filename + ``` + + **Step 2: Move the binary to a bin directory** + + ```bash + mkdir -p ~/bin + mv ap ~/bin/ + ``` + + **Step 3: Add to PATH** + + Add the following line to your `~/.zshrc` or `~/.bashrc`: + + ```bash + export PATH="$HOME/bin:$PATH" + ``` + + **Step 4: Reload your shell** + + ```bash + source ~/.zshrc # or source ~/.bashrc + ``` + + **Step 5: Verify the installation** + + ```bash + ap --version + ``` + +=== "Windows" + + **Step 1: Extract the binary** + + After downloading the zip file, right-click it and select **Extract All**, or run in PowerShell: + + ```powershell + Expand-Archive -Path ap-windows-amd64.zip -DestinationPath ap-windows-amd64 + ``` + + **Step 2: Move the binary to a bin directory** + + ```powershell + New-Item -ItemType Directory -Force -Path "$HOME\bin" + Move-Item ap-windows-amd64\ap.exe "$HOME\bin\ap.exe" + ``` + + **Step 3: Add to PATH** + + Run the following in PowerShell to permanently add `~/bin` to your user PATH: + + ```powershell + [Environment]::SetEnvironmentVariable("Path", $env:Path + ";$HOME\bin", "User") + ``` + + **Step 4: Reload your shell** + + Close and reopen PowerShell for the PATH change to take effect. + + **Step 5: Verify the installation** + + ```powershell + ap --version + ``` + +## Configure the build file + +The `build.yaml` file is included in the AI Gateway package you downloaded when [setting up the AI Gateway](../ai-gateways/setting-up.md). It declares the gateway version and the list of policies to include. Edit this file to add your custom AI policies before building the gateway image. + +!!! tip "Sample custom policies" + WSO2 provides sample custom policies, including AI-specific examples, in the [api-platform sample policies repository](https://github.com/wso2/api-platform/tree/main/gateway/sample-policies). + +### Structure + +```yaml +version: v1 +gateway: + version: 1.2.0-beta +policies: + - name: + gomodule: @ # for policy hub managed policies + - name: + filePath: # for custom AI policies +``` + +Each policy entry uses one of two source types: + +| Field | Description | +|---|---| +| `gomodule` | The Go module reference for a Policy Hub managed policy, for example `github.com/wso2/gateway-controllers/policies/pii-masking@v1` | +| `filePath` | The path from `build.yaml` to a local custom AI policy directory | + +### Add a custom AI policy + +Your custom AI policy can reside anywhere on the filesystem. Use a relative path from the `build.yaml` file to point to it. + +For example, if your directory layout is: + +```text +parent-directory/ +├── my-ai-gateway/ +│ └── build.yaml +└── my-ai-policy/ # policy lives outside the gateway directory + ├── policy-definition.yaml + └── myaipolicy.go +``` + +Add the policy to `build.yaml` using a relative `filePath`: + +```yaml +policies: + - name: my-ai-policy + filePath: ../my-ai-policy +``` + +!!! note + The path in `filePath` is always relative to the location of `build.yaml`, not the directory from which you run the `ap` command. + +## Build the gateway image + +Once `build.yaml` is ready, run the following command from the directory containing `build.yaml` to build the custom gateway image: + +```bash +ap gateway image build +``` + +This packages the gateway runtime together with every listed policy, built-in and custom, into a container image you use in place of the standard AI Gateway image. + +Once the build completes, the output lists the two image names produced. For example: + +```text +✓ Built gateway images with 1 policies: + • ghcr.io/wso2/api-platform/wso2apip-ai-gateway-1.2.0-beta-gateway-runtime:1.2.0-beta + • ghcr.io/wso2/api-platform/wso2apip-ai-gateway-1.2.0-beta-gateway-controller:1.2.0-beta +``` + +A `build-manifest.yaml` file is also written alongside `build.yaml`, recording the resolved versions of all policies included in the build. + +## Update the Docker Compose file + +After building, update the `image:` fields in your `docker-compose.yaml` to use the newly built images. + +Locate the `gateway-controller` and `gateway-runtime` services and replace their `image:` values with the images from the build output: + +```yaml +services: + gateway-controller: + image: ghcr.io/wso2/api-platform/wso2apip-ai-gateway-1.2.0-beta-gateway-controller:1.2.0-beta # (1) + + gateway-runtime: + image: ghcr.io/wso2/api-platform/wso2apip-ai-gateway-1.2.0-beta-gateway-runtime:1.2.0-beta # (2) +``` + +1. Replace with the `gateway-controller` image name from your build output. +2. Replace with the `gateway-runtime` image name from your build output. + +Once updated, start the gateway as usual. `api-platform.env` continues to be loaded automatically via the Compose `env_file:` directive: + +```bash +docker compose up +``` + +## Next steps + +- [Apply AI policies to proxies](apply-ai-policies-to-proxies.md): sync your custom AI policy to the organization and apply it to LLM providers, App LLM proxies, and MCP proxies diff --git a/en/docs/ai-workspace/1.0.0/policies/overview.md b/en/docs/ai-workspace/1.0.0/policies/overview.md new file mode 100644 index 000000000..1da4d913c --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/policies/overview.md @@ -0,0 +1,292 @@ +--- +title: "AI Workspace policies overview" +description: "What AI and MCP policies you can attach to LLM providers and proxies in AI Workspace, where each one is applied, and how the rate limit policies cap requests, tokens, and spend." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/overview.md +tags: + - cloud + - ai-workspace + - policies + - rate-limit +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-31 +content_type: "overview" +--- + +# Policies overview + +AI Workspace is where you attach policies to your large language model (LLM) providers and App LLM proxies and deploy them. The AI Gateway is what enforces them at request time. + +That split runs through this section. This page covers what's available to attach and where it takes effect in the workspace. For what a policy does to a request, its configuration fields, and its error behavior, follow each policy through to the [Policy Hub](https://wso2.com/api-platform/policy-hub/), the registry that holds the specification and version history for every policy listed here. + +Rate limiting is the exception: it's documented in full on this page, under [Rate limiting](#rate-limiting). + +## Where policies are applied + +Policies are configured through the management tabs of your LLM providers and App LLM proxies: + +- **LLM provider** — rate limits and guardrails configured on a provider apply to every proxy that uses it. +- **App LLM proxy** — guardrails configured on a proxy specialize the behavior for one GenAI application or agent. + +When both provider-level and proxy-level policies are active, both are enforced. Provider-level policies act as a baseline, and proxy-level policies add to it. + +## Policy scope: global or per resource + +Within a provider or proxy, each policy is attached at one of two scopes: + +| Scope | Applies to | Counter behavior | +|-------|-----------|------------------| +| **Global** | Every endpoint of the provider or proxy | One shared counter across all endpoints | +| **Per resource** | A specific endpoint (path and method) | An independent counter per endpoint | + +The difference is the scope of the counter for rate limits, and the breadth of application for guardrails: + +- A **global** rate limit keeps one bucket for the whole provider or proxy. With a global limit of 100 requests per hour, 60 requests to `/chat/completions` plus 40 to `/embeddings` exhausts it, and the next request to either endpoint is rejected. +- A **per-resource** rate limit keeps an independent bucket per endpoint. The same limit attached to both endpoints allows 100 on each, counted separately. +- A **global** guardrail runs on every endpoint; a **per-resource** guardrail runs only on the endpoints you attach it to. + +Global policies are evaluated before per-resource policies. Because of that ordering, a global rate limit counts every request attempt, including ones a tighter per-resource limit goes on to reject — making it a hard ceiling on total traffic through the provider or proxy. + +## AI policies + +These policies attach to LLM providers and App LLM proxies. Each links to its full reference. + +### Guardrails + +Guardrails inspect and act on request and response content: + +| Guardrail | What it does | +|-----------|--------------| +| [Semantic prompt guard](https://wso2.com/api-platform/policy-hub/policies/semantic-prompt-guard) | Blocks or allows prompts by semantic similarity to configured phrases. Needs an embedding provider. | +| [PII masking regex](https://wso2.com/api-platform/policy-hub/policies/pii-masking-regex) | Masks personally identifiable information (PII) using regex patterns, and restores it in the response. | +| [Azure content safety](https://wso2.com/api-platform/policy-hub/policies/azure-content-safety-content-moderation) | Filters harmful content using Azure Content Safety moderation. Needs an Azure subscription. | +| [Granite Guardian prompt injection](https://wso2.com/api-platform/policy-hub/policies/granite-guardian-prompt-injection) | Detects prompt injection and jailbreak attempts using IBM Granite Guardian, and rejects flagged requests. Needs a Granite Guardian inference endpoint. | +| [NeMo Guard content safety](https://wso2.com/api-platform/policy-hub/policies/nvidia-nemoguard-content-safety) | Classifies requests and responses across NVIDIA NeMo Guard safety categories. Needs a NeMo Guard inference endpoint. | +| [Word count](https://wso2.com/api-platform/policy-hub/policies/word-count-guardrail) | Enforces word count limits on prompts or responses. | +| [Sentence count](https://wso2.com/api-platform/policy-hub/policies/sentence-count-guardrail) | Enforces sentence count limits on prompts or responses. | +| [Content length](https://wso2.com/api-platform/policy-hub/policies/content-length-guardrail) | Enforces byte-length limits on prompts or responses. | +| [JSON schema](https://wso2.com/api-platform/policy-hub/policies/json-schema-guardrail) | Validates content against a JSON schema. | +| [Regex](https://wso2.com/api-platform/policy-hub/policies/regex-guardrail) | Blocks or allows content matching a regular expression. | +| [URL](https://wso2.com/api-platform/policy-hub/policies/url-guardrail) | Validates URLs found in content. | +| [AWS Bedrock guardrail](https://wso2.com/api-platform/policy-hub/policies/aws-bedrock-guardrail) | Applies an AWS Bedrock guardrail to requests and responses. | +| [Semantic tool filtering](https://wso2.com/api-platform/policy-hub/policies/semantic-tool-filtering) | Filters the tools exposed to a model by semantic relevance to the user query. | + +### Traffic management and prompt policies + +These policies shape how requests are routed and composed: + +| Policy | What it does | +|--------|--------------| +| [Model round robin](https://wso2.com/api-platform/policy-hub/policies/model-round-robin) | Distributes requests across models in round-robin order. | +| [Model weighted round robin](https://wso2.com/api-platform/policy-hub/policies/model-weighted-round-robin) | Distributes requests across models by assigned weight. | +| [LLM header router](https://wso2.com/api-platform/policy-hub/policies/llm-header-router) | Selects the target provider from a request header, so one OpenAI-shaped endpoint can fan out to several providers. | +| [Prompt decorator](https://wso2.com/api-platform/policy-hub/policies/prompt-decorator) | Prepends or appends content to every request. | +| [Prompt template](https://wso2.com/api-platform/policy-hub/policies/prompt-template) | Applies reusable parameterized prompt templates. | +| [Prompt compressor](https://wso2.com/api-platform/policy-hub/policies/prompt-compressor) | Compresses selected prompt text before the upstream call, using JSONPath targeting and rule-based thresholds. | +| [Semantic caching](https://wso2.com/api-platform/policy-hub/policies/semantic-cache) | Caches responses and serves them for semantically similar requests. | +| [Respond](https://wso2.com/api-platform/policy-hub/policies/respond) | Returns a response immediately without calling the upstream, for mocking and short-circuit logic. | + +### Provider transformation policies + +These policies translate an OpenAI Chat Completions request into another provider's API shape, and translate the response back. Pair them with the [LLM header router](https://wso2.com/api-platform/policy-hub/policies/llm-header-router) to route one endpoint across several providers. Use one on its own to point a single OpenAI-shaped endpoint at a different provider. For an end-to-end example configured on the gateway, see [Multi-provider routing](../../../ai-gateway/next/llm-proxy/multi-provider-routing.md). + +| Policy | Target provider | +|--------|-----------------| +| [OpenAI to Anthropic transformer](https://wso2.com/api-platform/policy-hub/policies/openai-to-anthropic-transformer) | Anthropic Messages API | +| [OpenAI to Azure OpenAI transformer](https://wso2.com/api-platform/policy-hub/policies/openai-to-azure-openai-transformer) | Azure OpenAI | +| [OpenAI to Bedrock transformer](https://wso2.com/api-platform/policy-hub/policies/openai-to-bedrock-transformer) | AWS Bedrock Converse | +| [OpenAI to Gemini transformer](https://wso2.com/api-platform/policy-hub/policies/openai-to-gemini-transformer) | Google Gemini `generateContent` | +| [OpenAI to Mistral transformer](https://wso2.com/api-platform/policy-hub/policies/openai-to-mistral-transformer) | Mistral | + +## Rate limiting + +AI services bill per token, so uncontrolled usage turns into unexpected cost. AI Workspace gives you five rate limit policies, each capping a different measure of traffic: + +| Policy | Caps | Specification | +|--------|------|---------------| +| [Rate limit - basic](#rate-limit-basic) | Request count | [Policy Hub](https://wso2.com/api-platform/policy-hub/policies/basic-ratelimit) | +| [Rate limit - advanced](#rate-limit-advanced) | Request count, with multi-dimensional and weighted quotas | [Policy Hub](https://wso2.com/api-platform/policy-hub/policies/advanced-ratelimit) | +| [Token-based rate limit](#token-based-rate-limit) | Prompt, completion, or total tokens | [Policy Hub](https://wso2.com/api-platform/policy-hub/policies/token-based-ratelimit) | +| [LLM cost](#llm-cost) | Nothing on its own—calculates the cost other policies spend against | [Policy Hub](https://wso2.com/api-platform/policy-hub/policies/llm-cost) | +| [LLM cost-based rate limit](#llm-cost-based-rate-limit) | Monetary spend in US dollars (USD) | [Policy Hub](https://wso2.com/api-platform/policy-hub/policies/llm-cost-based-ratelimit) | + +You attach all of them through the **Guardrails** tab of an LLM provider or App LLM proxy. LLM providers also have a built-in **Rate Limiting** tab that caps requests and tokens without attaching a policy. See [Rate limiting on a provider](../llm-providers/manage-provider.md#rate-limiting) for that tab. Use a policy when you need a per-route cap, token categories counted separately, or a spending budget. + +For how attachment scope changes what a counter covers, see [Policy scope: global or per resource](#policy-scope-global-or-per-resource). + +### Attach a rate limit policy + +The steps are the same for every rate limit policy: + +1. Navigate to **AI Workspace** > **LLM Providers** or **App LLM Proxies**. +2. Click the provider or proxy name. +3. Go to the **Guardrails** tab. +4. Click **+ Add Guardrail** and select the policy from the sidebar. +5. Configure its limits. +6. Click **Add** for a provider, or **Submit** for a proxy. +7. Deploy the provider or proxy to apply the change. + +### Rate limit - basic + +Caps the number of requests within a time window, regardless of token consumption. + +Configure a list of rules, each with a `count` of requests and a `duration` time window. When you configure several, the most restrictive one is enforced. + +**Behavior** + +- Requests are counted per route or API. +- Exceeding the limit within the window returns HTTP `429 Too Many Requests`. +- The counter resets once the duration elapses. + +### Rate limit - advanced + +Caps request count like the basic policy, with more control over how requests are counted and where counters are stored. Use it when a single count-per-window rule isn't enough. + +**Behavior** + +- Chooses between the generic cell rate algorithm (GCRA), for smooth limiting, and fixed window, for a plain counter. +- Supports several quotas at once, each with its own key extraction and cost extraction. +- Weights requests, so one request can consume more than one unit of quota. +- Stores counters in memory, in Redis, or in Redis with a local async cache. If Redis is unreachable, it fails open and lets requests through. +- Returns `X-RateLimit-*`, IETF `RateLimit-*`, and `Retry-After` response headers. + +!!! warning "Redis failures disable quota enforcement" + With a Redis-backed counter, a Redis outage stops quotas being enforced rather than blocking traffic. Requests pass through unrestricted for as long as Redis is unreachable, which can drive unbounded upstream traffic and LLM spend. Alert on Redis availability, and pair the policy with a limit that doesn't depend on Redis — an in-memory rate limit, or a [token-based](#token-based-rate-limit) or [LLM cost-based](#llm-cost-based-rate-limit) limit — if you need a ceiling that survives the outage. + +For the full configuration schema, see [Rate limit - advanced](https://wso2.com/api-platform/policy-hub/policies/advanced-ratelimit) in the Policy Hub. + +### Token-based rate limit + +Caps token consumption rather than request count. Limits apply to prompt (input) tokens, completion (output) tokens, or total tokens — independently or in combination. + +Configure at least one of the three categories; any combination is valid. Each entry takes a `count` and a `duration`. Within a category, the most restrictive limit is enforced. + +**Behavior** + +- Token counts are extracted from the provider's response using paths defined in the provider template. +- Exceeding any configured limit returns `429 Too Many Requests`. +- Each counter resets after its own duration elapses. +- Responses carry rate limit headers: + +| Header | Description | +|--------|-------------| +| `X-RateLimit-Limit` | Configured token limit | +| `X-RateLimit-Remaining` | Remaining tokens in the current window | +| `X-RateLimit-Reset` | Time, in epoch seconds, when the window resets | +| `RateLimit-*` | Internet Engineering Task Force (IETF) equivalents of the above | + +**Example: cap total tokens per minute** + +Block requests once 100,000 total tokens are consumed in a one-minute window: + +| Parameter | Value | +|-----------|-------| +| Total token limits — count | `100000` | +| Total token limits — duration | `1m` | + +**Example: separate prompt and completion limits** + +Limit prompt tokens to 50,000 per hour and completion tokens to 20,000 per hour, counted independently: + +| Category | count | duration | +|----------|-------|----------| +| Prompt token limits | `50000` | `1h` | +| Completion token limits | `20000` | `1h` | + +### LLM cost + +Calculates the monetary cost of each LLM call and makes the result available to other policies — primarily [LLM cost-based rate limit](#llm-cost-based-rate-limit). It runs in the response phase, takes no configuration, and never exposes the cost to the caller. + +**How it works** + +1. When the LLM response arrives, including streaming (SSE) responses, the policy reads the model name from the response body. +2. It looks the model up in the built-in pricing database. +3. It calculates the cost in USD from token usage, context window tier, and service tier. +4. It stores the result in `SharedContext.Metadata["x-llm-cost"]` as a 10-decimal USD string, for example `"0.0000423100"`. + +**Supported providers** + +| Provider | Notes | +|----------|-------| +| **OpenAI** | All models, including o-series reasoning tokens, batch API, and flex and priority service tiers | +| **Anthropic** | Claude models, including prompt caching (read and write tokens), extended thinking, and speed and geo routing | +| **Amazon Web Services (AWS) Bedrock** | Supported Bedrock models; extracts the model identifier (ID) from the request path | +| **Google Gemini** | Google AI Studio and Vertex AI, including multi-modal (audio, image), web search grounding, and thinking models | +| **Mistral** | All Mistral models, including audio duration-based billing (Voxtral) | + +**Behavior** + +- Handles streaming and non-streaming responses without configuration. +- Supports context-window tiered pricing, and the standard, priority, flex, and batch service tiers. +- If the model isn't in the pricing database, the cost is set to `0` and a warning is logged. The request isn't blocked. +- The pricing database ships with the gateway image and loads at startup, so a gateway restart is needed to pick up pricing updates. Its path is a gateway-level setting in `config.toml`. + +**Metadata written** + +| Key | Value | +|-----|-------| +| `x-llm-cost` | Cost in USD as a 10-decimal string, for example `"0.0000423100"` | +| `x-llm-cost-status` | `"calculated"` on success, `"not_calculated"` if the cost couldn't be determined | + +### LLM cost-based rate limit + +Enforces monetary spending budgets. It reads the per-call cost that the [LLM cost](#llm-cost) policy calculates, and blocks requests once a budget is exhausted within its time window. + +!!! important "Add LLM cost after this policy in the list" + The LLM cost policy must be attached to the same provider or proxy, and must sit **after** this one in the policy list. The gateway evaluates response-phase policies in reverse order, so that ordering is what calculates the cost before the budget is checked. Without it, no cost data exists and budget enforcement is skipped silently. + +Configure one or more budget limits, each with an `amount` in USD and a `duration` time window. When you configure several, all are enforced and the most restrictive active limit applies. + +**Behavior** + +- Cost is read from `x-llm-cost` in shared metadata. +- Accumulated spend is tracked per route within each window. +- Once spend reaches the budget, subsequent requests receive `429 Too Many Requests`. +- Each window resets automatically when its duration elapses. +- Responses carry both internal-unit and USD values: + +| Header | Description | +|--------|-------------| +| `X-RateLimit-Limit` | Budget limit in internal scaled units | +| `X-RateLimit-Remaining` | Remaining budget in internal scaled units | +| `x-ratelimit-cost-limit-dollars` | Budget limit in USD, for example `10.000000` | +| `x-ratelimit-cost-remaining-dollars` | Remaining budget in USD, for example `7.432100` | + +**Example: a $10 hourly and $100 daily budget** + +These two limits apply at once. Both must be satisfied for a request to proceed, and reaching either blocks requests until that window resets: + +| Budget limit | amount | duration | +|--------------|--------|----------| +| Hourly cap | `10` | `1h` | +| Daily cap | `100` | `24h` | + +## MCP policies + +These policies attach to Model Context Protocol (MCP) proxies. To attach them in the workspace, see [Apply policies to an MCP proxy](../mcp-proxies/apply-policies.md). + +| Policy | What it does | +|--------|--------------| +| [MCP access control](https://wso2.com/api-platform/policy-hub/policies/mcp-acl-list) | Allows or denies access to specific tools and resources. | +| [MCP authentication](https://wso2.com/api-platform/policy-hub/policies/mcp-auth) | Authenticates callers of the MCP proxy. | +| [MCP authorization](https://wso2.com/api-platform/policy-hub/policies/mcp-authz) | Authorizes tool and resource calls against the caller's privileges. | +| [MCP rewrite](https://wso2.com/api-platform/policy-hub/policies/mcp-rewrite) | Rewrites tool and resource definitions before they reach the client. | +| [MCP rate limit](https://wso2.com/api-platform/policy-hub/policies/mcp-ratelimit) | Limits the rate of MCP tool and method calls. | +| [CORS](https://wso2.com/api-platform/policy-hub/policies/cors) | Handles cross-origin resource sharing (CORS) preflight requests and adds CORS response headers. | +| [Set headers](https://wso2.com/api-platform/policy-hub/policies/set-headers) | Sets or appends request and response headers. | +| [Remove headers](https://wso2.com/api-platform/policy-hub/policies/remove-headers) | Removes request or response headers by name. | +| [Log message](https://wso2.com/api-platform/policy-hub/policies/log-message) | Logs request and response headers and payloads, including each chunk of a streaming response. | + +## Custom policies + +Beyond the built-in policies, you can write your own AI policy and deploy it to a gateway. Once a gateway runs a custom policy, it syncs into AI Workspace and appears under **Settings > Custom Policies**, where you can search, review, and delete the policies synced from your gateways. From there, attach it to a provider or proxy the same way as a built-in policy. + +To build and roll one out end to end: + +1. [Write an AI policy](writing-an-ai-policy.md)—implement the policy using the gateway SDK +2. [Build the gateway with AI policies](build-gateway-with-ai-policies.md)—package the policy into a custom AI Gateway image +3. [Apply AI policies to proxies](apply-ai-policies-to-proxies.md)—sync the policy to the organization and attach it to a provider or proxy + +## Policy Hub + +The policies in AI Workspace are powered by the [Policy Hub](https://wso2.com/api-platform/policy-hub/), a central registry of available policies and their versions. Visit it to browse every available policy alongside its documentation and configuration schema. diff --git a/en/docs/ai-workspace/1.0.0/policies/writing-an-ai-policy.md b/en/docs/ai-workspace/1.0.0/policies/writing-an-ai-policy.md new file mode 100644 index 000000000..f11a8db06 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/policies/writing-an-ai-policy.md @@ -0,0 +1,262 @@ +--- +title: "Write an AI policy for the AI Gateway" +description: "Build a custom AI policy using the gateway SDK, including support for buffered and streaming (SSE) LLM request and response bodies." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/writing-an-ai-policy/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/writing-an-ai-policy.md +tags: + - cloud + - ai-workspace + - custom-policy + - sdk +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-24 +content_type: "how-to" +--- + +# Write an AI policy for the AI Gateway + +An AI policy inspects, controls, and modifies the traffic going to and from a large language model (LLM) provider such as OpenAI or Anthropic. + +AI policies use the same `Policy` interface as standard gateway policies. For full API details, see the [policy SDK reference](https://pkg.go.dev/github.com/wso2/api-platform/sdk/core/policy/v1alpha2). + +The key difference is how you handle LLM request and response bodies, especially: + +- JSON responses +- Streaming responses that use server-sent events (SSE) + +## How it works + +Every request and response that flows through the gateway passes through a **policy chain**. Each policy declares which phases it participates in, and the kernel calls the appropriate hook for each phase: + +```text +Incoming Request + │ + ▼ + Request Headers ──► OnRequestHeaders() + │ + ▼ + Request Body ──► OnRequestBody() (or OnRequestBodyChunk() for streaming) + │ + ▼ + Upstream LLM + │ + ▼ + Response Headers ──► OnResponseHeaders() + │ + ▼ + Response Body ──► OnResponseBody() (or OnResponseBodyChunk() for streaming) + │ + ▼ + Downstream Client +``` + +!!! note + A **policy chain** is an ordered sequence of policies that the gateway runs on every request and response for a given LLM provider or App LLM proxy. Policies run in the order the runtime configuration lists them, and each policy sees the modifications the ones before it made. + +## Key idea + +LLM responses come in two formats: + +| Mode | Format | +|------|--------| +| Non-streaming | Single JSON object | +| Streaming | SSE events (`data: {...}`) | + +Your policy must handle **both formats**. + +## Which interfaces to implement + +Choose based on what your policy needs to do: + +| Goal | Interface | Mode setting | +|------|-----------|-------------| +| Inspect prompt / model | `RequestPolicy` | `RequestBodyMode: BodyModeBuffer` | +| Inspect headers (auth, routing) | `RequestHeaderPolicy` | `RequestHeaderMode: HeaderModeProcess` | +| Inspect or modify buffered (in-memory) response | `ResponsePolicy` | `ResponseBodyMode: BodyModeBuffer` | +| Inspect or modify streaming response | `StreamingResponsePolicy` (embeds `ResponsePolicy`) | `ResponseBodyMode: BodyModeStream` | + +## How to write an AI policy + +### Step 1: Create the policy + +Each policy lives in its own Go module. Create a `policies` directory inside your gateway: + +```text +/policies/my-ai-policy/ + ├── go.mod + ├── my_ai_policy.go + └── policy-definition.yaml +``` + +### Step 2: Implement the mode + +`Mode()` declares which phases this policy participates in and how bodies are handled. The kernel reads this once at startup — there is no per-request overhead. + +```go +package myaipolicy + +import ( + "bytes" + "context" + "fmt" + + policy "github.com/wso2/api-platform/sdk/core/policy/v1alpha2" +) + +type MyAIPolicy struct { + blockThreshold float64 +} + +func (p *MyAIPolicy) Mode() policy.ProcessingMode { + return policy.ProcessingMode{ + RequestBodyMode: policy.BodyModeBuffer, + ResponseBodyMode: policy.BodyModeStream, + } +} +``` + +!!! tip + If your policy doesn't need to inspect a phase, set that phase to `HeaderModeSkip` or `BodyModeSkip`. + +### Step 3: Implement request inspection + +`OnRequestBody` is called once the request body is fully buffered. Use it to inspect the model name, messages, or parameters before the request reaches the LLM provider. + +```go +func (p *MyAIPolicy) OnRequestBody( + ctx context.Context, + reqCtx *policy.RequestContext, + params map[string]interface{}, +) policy.RequestAction { + // Inspect model + messages + return nil +} +``` + +### Step 4: Implement response handling + +For most AI policies, implement both: + +- **`ResponsePolicy`**: handles buffered responses, where the entire response is available at once. That's either a non-streaming JSON response or concatenated SSE events. +- **`StreamingResponsePolicy`**: handles streaming responses, either JSON or SSE events. + +!!! tip + The gateway automatically chooses which handler to call. The gateway calls `OnResponseBodyChunk` only when the entire policy chain is streaming-compatible. If any policy in the chain doesn't support streaming, the gateway falls back to `OnResponseBody`, so implement both even when streaming is your primary target. + +```go +// Streaming Response Handling +func (p *MyAIPolicy) OnResponseBodyChunk( + ctx context.Context, + respCtx *policy.ResponseStreamContext, + chunk *policy.StreamBody, + params map[string]interface{}, +) policy.StreamingResponseAction { + // Accumulate + process + return policy.ForwardResponseChunk{} +} + +// Gating response chunks before processing response +func (p *MyAIPolicy) NeedsMoreResponseData(_ []byte) bool { + return false +} + +// Buffered Fallback +func (p *MyAIPolicy) OnResponseBody( + ctx context.Context, + respCtx *policy.ResponseContext, + params map[string]interface{}, +) policy.ResponseAction { + // Same logic as streaming, applied to the full body + return nil +} +``` + +#### Gate-then-stream pattern + +A common pattern for AI guardrails is to accumulate chunks until you have a complete SSE event to inspect, then switch to pass-through: + +```go +// Buffer until we can parse a complete SSE event, then stream freely +func (p *MyAIPolicy) NeedsMoreResponseData(accumulated []byte) bool { + return !bytes.Contains(accumulated, []byte("\n\n")) +} +``` + +### Step 5: Factory function + +Initialize your policy and validate parameters: + +```go +func GetPolicy( + metadata policy.PolicyMetadata, + params map[string]interface{}, +) (policy.Policy, error) { + + threshold, ok := params["blockThreshold"].(float64) + if !ok { + return nil, fmt.Errorf("invalid blockThreshold") + } + + return &MyAIPolicy{blockThreshold: threshold}, nil +} +``` + +### Step 6: Define parameters + +Create a `policy-definition.yaml` in your policy directory: + +```yaml +name: my-ai-policy +displayName: my ai policy +version: v1.0.0 + +parameters: + type: object + properties: + blockThreshold: + type: number + default: 0.8 +``` + +### Step 7: Share data between phases + +Use the `Metadata` map to pass data between request and response phases — for example, the model name read from the request, used later to apply model-specific logic in the response phase: + +```go +// In request phase +reqCtx.Metadata["model"] = model + +// In response phase +model := respCtx.Metadata["model"] +``` + +### Step 8: Register and build + +Add your policy to the gateway folder's `build.yaml` under `policies:` using `filePath` for local development: + +```yaml +policies: + - name: my-ai-policy + filePath: ./policies/my-ai-policy +``` + +For published policies (production), use the module reference instead: + +```yaml +policies: + - name: my-ai-policy + gomodule: github.com/abc/policy-repo/policies/my-ai-policy@v1 +``` + +## Best practices + +- **Always handle both streaming and non-streaming.** The gateway falls back to buffered mode when any policy in the chain doesn't support streaming. +- **Use the `Metadata` map to share state.** Pass data between the request and response phases through it. +- **Implement streaming and the buffered fallback.** Your policy then works whether the chain runs in streaming or buffered mode. +- **Parse SSE incrementally.** When gating on streaming responses, buffer only until you have a complete SSE event, terminated by `\n\n`, rather than the entire response. This keeps latency low. + +## Next steps + +- [Build the gateway with AI policies](build-gateway-with-ai-policies.md): build a gateway image that includes your custom AI policy +- [Apply AI policies to proxies](apply-ai-policies-to-proxies.md): sync your custom AI policy to the organization and apply it to LLM providers, App LLM proxies, and MCP proxies +- [Write a custom policy for the self-hosted gateway](../../../cloud/api-platform-gateway/writing-a-custom-policy.md): the general-purpose policy SDK that the AI Gateway's policy engine builds on diff --git a/en/docs/ai-workspace/1.0.0/secrets-management.md b/en/docs/ai-workspace/1.0.0/secrets-management.md new file mode 100644 index 000000000..511fe9a49 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/secrets-management.md @@ -0,0 +1,352 @@ +--- +title: "Secrets management" +description: "Store and manage encrypted secrets in AI Workspace and reference them securely in artifact configurations without exposing plaintext credentials." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/secrets-management/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/secrets-management.md +tags: + - cloud + - ai-workspace + - secrets + - security +author: WSO2 API Platform Documentation Team +last_updated: 2026-06-26 +content_type: "how-to" +--- + +# Secrets management + +AI Workspace lets you store sensitive credentials as **secrets** and reference them securely in artifact configurations. Secrets are encrypted at rest using the Advanced Encryption Standard with 256-bit keys in Galois/Counter Mode (AES-256-GCM). Plaintext values are never written to the database and are **never returned in any API response**—not even the creation response. + +Use secrets to keep raw API keys, tokens, and passwords out of your artifact configurations. They apply to the following: + +- Large language model (LLM) provider configurations +- Model Context Protocol (MCP) proxy configurations +- API backend settings + +!!! important "Not the same as `config.toml` interpolation" + Secrets apply to artifact configurations only, and the gateway resolves the {% raw %}`{{ secret "handle" }}`{% endraw %} placeholder at request time. The services have their own startup credentials: the database password, the OpenID Connect (OIDC) client secret, and the at-rest encryption key. Those are supplied through the separate {% raw %}`{{ env }}`{% endraw %} and {% raw %}`{{ file }}`{% endraw %} tokens in `config.toml`. See [Sensitive values in `config.toml`](setting-up/configuration.md#sensitive-values-in-configtoml). The two placeholder sets aren't interchangeable. + +## How it works + +1. Create a secret via the Platform API with a unique `handle` and the plaintext `value`. +2. Reference the secret in any artifact configuration using the placeholder syntax: + + {% raw %} + + ```text + {{ secret "your-secret-handle" }} + ``` + + {% endraw %} + +3. When an artifact that contains a placeholder is deployed, the gateway resolves it with the decrypted value at runtime — the plaintext never appears in the control-plane database or configuration files. +4. To rotate a credential, call `PUT /api/v0.9/secrets/{handle}` with the new value. Because artifacts reference the secret by handle, no artifact changes or redeployment are required. + +## Automatic encryption in the AI Workspace UI + +When you create or update an **LLM Provider** or **MCP Proxy** through the AI Workspace UI and fill in an upstream API key or auth value, the UI automatically: + +1. Creates a secret via `POST /api/v0.9/secrets` using a deterministic handle derived from the resource ID (for example, `wso2-openai-provider-api-key`). +2. Substitutes the credential with the {% raw %}`{{ secret "handle" }}`{% endraw %} placeholder before saving the resource. + +The raw credential is sent to the secrets API only once and is never stored in the artifact configuration. Re-saving a resource that already contains a placeholder skips the secret creation step. + +## API reference + +The secrets API is available at `/api/v0.9/secrets`. All requests require a valid JSON Web Token (JWT) with the appropriate scope. + +### Required scopes + +| Scope | Grants | +|--------------------|------------------------------------------| +| `ap:secret:read` | List secrets and get metadata by handle | +| `ap:secret:create` | Create a new secret | +| `ap:secret:update` | Rotate or update a stored secret value | +| `ap:secret:delete` | Delete a secret | +| `ap:secret:manage` | All of the above | + +### Create a secret + +```http +POST /api/v0.9/secrets +Content-Type: multipart/form-data +``` + +Stores a new encrypted secret. The plaintext value is never returned — not even in this response. + +!!! note "Why multipart/form-data?" + `multipart/form-data` carries the `value` field as arbitrary content, so the same endpoint accepts text and binary secrets under one content type. + +**Request fields** + +| Field | Required | Description | +|-------|----------|-------------| +| `handle` | Yes | Unique identifier within the organization. Used in {% raw %}`{{ secret "handle" }}`{% endraw %} references. Immutable after creation. | +| `name` | Yes | Human-readable display name for the secret. | +| `value` | Yes | The sensitive value to encrypt and store. | +| `description` | No | Optional description. | +| `type` | No | Secret type. `GENERIC` (default) for API keys and tokens. | + +**Example request** + +```http +POST /api/v0.9/secrets +Authorization: Bearer +Content-Type: multipart/form-data; boundary=----FormBoundary + +------FormBoundary +Content-Disposition: form-data; name="handle" + +wso2-openai-key +------FormBoundary +Content-Disposition: form-data; name="name" + +WSO2 OpenAI API Key +------FormBoundary +Content-Disposition: form-data; name="description" + +API key for WSO2 OpenAI integration +------FormBoundary +Content-Disposition: form-data; name="type" + +GENERIC +------FormBoundary +Content-Disposition: form-data; name="value" + +sk-xxx +------FormBoundary-- +``` + +**Response — 201 Created** + +```json +{ + "uuid": "a1b2c3d4-...", + "handle": "wso2-openai-key", + "name": "WSO2 OpenAI API Key", + "createdAt": "2026-01-12T10:00:00Z", + "updatedAt": "2026-01-12T10:00:00Z" +} +``` + +The response doesn't include the `value`. Store the plaintext in a secure location before submitting it—you can't retrieve it later. + +**Error responses** + +| Status | Reason | +|--------|--------| +| 400 | Missing required fields or invalid request | +| 409 | A secret with the same `handle` already exists in the organization | + +### List secrets + +```http +GET /api/v0.9/secrets +``` + +Returns metadata for all secrets in the organization. Plaintext values are never included. + +**Query parameters** + +| Parameter | Default | Max | Description | +|-----------|---------|-----|-------------| +| `limit` | 25 | 100 | Maximum number of results to return | +| `offset` | 0 | — | Number of results to skip for pagination | + +**Response — 200 OK** + +```json +{ + "list": [ + { + "uuid": "a1b2c3d4-...", + "handle": "wso2-openai-key", + "name": "WSO2 OpenAI API Key", + "type": "GENERIC", + "provider": "IN_BUILT", + "createdAt": "2026-01-12T10:00:00Z", + "updatedAt": "2026-01-12T10:00:00Z" + } + ], + "pagination": { + "total": 1, + "limit": 25, + "offset": 0 + } +} +``` + +### Get secret metadata + +```http +GET /api/v0.9/secrets/{handle} +``` + +Returns metadata for a single secret. The plaintext value isn't included. + +**Response — 200 OK** + +```json +{ + "uuid": "a1b2c3d4-...", + "handle": "wso2-openai-key", + "name": "WSO2 OpenAI API Key", + "type": "GENERIC", + "provider": "IN_BUILT", + "createdAt": "2026-01-12T10:00:00Z", + "updatedAt": "2026-01-12T10:00:00Z" +} +``` + +**Error responses** + +| Status | Reason | +|--------|--------| +| 404 | No secret found with the given handle in this organization | + +### Rotate a secret + +```http +PUT /api/v0.9/secrets/{handle} +Content-Type: multipart/form-data +``` + +Re-encrypts and stores a new value. Because `handle` is immutable, all {% raw %}`{{ secret "handle" }}`{% endraw %} placeholders across existing resources remain valid without modification. The plaintext value isn't returned in the response. + +**Request fields** + +| Field | Required | Description | +|-------|----------|-------------| +| `value` | Yes | The new sensitive value to encrypt and store. | +| `name` | No | Updated display name. | +| `description` | No | Updated description. | + +**Example request** + +```http +PUT /api/v0.9/secrets/wso2-openai-key +Authorization: Bearer +Content-Type: multipart/form-data; boundary=----FormBoundary + +------FormBoundary +Content-Disposition: form-data; name="value" + +sk-new-value +------FormBoundary +Content-Disposition: form-data; name="name" + +WSO2 OpenAI API Key (rotated) +------FormBoundary +Content-Disposition: form-data; name="description" + +Rotated on 2026-06-26 — old key decommissioned +------FormBoundary-- +``` + +**Response — 200 OK** + +```json +{ + "uuid": "a1b2c3d4-...", + "handle": "wso2-openai-key", + "name": "WSO2 OpenAI API Key (rotated)", + "createdAt": "2026-01-12T10:00:00Z", + "updatedAt": "2026-06-26T11:30:00Z" +} +``` + +**Notes** + +- If the secret's status is `DEPRECATED` (previously soft-deleted), a successful rotation sets it back to `ACTIVE`. New resources can reference the secret again, and the gateway includes it in the next sync. +- The gateway picks up the updated value on its next sync cycle — no redeployment of referencing artifacts is required. + +**Error responses** + +| Status | Reason | +|--------|--------| +| 404 | No secret found with the given handle | + +### Delete a secret + +```http +DELETE /api/v0.9/secrets/{handle} +``` + +Soft-deletes a secret by setting its status to `DEPRECATED`. Deletion is blocked with `409 Conflict` if any artifact references the secret — either in its saved configuration or in a snapshot deployed to a gateway. + +**Response — 204 No Content** + +**Error responses** + +| Status | Reason | +|--------|--------| +| 404 | No secret found with the given handle | +| 409 | Secret is still referenced by one or more artifacts | + +**409 response example** + +```json +{ + "error": "secret is referenced by active resources", + "references": [ + { "type": "llm_provider", "handle": "openai-provider", "name": "OpenAI Provider" }, + { "type": "mcp_proxy", "handle": "my-mcp-proxy", "name": "My MCP Proxy" } + ] +} +``` + +## Reference a secret in an artifact configuration + +Use the following placeholder syntax wherever a configuration field accepts a sensitive string value: + +{% raw %} + +```text +{{ secret "your-secret-handle" }} +``` + +{% endraw %} + +**Example — LLM provider upstream API key** + +{% raw %} + +```yaml +spec: + upstream: + auth: + type: api-key + header: Authorization + value: 'Bearer {{ secret "wso2-openai-key" }}' +``` + +{% endraw %} + +**Validation at save time** + +When creating or updating any resource that contains {% raw %}`{{ secret "..." }}`{% endraw %} references, the Platform API validates that an active secret with the referenced handle exists in the organization. If any placeholder can't be resolved, the request is rejected with `400 Bad Request` and the list of unresolvable handles. + +## Rotate a credential + +To rotate a credential without touching artifact configurations: + +1. Call `PUT /api/v0.9/secrets/{handle}` with the new value. +2. The gateway picks up the updated secret on the next sync cycle. + +No artifact changes or redeployment are required because resources reference the secret by handle, not by value. + +## Delete a secret safely + +Deleting a secret that's still in use returns HTTP 409. To remove it cleanly: + +1. Inspect the `references` list in the 409 response. +2. Update each referencing artifact to remove or replace the {% raw %}`{{ secret "handle" }}`{% endraw %} reference. +3. Redeploy the updated artifacts to the gateway. +4. Retry `DELETE /api/v0.9/secrets/{handle}`. + +## Encryption key + +Secrets are encrypted at rest with the Platform API's at-rest encryption key, which also protects subscription tokens and WebSub hash-based message authentication code (HMAC) secrets. The setup script provisions this key for Docker Compose deployments. For how to generate, mount, and reference it, see [Provision the at-rest encryption key manually](./getting-started.md#provision-the-at-rest-encryption-key-manually). + +!!! warning + Use the same encryption key across restarts and across all replicas. Changing or rotating it makes previously-encrypted secrets unreadable. diff --git a/en/docs/ai-workspace/1.0.0/setting-up/authentication/asgardeo-setup.md b/en/docs/ai-workspace/1.0.0/setting-up/authentication/asgardeo-setup.md new file mode 100644 index 000000000..ad367b7c8 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/setting-up/authentication/asgardeo-setup.md @@ -0,0 +1,192 @@ +--- +title: "Set up Asgardeo as your identity provider" +description: "Configure Asgardeo as the identity provider for a production AI Workspace deployment, from application registration to Platform API configuration." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/asgardeo-setup/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/asgardeo-setup.md +tags: + - cloud + - ai-workspace + - authentication +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-23 +content_type: "how-to" +--- + +# Set up Asgardeo as your identity provider + +This guide walks you through configuring Asgardeo as the identity provider for a production AI Workspace deployment. For background on how identity provider authentication works, see [Authentication in AI Workspace](overview.md). + +## Prerequisites + +- An Asgardeo account at [console.asgardeo.io](https://console.asgardeo.io) +- AI Workspace and Platform API accessible at known hostnames +- The [`register_asgardeo_scopes.sh`](https://github.com/wso2/api-platform/blob/main/portals/ai-workspace/production/scripts/register_asgardeo_scopes.sh) helper script, downloaded from a pinned release tag of the WSO2 API Platform GitHub repository and verified before you run it, as [Step 4](#step-4-register-a-system-application-for-scope-registration) describes + +## Step 1: Set up your organization + +1. Log in to [console.asgardeo.io](https://console.asgardeo.io). +2. Create or select your root organization, for example `default`. +3. If you need multiple tenants, create sub-organizations at `https://console.asgardeo.io/t//app/organizations`. + +## Step 2: Register the AI Workspace application + +AI Workspace runs a backend-for-frontend (BFF) that acts as a confidential OpenID Connect (OIDC) client. The BFF holds the client secret and completes the authorization-code and Proof Key for Code Exchange (PKCE) exchange on the back channel. Register it as a confidential web application, not a single-page application. A single-page application is a public client. The token endpoint rejects the BFF's exchange with this error: "The authenticated client is not authorized to use the requested grant type." + +1. In the root organization, go to **Applications > New Application**. +2. Choose **Standard-Based Application > OpenID Connect** (Traditional Web Application) and name it `AI Workspace`. +3. Add the authorized redirect URL, which is the BFF callback rather than `/signin`: `https:///api/auth/callback`. +4. Enable **Share with all organizations** so users in sub-organizations can log in. +5. Under the **Protocol** tab, set: + - **Allowed grant types**: Authorization Code and Refresh Token + - **PKCE**: enabled + - **Access Token Type**: JWT +6. Under the **Login Flow** tab, configure authentication as needed, for example SSO authentication. +7. Under the **User Attributes** tab, add these attributes to the token: `username`, `given_name`, `family_name`, `roles`, `email`, and `scope`. You create the `scope` attribute in the next step. + +Note the client ID and client secret from the **Protocol** tab. The BFF needs both, and the client ID is also used as the audience in the Platform API configuration. + +## Step 3: Add a custom scope attribute + +1. Create a custom attribute at `https://console.asgardeo.io/t//app/attributes` named `scope`, used to carry OAuth2 scopes granted to the user. +2. Add OIDC scope mappings at `https://console.asgardeo.io/t//app/oidc-scopes` and map the `scope` OIDC claim to the custom `scope` attribute. + +## Step 4: Register a system application for scope registration + +AI Workspace and the Platform API communicate using `ap:*` scopes. Register these scopes in Asgardeo before assigning them to users, using a dedicated system application. + +1. Create a new OIDC application, for example named `AI Platform System`. +2. Under **API Authorization**, add **API Resource Management API** and **Application Management API**. +3. Note the client ID and client secret. +4. Download the scope registration script from a pinned release tag rather than the `main` branch, so the content can't change between the checksum you verify and the code you run. Replace `` with the tag you're deploying: + + ```bash + RELEASE_TAG= + curl -fsSLO "https://raw.githubusercontent.com/wso2/api-platform/${RELEASE_TAG}/portals/ai-workspace/production/scripts/register_asgardeo_scopes.sh" + ``` + + `--fail` makes `curl` exit non-zero on an HTTP error, so a 404 from a mistyped tag doesn't leave an error page saved as the script. + +5. Verify the download against the checksum published with that release, and read the script before you run it: + + ```bash + shasum -a 256 register_asgardeo_scopes.sh + ``` + +6. Run the script only after the checksum matches: + + ```bash + chmod +x register_asgardeo_scopes.sh + + ASGARDEO_TENANT= \ + ASGARDEO_CLIENT_ID= \ + ASGARDEO_CLIENT_SECRET= \ + ASGARDEO_RESOURCE_IDENTIFIER=https:// \ + ./register_asgardeo_scopes.sh + ``` + +This registers an API resource in Asgardeo that represents the Platform API, with all `ap:*` scopes registered under it. For local testing, the default `ASGARDEO_RESOURCE_IDENTIFIER=https://localhost:9243` works without changes. + +## Step 5: Link scopes to the AI Workspace application + +1. Open the AI Workspace application you registered in Step 2. +2. Under **API Authorization**, add the API resource you created in Step 4. +3. Create an application role, for example `ap_admin`. +4. Assign all `ap:*` scopes to that role. + +## Step 6: Add sub-organization users + +For each sub-organization that needs access: + +1. Register users under the sub-organization. +2. Assign the shared `ap_admin` role to each user. + +## Step 7: Configure the Platform API + +AI Workspace and the Platform API share a single `configs/config.toml` file. The Platform API reads its own `[platform_api.*]` tables from it and ignores the `[ai_workspace.*]` tables, and AI Workspace does the reverse. Update the `[platform_api.auth]` section: + +```toml +[platform_api.auth] +mode = "idp" + +[platform_api.auth.idp] +name = "asgardeo" +jwks_url = "https://api.asgardeo.io/t//oauth2/jwks" +issuer = ["https://api.asgardeo.io/t//oauth2/token"] +audience = [""] + +[platform_api.auth.claim_mappings] +organization = "org_id" +org_name = "org_name" +org_handle = "org_handle" +``` + +`mode` selects exactly one Platform API auth mode, so setting it to `"idp"` stops the file-based login endpoint from being used. Asgardeo uses `org_id` as the claim for the organization UUID, while the Platform API defaults to `organization`. The claim name override above is required to bridge the two. + +## Step 8: Configure AI Workspace + +In the same `configs/config.toml`, update the `[ai_workspace.auth]` tables: + +{% raw %} + +```toml +[ai_workspace] +domain = "" +default_org_region = "us" + +[ai_workspace.control_plane] +url = "https://" + +[ai_workspace.gateway] +controlplane_host = "" + +[ai_workspace.auth] +mode = "oidc" + +[ai_workspace.auth.oidc] +authority = "https://api.asgardeo.io/t//oauth2/token" +client_id = "" +client_secret = '{{ env "APIP_AIW_AUTH_OIDC_CLIENT_SECRET" }}' +redirect_url = "https:///api/auth/callback" +post_logout_redirect_url = "https:///login" + +# A sibling of [ai_workspace.auth.oidc], not nested in it — applies to both auth modes. +[ai_workspace.auth.claim_mappings] +organization = "org_id" +org_name = "org_name" +org_handle = "org_handle" +``` + +{% endraw %} + +`redirect_url` must exactly match the authorized redirect URL you registered in Step 2. + +Never write the client secret as a literal in `config.toml`. The `{% raw %}{{ env }}{% endraw %}` placeholder above reads it from an environment variable instead, so it never has to be committed to source control: + +```bash +read -rs -p "AI Workspace client secret: " APIP_AIW_AUTH_OIDC_CLIENT_SECRET +export APIP_AIW_AUTH_OIDC_CLIENT_SECRET +``` + +Reading the value from a prompt keeps the secret out of your shell history and out of the process list. + +In production, supply the secret from a mounted secret file. Swap the token in `config.toml` for {% raw %}`'{{ file "/secrets/ai-workspace/oidc_client_secret" }}'`{% endraw %}, then mount the secret at that path on the AI Workspace service only. That path is one of the BFF's allowed file sources, and the Platform API can't read it. Resolution fails closed, so a missing or unreadable file aborts startup rather than falling back to an empty credential. + +Don't put the client secret in `api-platform.env`. Compose loads that file into every service, so the Platform API and the API Portal would receive a credential only the BFF needs. For local testing, set the variable in the shell you start the stack from, or in an environment file mounted on the `ai-workspace` service alone. + +Once configured, opening AI Workspace redirects you to the Asgardeo-hosted login page instead of the file-based login form: + +![AI Workspace login page redirecting to the Asgardeo-hosted login page](../../../../assets/img/ai-gateway/standalone-ai-workspace/authentication/oidc-login-redirect.png) + +## Claim flow summary + +The Asgardeo token carries these claims through to the Platform API: + +| Claim | Purpose | Configured as | +|-------|---------|----------------| +| `sub` | User identity | N/A | +| `org_id` | Organization UUID | `organization` in both `[ai_workspace.auth.claim_mappings]` (AI Workspace) and `[platform_api.auth.claim_mappings]` (Platform API) | +| `org_name` | Organization display name | `org_name` in both | +| `org_handle` | Organization slug | `org_handle` in both | +| `scope` | Space-separated `ap:*` scopes | Validated by the Platform API | + +Keep these claim names consistent across three places: the Asgardeo token mapper output, and the `[ai_workspace.auth.claim_mappings]` and `[platform_api.auth.claim_mappings]` tables. Only the latter two are defined in `config.toml` — the token mapper output is configured in Asgardeo. diff --git a/en/docs/ai-workspace/1.0.0/setting-up/authentication/connect-an-identity-provider.md b/en/docs/ai-workspace/1.0.0/setting-up/authentication/connect-an-identity-provider.md new file mode 100644 index 000000000..684f16f0a --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/setting-up/authentication/connect-an-identity-provider.md @@ -0,0 +1,222 @@ +--- +title: "Connect an identity provider to AI Workspace" +description: "Configure AI Workspace and the Platform API to delegate login to an OIDC identity provider: client registration, claim mappings, and authorization." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/connect-an-identity-provider/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/connect-an-identity-provider.md +tags: + - cloud + - ai-workspace + - authentication + - oidc +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-31 +content_type: "how-to" +--- + +# Connect an identity provider to AI Workspace + +AI Workspace delegates user login to any identity provider (IdP) that speaks OpenID Connect (OIDC). This guide is written for the administrator who deploys AI Workspace, and it covers the configuration every IdP needs. For an example of these steps applied to one specific IdP, see [Set up Asgardeo as your identity provider](asgardeo-setup.md). + +## How OIDC login works here + +AI Workspace is served by a Backend-for-Frontend (BFF), and the BFF is the OIDC client — not the browser. It runs the whole authorization code exchange server-side, keeps the resulting tokens in a server-side session, and gives the browser only an `HttpOnly` cookie. Register AI Workspace as a **confidential** client, not a public or single-page application client. + +Two services need configuring, and they must agree: + +- **AI Workspace** sends users to the IdP and reads identity claims out of the returned token. +- **Platform API** verifies every token against the IdP's JWKS endpoint and authorizes each request from the token's privileges. + +Both read the same `configs/config.toml` — AI Workspace from the `[ai_workspace.*]` tables, the Platform API from the `[platform_api.*]` tables. + +## What your identity provider must support + +Check your IdP against these requirements before you start: + +| Requirement | Details | +|-------------|---------| +| OIDC discovery | Serves `/.well-known/openid-configuration` at its authority URL. AI Workspace discovers every endpoint from there, so you configure the authority and nothing else. | +| JWT access tokens | Access tokens are signed JWTs. The Platform API reads claims out of the token directly, so opaque tokens don't work. | +| JWKS endpoint | Publishes the signing keys the Platform API verifies token signatures against. | +| Confidential client | Issues a client secret. | +| Authorization code and refresh token grants | Both enabled on the application. | +| Custom claims | Emits organization identity in the token. Claim names are configurable, the claims themselves are required. | + +## Step 1: Register AI Workspace as a confidential client + +In your IdP, create a confidential OIDC application with these settings. Replace `` with the address users reach AI Workspace at, including the port when it isn't 443—`localhost:9643` for the Docker Compose quickstart. + +1. Set the authorized redirect URL to `https:///api/auth/callback`. This is the BFF's own server-side callback route, not a page in the app. +2. Set the post-logout redirect URL to `https:///login`. +3. Enable the **Authorization Code** and **Refresh Token** grants. +4. Record the client ID and client secret. + +## Step 2: Emit the claims AI Workspace expects + +Both services read user and organization identity out of the access token by claim name. The defaults are listed here; you can either configure your IdP to emit these names, or keep your IdP's names and map them in Step 4. + +| Default claim | Carries | +|---------------|---------| +| `sub` | The user's unique ID | +| `username` | The username shown in the workspace | +| `email` | The user's email address | +| `organization` | Organization ID | +| `org_name` | Organization display name | +| `org_handle` | Organization URL slug | +| `scope` | Space-separated scope string | +| `roles` | The user's roles | + +The organization claims matter beyond display: AI Workspace creates an organization on a user's first login from them, and assigns it the region set in `[ai_workspace] default_org_region`. + +## Step 3: Choose how privileges reach the token + +The Platform API authorizes each request in one of two modes, set in `[platform_api.auth.authorization]`. Pick the one that matches what your IdP can put in a token. + +**Scope mode** (`mode = "scope"`, the default) reads the `scope` claim and checks it against the scope each endpoint requires. Your IdP has to issue tokens carrying the platform's `ap:*` scopes, which usually means registering those scopes in the IdP and granting them to the application. + +**Role mode** (`mode = "role"`) reads the `roles` claim and expands each role into scopes through a YAML mapping file. Your IdP only has to emit role names, which most products do out of the box. Set `role_to_scope_mapping` to the path of that file—the packs mount an editable copy at `/etc/platform-api/role-to-scope-mapping.yaml`. It ships these roles: + +| Role | Grants | +|------|--------| +| `ap_admin` | Full access to every resource and operation | +| `ap_operator` | Gateway and deployment operations | +| `ap_publisher` | Creating and publishing APIs and proxies | +| `ap_subscriber` | Applications and subscriptions | +| `ap_viewer` | Read-only access | + +Edit that file to change what a role grants, or to add your own. It's mounted configuration rather than part of the image, and the Platform API reads it at startup, so a change needs a restart. + +Role mode is the lighter integration of the two. Map your IdP's groups onto these role names, and you never register a scope in the IdP at all. + +## Step 4: Configure the Platform API + +Set the Platform API to verify tokens against your IdP. `mode` selects exactly one authentication mode, so setting it to `"idp"` also stops the file-based login endpoint from being used. + +```toml +[platform_api.auth] +mode = "idp" + +[platform_api.auth.idp] +name = "my-idp" # friendly name, used in logs +jwks_url = "https://idp.example.com/oauth2/jwks" +issuer = ["https://idp.example.com"] # accepted "iss" values +audience = [""] # accepted "aud" values; empty skips the check + +[platform_api.auth.authorization] +enabled = true +mode = "role" # or "scope" +role_to_scope_mapping = "/etc/platform-api/role-to-scope-mapping.yaml" +``` + +Set `issuer` to the exact value your IdP puts in the token's `iss` claim. Read it from the `issuer` field of the IdP's `/.well-known/openid-configuration` document rather than assuming it. `issuer` takes a list, so a provider that mints tokens under more than one issuer URL is accommodated by naming each. Set `audience` to your client ID rather than leaving it empty, so a token minted for a different application is rejected. + +If your IdP's claim names differ from the defaults in Step 2, override them: + +```toml +[platform_api.auth.claim_mappings] +organization = "org_id" +org_name = "org_name" +org_handle = "org_handle" +user_id = "sub" +username = "username" +email = "email" +scope = "scope" +roles = "roles" +``` + +Each value is either a flat top-level claim name or a dot-separated path into a nested claim. That path syntax is what accommodates providers that nest their claims — Keycloak puts roles under `realm_access.roles`, for example. + +## Step 5: Configure AI Workspace + +In the same file, point AI Workspace at the IdP: + +{% raw %} + +```toml +[ai_workspace] +domain = "" +default_org_region = "us" + +[ai_workspace.auth] +mode = "oidc" + +[ai_workspace.auth.oidc] +authority = "https://idp.example.com" +client_id = "" +client_secret = '{{ env "APIP_AIW_AUTH_OIDC_CLIENT_SECRET" }}' +redirect_url = "https:///api/auth/callback" +post_logout_redirect_url = "https:///login" + +# Must match [platform_api.auth.authorization] — see below. +[ai_workspace.auth.authorization] +mode = "role" # or "scope" +role_to_scope_mapping = "/etc/ai-workspace/role-to-scope-mapping.yaml" + +# A sibling of [ai_workspace.auth.oidc], not nested in it — applies to both auth modes. +[ai_workspace.auth.claim_mappings] +organization = "org_id" # claim carrying the org ID +org_name = "org_name" # org display name +org_handle = "org_handle" # org URL slug +username = "username" +email = "email" +scope = "scope" # space-separated scope string +roles = "roles" +``` + +{% endraw %} + +Four things to get right: + +- **`[ai_workspace.auth.authorization]`** needs the same `mode` as `[platform_api.auth.authorization]`. In role mode, mount the same mapping file into the `ai-workspace` container. The UI gates every action on the scopes `/api/session` reports, and AI Workspace derives those scopes from the token. In role mode it expands the `roles` claim through the mapping file. Omit this table, and the UI blocks operations the Platform API would authorize. +- **`authority`** is the issuer URL. Endpoints are discovered from it, so it must be the URL whose `/.well-known/openid-configuration` describes your IdP. +- **`redirect_url`** must match the URL registered in Step 1 exactly, character for character. +- **`[ai_workspace.auth.claim_mappings]`** must give every key it shares with `[platform_api.auth.claim_mappings]` the same value. Both services read the same token, so a mismatch means one of them reads the wrong claim. AI Workspace uses `username` and `email` to render the signed-in user, so they matter here as much as the organization claims. The table has no `user_id` key — only the Platform API maps that claim. + +Every `[ai_workspace.auth.oidc]` key except `scope` defaults to empty. In OIDC mode the server refuses to start until each one is set, so a misconfiguration fails at startup rather than at a user's first login. + +Leave `scope` unset to request the full `ap:*` scope set the Platform API authorizes against, which is the recommended starting point. If you trim it, keep `offline_access`, or token refresh stops working. + +### Supply the client secret + +Never write the client secret as a literal in `config.toml`. The token above reads it from an environment variable instead: + +```bash +APIP_AIW_AUTH_OIDC_CLIENT_SECRET= +``` + +In the Docker Compose distribution, set it in the git-ignored `api-platform.env`, which is the file Compose loads into both containers. In production, prefer a mounted secret file. Swap the token for {% raw %}`'{{ file "/secrets/ai-workspace/oidc_client_secret" }}'`{% endraw %}, then mount the secret at that path. Both forms fail closed: a missing variable or unreadable file aborts startup rather than falling back to an empty credential. See [Sensitive values in `config.toml`](../configuration.md#sensitive-values-in-configtoml). + +## Step 6: Restart and verify + +Restart both services so they reload the configuration: + +```bash +docker compose up --force-recreate +``` + +Open AI Workspace. Instead of the username and password form, you're redirected to your IdP's hosted login page, and land back in the workspace after signing in. + +If sign-in fails, the mismatch is usually in one of these: + +- The `redirect_url` in `config.toml` differs from the one registered in the IdP. +- The IdP issues opaque access tokens rather than JWTs. +- `issuer` doesn't match the token's `iss` claim, or `audience` doesn't match its `aud` claim. +- The token carries organization claims under names the `claim_mappings` tables don't name. + +Set `[ai_workspace.logging] level = "debug"` and `[platform_api.logging] level = "debug"` to see which claim or check fails. + +## Claim names for common identity providers + +The roles claim is the one that most often differs. These paths are known to work: + +| Identity provider | `claim_mappings` roles value | +|-------------------|------------------------------| +| Asgardeo | `roles` | +| Microsoft Entra ID | `roles` | +| Keycloak | `realm_access.roles`, or `resource_access..roles` | + +## Next steps + +- [Set up Asgardeo as your identity provider](asgardeo-setup.md): an example of these steps applied to Asgardeo, including scope registration +- [Set up Microsoft Entra ID as your identity provider](entra-id-setup.md): these steps applied to Entra ID, where authorization runs on app roles rather than scope-based authorization, and an API access scope such as `api:///access` is still required to acquire a token +- [Authentication in AI Workspace](overview.md): how identity provider authentication compares with file-based authentication +- [AI Workspace configuration](../configuration.md): how interpolation tokens deliver values into `config.toml` diff --git a/en/docs/ai-workspace/1.0.0/setting-up/authentication/entra-id-setup.md b/en/docs/ai-workspace/1.0.0/setting-up/authentication/entra-id-setup.md new file mode 100644 index 000000000..8a35e9234 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/setting-up/authentication/entra-id-setup.md @@ -0,0 +1,329 @@ +--- +title: "Set up Microsoft Entra ID as your identity provider" +description: "Configure Microsoft Entra ID for a production AI Workspace deployment: application registration, app roles, and the config.toml settings both services read." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/entra-id-setup/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/entra-id-setup.md +tags: + - cloud + - ai-workspace + - authentication + - oidc + - entra-id +author: WSO2 API Platform Documentation Team +last_updated: 2026-08-03 +content_type: "how-to" +--- + +# Set up Microsoft Entra ID as your identity provider + +This guide walks you through registering a Microsoft Entra ID application and configuring AI Workspace and the Platform API to authenticate against it. + +For background on identity provider (IdP) authentication, see [Authentication in AI Workspace](overview.md). For the configuration common to all identity providers, see [Connect an identity provider to AI Workspace](connect-an-identity-provider.md). + +## Prerequisites + +Before you begin, make sure you have: + +- A Microsoft Entra ID tenant. +- Permission to register applications and grant admin consent. +- AI Workspace and the Platform API accessible over HTTPS. +- Access to the `configs/config.toml` file both services read. + +This guide uses the following placeholders: + +| Placeholder | Description | +|-------------|-------------| +| `` | The **Directory (tenant) ID** from the application's **Overview** page | +| `` | The **Application (client) ID** from the application's **Overview** page | +| `` | The AI Workspace host as the browser reaches it, including the port when it isn't `443`—for example, `localhost:9643` | + +## Configure Microsoft Entra ID + +### Step 1: Register the application + +Register the application with the following settings: + +| Setting | Value | +|---------|-------| +| **Name** | `AI Workspace` | +| **Supported account types** | Accounts in this organizational directory only | +| **Redirect URI platform** | Web | +| **Redirect URI** | `https:///api/auth/callback` | +| **Additional redirect URI** | `https:///login` | + +Choose **Web** as the platform, not **Single-page application**, and register both redirect URIs on that platform. The first receives the sign-in callback. The second is where Entra ID returns the browser after sign-out. Entra ID validates the `post_logout_redirect_uri` against the registered redirect URIs, so the `/login` destination you set as `post_logout_redirect_url` in [Step 10](#step-10-configure-oidc-authentication) has to appear here. + +To register the application, follow these steps: + +1. In the Azure portal, go to **Microsoft Entra ID > App registrations > New registration**. +2. Enter the settings from the preceding table. +3. Select **Register**. +4. Open the application's **Overview** page and record the **Application (client) ID** and the **Directory (tenant) ID**. + +### Step 2: Expose an API + +Go to **App registrations > AI Workspace > Expose an API**. + +#### 2.1 Configure the application ID URI + +1. Next to **Application ID URI**, select **Add**. +2. Keep the default value `api://`. +3. Select **Save**. + +#### 2.2 Add an API scope + +Add a scope with the following settings: + +| Setting | Value | +|---------|-------| +| **Scope name** | `access` | +| **Who can consent** | Admins and users | +| **Admin consent display name** | Access AI Workspace | +| **State** | Enabled | + +To add the scope, follow these steps: + +1. Select **Add a scope**. +2. Enter the settings from the preceding table. +3. Select **Add scope**. + +#### 2.3 Add API permissions + +1. Go to **API permissions > Add a permission > My APIs**. +2. Choose the **AI Workspace** application. +3. Under **Delegated permissions**, select the `access` scope. +4. Confirm with **Add permissions**. + +#### 2.4 Grant admin consent + +On the **API permissions** page, select **Grant admin consent** and confirm the permission shows as **Granted**. + +### Step 3: Create a client secret + +1. Go to **Certificates & secrets > New client secret**. +2. Enter a description and select an expiration period. +3. Select **Add**. +4. Copy the **Value** of the client secret. + + !!! warning + Copy the **Value**, not the **Secret ID**. Entra ID shows the secret value only when the secret is created. + +Store the secret securely. You configure it in AI Workspace in [Step 10](#step-10-configure-oidc-authentication). + +### Step 4: Configure version 2.0 access tokens + +1. Go to **App registrations > AI Workspace > Manifest**. +2. Find the `api` section and set `requestedAccessTokenVersion` to `2`: + + ```json + "api": { + "requestedAccessTokenVersion": 2 + } + ``` + +3. Save the manifest. + +### Step 5: Create application roles + +Each app role takes the following settings, shown here for `ap_admin`: + +| Setting | Value | +|---------|-------| +| **Display name** | `ap_admin` | +| **Allowed member types** | Users/Groups | +| **Value** | `ap_admin` | +| **Do you want to enable this app role?** | Enabled | + +**Value** must match the corresponding role name in `role-to-scope-mapping.yaml`. + +To create a role, follow these steps: + +1. Go to **App registrations > AI Workspace > App roles**. +2. Select **Create app role**. +3. Enter the settings from the preceding table. +4. Repeat for each role you need. + +The default roles are: + +| Role | Grants | +|------|--------| +| `ap_admin` | Full access to every resource and operation | +| `ap_operator` | Gateway and deployment operations | +| `ap_publisher` | Creating and publishing APIs and proxies | +| `ap_subscriber` | Applications and subscriptions | +| `ap_viewer` | Read-only access | + +### Step 6: Assign roles to users or groups + +1. Go to **Microsoft Entra ID > Enterprise applications** and select the **AI Workspace** application. +2. Open **Users and groups > Add user/group**. +3. Choose the user or group, then choose the application role to assign. +4. Select **Assign**. + +### Step 7: Add optional claims + +1. Go to **App registrations > AI Workspace > Token configuration**. +2. Select **Add optional claim**. +3. Select **Access** as the token type. +4. Add the `email` claim. Entra ID includes `preferred_username` in a v2.0 access token from the `profile` scope, which the application requests already. The `email` claim depends on the `email` scope and is absent for a user who has no email address recorded. + +Don't add `tid` or `oid` here. Entra ID includes both in a v2.0 access token without any optional-claim configuration. + +### Step 8: Get the OpenID Connect (OIDC) endpoints + +Go to **App registrations > AI Workspace > Overview > Endpoints** and use the version 2.0 endpoints: + +| Endpoint | URL | +|----------|-----| +| OIDC metadata | `https://login.microsoftonline.com//v2.0/.well-known/openid-configuration` | +| Issuer | `https://login.microsoftonline.com//v2.0` | +| JSON Web Key Set (JWKS) | `https://login.microsoftonline.com//discovery/v2.0/keys` | + +These values go into the Platform API configuration in the next step. + +## Configure the Platform API + +### Step 9: Configure Platform API authentication + +AI Workspace and the Platform API read the same `configs/config.toml` file. Update the `[platform_api.auth]` tables: + +```toml +# Delegate authentication to the external identity provider. +[platform_api.auth] +mode = "idp" + +# JWKS-based validation against Microsoft Entra ID. +[platform_api.auth.idp] +name = "entra" +jwks_url = "https://login.microsoftonline.com//discovery/v2.0/keys" +issuer = ["https://login.microsoftonline.com//v2.0"] +audience = [""] + +# Use application roles for authorization. +[platform_api.auth.authorization] +enabled = true +mode = "role" +role_to_scope_mapping = "/etc/platform-api/role-to-scope-mapping.yaml" + +# Microsoft Entra ID claim mappings. +[platform_api.auth.claim_mappings] +organization = "tid" +org_handle = "tid" +org_name = "tid" +user_id = "sub" +username = "preferred_username" +email = "email" +roles = "roles" +``` + +All three organization keys map to `tid`, the directory (tenant) ID. `tid` is the only tenant-level identifier a version 2.0 access token carries by default, so every user in the tenant resolves to the same organization. For a readable organization name or slug, add a custom claim that carries the same value for every user in the tenant, then map `org_name` and `org_handle` to it. + +## Configure AI Workspace + +### Step 10: Configure OIDC authentication + +In the same `configs/config.toml` file, update the `[ai_workspace.auth]` tables: + +{% raw %} + +```toml +[ai_workspace.auth] +mode = "oidc" + +[ai_workspace.auth.oidc] +authority = "https://login.microsoftonline.com//v2.0" +client_id = "" +client_secret = '{{ file "/secrets/ai-workspace/oidc_client_secret" }}' +redirect_url = "https:///api/auth/callback" +post_logout_redirect_url = "https:///login" + +# Microsoft Entra ID scope configuration. +scope = "openid profile email offline_access api:///access" + +# Use application roles for authorization. +[ai_workspace.auth.authorization] +mode = "role" +role_to_scope_mapping = "/etc/ai-workspace/role-to-scope-mapping.yaml" + +# Microsoft Entra ID claim mappings. +[ai_workspace.auth.claim_mappings] +organization = "tid" +org_name = "tid" +org_handle = "tid" +username = "preferred_username" +email = "email" +roles = "roles" +``` + +{% endraw %} + +#### Supply the client secret + +Never write the client secret as a literal in `config.toml`. For production deployments, read it from a mounted secret file: + +{% raw %} + +```toml +client_secret = '{{ file "/secrets/ai-workspace/oidc_client_secret" }}' +``` + +{% endraw %} + +For local development, read it from an environment variable instead: + +{% raw %} + +```toml +client_secret = '{{ env "APIP_AIW_AUTH_OIDC_CLIENT_SECRET" }}' +``` + +{% endraw %} + +For more information, see [Sensitive values in `config.toml`](../configuration.md#sensitive-values-in-configtoml). + +## Restart and verify + +### Step 11: Restart the services + +Restart AI Workspace and the Platform API so they load the updated configuration. For Docker Compose: + +```bash +docker compose up --force-recreate +``` + +### Step 12: Sign in to AI Workspace + +1. Open AI Workspace in your browser. You're redirected to the Microsoft sign-in page. +2. Sign in as a user who has an application role assigned. After authentication, you land back in AI Workspace. + +### Step 13: Verify the access token + +!!! warning "Never paste a production token into a web decoder" + An access token is a bearer credential. + +Decode the access token and check its claims. A correctly configured token contains values similar to these: + +```json +{ + "aud": "", + "iss": "https://login.microsoftonline.com//v2.0", + "ver": "2.0", + "tid": "", + "preferred_username": "user1@example.onmicrosoft.com", + "roles": [ + "ap_admin" + ], + "scp": "access" +} +``` + +Check the following fields: + +| Field | Expected value | +|-------|----------------| +| `aud` | `` | +| `iss` | `https://login.microsoftonline.com//v2.0` | +| `ver` | `2.0` | +| `tid` | `` | +| `roles` | The assigned application role | +| `scp` | `access` | diff --git a/en/docs/ai-workspace/1.0.0/setting-up/authentication/overview.md b/en/docs/ai-workspace/1.0.0/setting-up/authentication/overview.md new file mode 100644 index 000000000..f9a1aa238 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/setting-up/authentication/overview.md @@ -0,0 +1,113 @@ +--- +title: "Authentication in AI Workspace" +description: "Understand the two ways users sign in to AI Workspace: file-based authentication for local use and an identity provider for production." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/overview.md +tags: + - cloud + - ai-workspace + - authentication +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-23 +content_type: "concept" +--- + +# Authentication in AI Workspace + +AI Workspace and the Platform API read their settings from a single `config.toml` file. AI Workspace's `[ai_workspace.*]` tables and the Platform API's `[platform_api.*]` tables live side by side in it. Authentication is set independently in each service's table, though both must agree for a given mode to work end to end. A running instance uses one mode at a time. + +| Mode | `[ai_workspace.auth] mode` | `[platform_api.auth] mode` | Best for | +|------|------------------------------|-------------------------------|----------| +| File-based | `basic` | `file` | Local use and demos, no identity provider required | +| Identity provider | `oidc` | `idp` | Production, where a dedicated identity provider manages user login | + +## File-based authentication + +File-based authentication stores a list of users directly in the Platform API's configuration table. It requires no external identity provider, which makes it the default when you [get started with AI Workspace](../../getting-started.md) using Docker Compose. + +When `[ai_workspace.auth] mode = "basic"`, the AI Workspace login page renders a username and password form. The Platform API validates the credentials against a hashed user list defined in `[platform_api.auth.file.users]`: + +![AI Workspace file-based login window with Username and Password fields](../../../../assets/img/ai-gateway/standalone-ai-workspace/authentication/filebased-login.png) + +```toml +[platform_api.auth] +mode = "file" + +[[platform_api.auth.file.users]] +username = "admin" +password_hash = "$2a$12$..." # bcrypt hash of the password +roles = ["ap_admin"] +``` + +Generate a bcrypt hash for the password with any standard tool. This example prompts for the password, so it never reaches your shell history or the process list: + +```bash +htpasswd -nBC 12 "" | tr -d ':\n' +``` + +### Add more users + +`[[platform_api.auth.file.users]]` is an array of tables, so repeating the whole block—double brackets and all—defines another user. Give each one the roles that match what that person does: + +```toml +[[platform_api.auth.file.users]] +username = "admin" +password_hash = "$2a$12$..." +roles = ["ap_admin"] + +[[platform_api.auth.file.users]] +username = "developer" +password_hash = "$2a$12$..." +roles = ["ap_publisher", "ap_subscriber"] + +[[platform_api.auth.file.users]] +username = "auditor" +password_hash = "$2a$12$..." +roles = ["ap_viewer"] +``` + +A user's `roles` define their complete grant. There's no per-user scope list. + +- Each role maps to a set of scopes through the role-to-scope mapping file at `[platform_api.auth.authorization] role_to_scope_mapping`. The distributions mount it at `/etc/platform-api/role-to-scope-mapping.yaml`. +- The login endpoint expands those roles into the token's scope claim. +- Assigning several roles combines their grants, as `developer` does above. +- A user with no roles fails startup, and so does a user naming a role the mapping file doesn't define. The failure comes at startup rather than after signing in and being denied every request. + +The mapping file ships with these roles: + +| Role | Grants | +|------|--------| +| `ap_admin` | Full access to every resource and operation | +| `ap_operator` | Gateway and deployment operations | +| `ap_publisher` | Creating and publishing APIs and proxies | +| `ap_subscriber` | Applications and subscriptions | +| `ap_viewer` | Read-only access | + +Edit that file to change what a role grants, or to add your own. + +The `setup.sh` script bundled with the Docker Compose distribution provisions the Platform API's admin credentials. It prompts for the username and password, generates the password if you accept the default, and prints that password to the terminal once. See [Getting Started](../../getting-started.md). + +File-based authentication has two limitations: + +- It supports a single organization only. Multiple organizations require an identity provider. +- The user list is static. Changes require restarting the Platform API container. + +Because of these limitations, file-based authentication isn't suitable for production or shared environments. + +## Identity provider authentication + +For production, configure AI Workspace to delegate login to an identity provider (IdP) over OpenID Connect (OIDC). AI Workspace works with any OIDC-compliant IdP that meets these requirements: + +| Requirement | Details | +|-------------|---------| +| OIDC discovery | The IdP exposes `/.well-known/openid-configuration` at its authority URL | +| JSON Web Token (JWT) access tokens | Access tokens are JWTs, not opaque tokens | +| JSON Web Key Set (JWKS) endpoint | The IdP exposes a JWKS endpoint so the Platform API can verify token signatures | +| Custom claims | Tokens carry organization identity as custom claims (claim names are configurable) | +| Confidential client | AI Workspace is registered as a confidential client with a client secret, not a public or single-page application client | + +[Connect an identity provider to AI Workspace](connect-an-identity-provider.md) covers the configuration for any such IdP. It walks through client registration, claim mappings, and the choice between scope and role authorization. Two guides apply those steps to a specific IdP: [Set up Asgardeo as your identity provider](asgardeo-setup.md) and [Set up Microsoft Entra ID as your identity provider](entra-id-setup.md). + +## Choosing a mode + +Use file-based authentication when you're trying out AI Workspace, running a demo, or don't yet have an identity provider available. Move to an identity provider before you deploy to a shared or production environment. Switch to one as well when you need multiple organizations, or want single sign-on with an existing identity system. diff --git a/en/docs/ai-workspace/1.0.0/setting-up/configuration.md b/en/docs/ai-workspace/1.0.0/setting-up/configuration.md new file mode 100644 index 000000000..6bcb03e96 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/setting-up/configuration.md @@ -0,0 +1,105 @@ +--- +title: "AI Workspace configuration and environment interpolation" +description: "How AI Workspace and the Platform API load config.toml, inject environment values and mounted files through interpolation tokens, and keep secrets out of it." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/configuration/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/configuration.md +tags: + - ai-workspace + - configuration + - interpolation +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-24 +content_type: "reference" +--- + +# AI Workspace configuration and environment interpolation + +The AI Workspace stack has two services: the AI Workspace Backend-for-Frontend (BFF) and the Platform API it proxies to. Each reads its configuration from a TOML file (`config.toml`) layered over built-in defaults. + +This page explains how each service loads its config file. It also covers how environment values and mounted files are injected through interpolation tokens, and how to keep sensitive values out of the file. For provisioning the keys, certificates, and credentials those tokens resolve to, see [Get started with AI Workspace](../getting-started.md). + +## How configuration is loaded + +Each service reads a TOML file mounted into its container, layered over that service's built-in defaults: + +- **AI Workspace (BFF)** — `/etc/ai-workspace/config.toml`; every key lives under the `[ai_workspace]` table. +- **Platform API** — `/etc/platform-api/config.toml`; every key lives under the `[platform_api]` table. + +The per-service namespacing (`[ai_workspace]`, `[platform_api]`, `[api_portal]`) lets one `config.toml` hold multiple services' sections side by side without their keys colliding—each service reads only its own table. + +!!! important "Environment variables don't override config keys directly" + There is **no prefix that auto-maps environment variables onto config keys.** An environment value reaches a setting **only** through an explicit interpolation token written into the config file, resolved when the file is loaded. A key written as a plain literal — or absent from the file — ignores the matching variable entirely. + +## Interpolation tokens + +Two functions are available inside `config.toml`: + +{% raw %} + +| Token | Behavior | +|-------|----------| +| `{{ env "NAME" "default" }}` | Substitutes the value of environment variable `NAME`. If the variable is unset **or set-but-empty**, the `default` is used. If no default is given, an unset variable fails startup. | +| `{{ file "PATH" }}` | Reads a secret value from a mounted file at `PATH` — for injecting secrets from a mounted volume rather than an environment variable. A trailing newline is trimmed. | + +An example from the shipped AI Workspace `config.toml`: + +```toml +[ai_workspace.control_plane] +url = '{{ env "APIP_AIW_CONTROL_PLANE_URL" "https://platform-api:9243" }}' + +[ai_workspace.auth] +mode = '{{ env "APIP_AIW_AUTH_MODE" "basic" }}' +``` + +{% endraw %} + +Most tokens in the shipped config carry a default, so an unset variable keeps the built-in value. A token written without a default names a required secret, and startup fails when that variable isn't set. + +!!! note "The variable name is a naming convention, not a prefix override" + By convention each token names the key's dotted path, uppercased with dots as underscores, behind a per-service prefix. The prefixes are `APIP_AIW_` for AI Workspace, `APIP_CP_` for the Platform API, and `APIP_AP_` for the API Portal. For example, `[ai_workspace.control_plane] url` becomes `APIP_AIW_CONTROL_PLANE_URL`. The loader doesn't interpret the prefix — the name is only the literal string you pass to the interpolation function. You can rename any variable, as long as you edit the matching token in `config.toml` to agree. + +### Which variables your deployment reads + +Because the variable names live in `config.toml`, the config file is the authoritative list for your deployment — not this page. To see which settings your stack injects from the environment, search the mounted `config.toml` for {% raw %}`{{ env`{% endraw %} and read the name out of each token. + +For every configurable option and the tokens the shipped files carry, refer to the config templates: [AI Workspace](https://github.com/wso2/api-platform/blob/main/portals/ai-workspace/configs/config-template.toml) and [Platform API](https://github.com/wso2/api-platform/blob/main/platform-api/config/config-template.toml). + +## Sensitive values in `config.toml` + +This section covers credentials the services need to start — database passwords, the OpenID Connect (OIDC) client secret, and the at-rest encryption key. It's a separate mechanism from the [AI Workspace secrets](../secrets-management.md) feature, which stores encrypted credentials you reference from artifacts. + +Never write a sensitive value as a literal in `config.toml`, and never hardcode one in `docker-compose.yaml`. Reference each with an interpolation token—from an environment variable or, preferably, from a mounted file: + +{% raw %} + +```toml +# Platform API at-rest encryption key - the shipped default reads a mounted file: +encryption_key = '{{ file "/etc/platform-api/keys/encryption.key" }}' +# or, alternatively, from an environment variable: +# encryption_key = '{{ env "APIP_CP_ENCRYPTION_KEY" }}' + +# AI Workspace OIDC client secret (oidc mode) - env var, or a mounted file: +client_secret = '{{ env "APIP_AIW_AUTH_OIDC_CLIENT_SECRET" }}' +# client_secret = '{{ file "/secrets/ai-workspace/oidc_client_secret" }}' +``` + +{% endraw %} + +Neither `encryption_key` nor `client_secret` carries a default, so each is a required secret. Both forms fail closed: if the variable is unset or empty, or the file is missing or outside the allowed source directories, the service refuses to start. A {% raw %}`{{ file }}`{% endraw %} path must live under an allowed directory — `/etc/ai-workspace` or `/secrets/ai-workspace` for the BFF, `/etc/platform-api` or `/secrets/platform-api` for the Platform API. Override the list with the shared `APIP_CONFIG_FILE_SOURCE_ALLOWLIST` (comma-separated; it **replaces** the defaults rather than extending them). + +!!! important "Two unrelated mechanisms" + The {% raw %}`{{ env }}`{% endraw %} and {% raw %}`{{ file }}`{% endraw %} tokens on this page are resolved by the service's config loader at startup, and only inside `config.toml`. The {% raw %}`{{ secret "handle" }}`{% endraw %} placeholder of [Secrets management](../secrets-management.md) is resolved by the gateway at request time, and only inside artifact configurations. Neither works in the other's place. + +## Where the values come from + +Provisioning the keys, certificates, and credentials the tokens resolve to is part of setting the stack up, not part of how the config loader works. For those steps, see: + +- [Run the setup script](../getting-started.md#step-2-run-the-setup-script) — what a fresh stack is given, and where each artifact lands. +- [Rerun the setup script](../getting-started.md#rerun-the-setup-script) — the flags, and which of them rotate what. +- [Provision the at-rest encryption key manually](../getting-started.md#provision-the-at-rest-encryption-key-manually) — the path for a deployment that doesn't use the script. +- [Change environment values after setup](../getting-started.md#change-environment-values-after-setup) — editing `api-platform.env`, the file Compose loads into the containers. + +## Related + +- [Get started with AI Workspace](../getting-started.md): provision the keys, certificates, and credentials the tokens resolve to +- [Change the ports AI Workspace uses](ports.md): move the stack off its default ports diff --git a/en/docs/ai-workspace/1.0.0/setting-up/database.md b/en/docs/ai-workspace/1.0.0/setting-up/database.md new file mode 100644 index 000000000..9ccdfe26c --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/setting-up/database.md @@ -0,0 +1,213 @@ +--- +title: "Connect a database to the Platform API" +description: "Move the Platform API from SQLite onto PostgreSQL or SQL Server: create the database, configure the connection, secure it with TLS, and tune the pool." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/database/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/database.md +tags: + - cloud + - ai-workspace + - configuration + - database +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-31 +content_type: "how-to" +--- + +# Connect a database to the Platform API + +The Platform API stores every artifact you create in AI Workspace — gateways, LLM providers, proxies, applications, and subscriptions. This guide is written for the administrator who deploys AI Workspace, and it covers moving that storage from the default SQLite file onto a database server. + +AI Workspace itself holds no persistent data of its own, so this is the only database in the stack. + +## Supported drivers + +Set the driver in `[platform_api.database]`. These values are accepted: + +| `driver` value | Database | +|----------------|----------| +| `sqlite3` | SQLite, the default—a single file, no server to run | +| `postgres`, `postgresql`, `pgx` | PostgreSQL | +| `sqlserver`, `mssql` | Microsoft SQL Server | + +## The default: SQLite + +Out of the box the Platform API writes to a SQLite file inside the container, persisted in the `platform-api-data` Docker volume: + +```toml +[platform_api.database] +driver = "sqlite3" +path = "./data/api_platform.db" +``` + +The path is relative to the container's working directory, `/app`, and the volume mounts at `/app/data`. The data survives `docker compose down` and restarts. Only `docker compose down -v` deletes it. + +SQLite suits a single instance you're evaluating or running for one team. Move to a database server when you need any of the following: + +- More than one Platform API replica, since they can't share a SQLite file +- Backup, restore, and point-in-time recovery run by your existing database tooling +- Storage that outlives the container host + +## Step 1: Create the database and user + +Create the database and the user the Platform API connects as. Don't grant the user rights to create tables. A database administrator provisions the schema in the next step, and the Platform API needs only to read and write the rows. + +=== "PostgreSQL" + + Run these statements on your PostgreSQL server: + + ```sql + CREATE DATABASE platform_api; + CREATE USER platform_api WITH PASSWORD ''; + GRANT CONNECT ON DATABASE platform_api TO platform_api; + ``` + + Then connect to the `platform_api` database as an administrator. Revoke the ability to create objects in the `public` schema. PostgreSQL grants that ability to every user before version 15: + + ```sql + REVOKE CREATE ON SCHEMA public FROM platform_api; + REVOKE CREATE ON SCHEMA public FROM PUBLIC; + ``` + +=== "SQL Server" + + Create the database and a server login of the same name. Run these statements against the `master` database: + + ```sql + CREATE DATABASE platform_api; + CREATE LOGIN platform_api WITH PASSWORD = ''; + ``` + + Step 2 maps that login to a database user. + +## Step 2: Provision the schema + +Run the bundled script for your database yourself, as a user that can create tables, before the first start: + +=== "PostgreSQL" + + ```bash + psql -h -U -d platform_api \ + -f resources/platform-api/db-scripts/schema.postgres.sql + ``` + +=== "SQL Server" + + ```bash + sqlcmd -S -U -d platform_api \ + -i resources/platform-api/db-scripts/schema.sqlserver.sql + ``` + +The distribution ships one script per database at `resources/platform-api/db-scripts/`: + +| Database | Script | +|----------|--------| +| PostgreSQL | `schema.postgres.sql` | +| SQL Server | `schema.sqlserver.sql` | +| SQLite | `schema.sqlite.sql` — applied for you, see [The default: SQLite](#the-default-sqlite) | + +Then grant the Platform API's user read and write access to what the script created. Run the statements for your database, connected to the `platform_api` database. + +=== "PostgreSQL" + + ```sql + GRANT USAGE ON SCHEMA public TO platform_api; + GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO platform_api; + GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO platform_api; + ``` + +=== "SQL Server" + + Create a database user for the login and add it to the two built-in data roles: + + ```sql + CREATE USER platform_api FOR LOGIN platform_api; + ALTER ROLE db_datareader ADD MEMBER platform_api; + ALTER ROLE db_datawriter ADD MEMBER platform_api; + ``` + +## Step 3: Configure the connection + +Edit `[platform_api.database]` in `configs/config.toml`. These keys carry no interpolation tokens in the shipped file, so set them in the file directly: + +{% raw %} + +```toml +[platform_api.database] +driver = "postgres" +host = "postgres.example.com" +port = 5432 +name = "platform_api" +user = "platform_api" +password = '{{ file "/secrets/platform-api/postgres_password" }}' +ssl_mode = "require" +``` + +{% endraw %} + +`host`, `port`, `name`, and `user` are all required for any driver other than `sqlite3`. Leaving one unset fails config load rather than starting with an incomplete connection. The `path` key is ignored once the driver isn't `sqlite3`, so you can leave it in place. + +For SQL Server, set `driver = "sqlserver"` and `port = 1433`. + +!!! warning "Never write the database password as a literal" + The {% raw %}`{{ file }}`{% endraw %} token above reads it from a mounted file, which is the right choice in production. Resolution fails closed, so a missing or unreadable file aborts startup rather than connecting with an empty password. Mount the secret at that path and add the mount to the `platform-api` service in `docker-compose.yaml`. To read it from `api-platform.env` instead, swap the token for {% raw %}`'{{ env "APIP_CP_DATABASE_PASSWORD" }}'`{% endraw %}. + +## Step 4: Secure the connection with TLS + +`ssl_mode` takes one of four values, in increasing order of strictness: + +| Value | Behavior | +|-------|----------| +| `disable` | No TLS. Use only when the database is reachable over a trusted network path. | +| `require` | Encrypt, but don't verify the server's certificate. | +| `verify-ca` | Encrypt and verify the certificate against a certificate authority (CA). | +| `verify-full` | Encrypt, verify the certificate, and check that its hostname matches. | + +`verify-ca` and `verify-full` need the CA certificate that signed the server's certificate: + +```toml +ssl_mode = "verify-full" +ssl_root_cert = "/etc/platform-api/certs/db-ca.pem" +``` + +For mutual TLS, add a client certificate and key. Set both or neither: + +```toml +ssl_cert = "/etc/platform-api/certs/db-client.pem" +ssl_key = "/etc/platform-api/certs/db-client-key.pem" +``` + +Mutual TLS is PostgreSQL only. The SQL Server driver has no client-certificate support, so `ssl_cert` and `ssl_key` don't apply there. + +Mount each of these files into the `platform-api` container and point the keys at the mounted paths. + +## Step 5: Tune the connection pool + +The defaults suit a single instance. Raise them for production traffic, keeping `max_open_conns` within what your server's own connection limit allows across every client: + +```toml +max_open_conns = 25 # maximum open connections +max_idle_conns = 10 # maximum idle connections kept in the pool +conn_max_lifetime = 300 # seconds before a connection is recycled +``` + +## Step 6: Restart and verify + +Recreate the container so it reloads the configuration: + +```bash +docker compose up --force-recreate platform-api +``` + +Confirm the service is healthy: + +```bash +curl -fk https://localhost:9243/health +``` + +!!! important "Switching drivers starts from an empty database" + Changing `driver` points the Platform API at different storage; it doesn't move anything there. The gateways, providers, proxies, and users in your SQLite file don't appear in PostgreSQL, and the distribution ships no tool to copy them across. Choose your database before you build out an environment, or plan to recreate its artifacts. + +## Next steps + +- [AI Workspace configuration](configuration.md): how interpolation tokens deliver values into `config.toml`, including the database password from a mounted file +- [Connect an identity provider](authentication/connect-an-identity-provider.md): move user login off the local user list as well diff --git a/en/docs/ai-workspace/1.0.0/setting-up/ports.md b/en/docs/ai-workspace/1.0.0/setting-up/ports.md new file mode 100644 index 000000000..bd8838518 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/setting-up/ports.md @@ -0,0 +1,137 @@ +--- +title: "Change the ports AI Workspace uses" +description: "Move the AI Workspace and Platform API off their default ports, either by remapping the published host port or by changing the port each service listens on." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/ports/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/ports.md +tags: + - cloud + - ai-workspace + - configuration +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-31 +content_type: "how-to" +--- + +# Change the ports AI Workspace uses + +The stack listens on these ports by default: + +| Port | Service | Purpose | +|------|---------|---------| +| `9643` | AI Workspace | HTTPS — the browser entry point | +| `9243` | Platform API | HTTPS — the backend REST API | +| `9543` | API Portal | HTTPS — only when you enable the `api-portal` profile | + +If another process on your machine holds one of these, or your organization reserves it, two approaches move the stack off it. They solve different problems. + +## Remap the published host port + +The containers keep listening on 9643 and 9243, and Docker publishes them on host ports you choose. This is the shorter change, and it's enough when the conflict is on your own machine. + +1. In `docker-compose.yaml`, edit the host side — the left number — of each `ports:` mapping. This example moves the AI Workspace to `8443` and the Platform API to `8244`: + + ```yaml + services: + platform-api: + ports: + - "8244:9243" + + ai-workspace: + ports: + - "8443:9643" + ``` + + Leave the container side and the `healthcheck` entries alone. Both run inside the container, where the original ports still apply. + +2. In `configs/config.toml`, point `controlplane_host` at the new Platform API host port, so the gateway setup commands the workspace shows carry the published port. Use an address that your gateway can reach: + + ```toml + [ai_workspace.gateway] + controlplane_host = ":8244" + ``` + + For a gateway in another container on the same machine, `host.docker.internal:8244` works. Docker Desktop on macOS and Windows resolves that name automatically. On Linux, add an `extra_hosts` entry to the gateway's Compose service, `host.docker.internal:host-gateway`, or use an address the gateway can reach instead. From anywhere else, use the machine's hostname or IP address. Leave `[ai_workspace.control_plane] url` on `https://platform-api:9243` — see [Two keys that aren't interchangeable](#two-keys-that-arent-interchangeable). + +3. Set `APIP_AIW_DOMAIN=localhost:8443` in `api-platform.env`, so the startup log banner prints an address you can open. + +## Change the port each service listens on + +The services bind to different ports themselves. Choose this when something inside the Docker network, such as a reverse proxy sharing it, needs the new port too. + +1. In `configs/config.toml`, set the AI Workspace listener port under `[ai_workspace.server.https]`, or set `APIP_AIW_SERVER_HTTPS_PORT=8443` in `api-platform.env` — the shipped key reads that variable. + +2. Add a `[platform_api.server.https]` table. The shipped file omits it, so add the whole table including the certificate paths, which the Platform API requires on its HTTPS listener: + + ```toml + [platform_api.server.https] + enabled = true + port = 8244 + cert_file = "/app/data/certs/cert.pem" + key_file = "/app/data/certs/key.pem" + ``` + +3. Point the AI Workspace at the new Platform API port, and give gateways a host address they can reach on the new port: + + ```toml + [ai_workspace.control_plane] + url = "https://platform-api:8244" + + [ai_workspace.gateway] + controlplane_host = ":8244" + ``` + + As in the previous approach, `host.docker.internal:8244` works for a gateway in another container on the same machine. Docker Desktop maps that hostname automatically on macOS and Windows. On Linux, map it yourself with `extra_hosts`. + +4. In `docker-compose.yaml`, update both sides of each mapping, and the health check URLs, which run inside the container against the new listener ports: + + ```yaml + services: + platform-api: + ports: + - "8244:8244" + healthcheck: + test: ["CMD", "curl", "-fk", "https://localhost:8244/health"] + + ai-workspace: + ports: + - "8443:8443" + healthcheck: + test: ["CMD-SHELL", "curl -fs http://localhost:9680/healthz || curl -fk https://localhost:8443/healthz"] + ``` + +5. Set `APIP_AIW_DOMAIN=localhost:8443` in `api-platform.env`. + +## Two keys that aren't interchangeable + +Both approaches touch `url` and `controlplane_host`, which sit either side of the Compose network boundary: + +| Key | Who connects to it | Value | +|-----|--------------------|-------| +| `[ai_workspace.control_plane] url` | The AI Workspace container, over the Compose network | A full URL using the internal name and the **container** port | +| `[ai_workspace.gateway] controlplane_host` | An AI gateway deployed outside the stack | A bare `host:port` with no scheme, using the **published** port | + +Nothing in AI Workspace connects to `controlplane_host`. The value is display-only: the workspace substitutes it into the gateway setup commands the **Get Started** section shows an admin, such as the `APIP_GW_CONTROLLER_CONTROLPLANE_HOST` line and the Helm `--set gateway.controller.controlPlane.host` flag. The admin then copies those commands to the machine running the gateway, which is what makes the connection. + +So a wrong value leaves AI Workspace working normally and breaks the gateway instead. The printed commands look right, but the gateway they configure can't reach the control plane and never registers. + +Set `controlplane_host` to an address that's reachable from the gateway's network: + +- For a gateway in another container on the same machine, use `host.docker.internal`. +- For a gateway anywhere else, use the machine's hostname or IP address. +- Don't use `platform-api`. That name resolves only inside the Compose network, so a gateway outside the stack can't reach it. + +## Apply the change + +Recreate the containers so they pick up the new values: + +```bash +docker compose up --force-recreate +``` + +!!! note "Ports in an OpenID Connect (OIDC) setup" + OIDC redirect URLs carry the port. Update `redirect_url` and `post_logout_redirect_url` under `[ai_workspace.auth.oidc]` in `configs/config.toml`, and update the matching URLs registered in your identity provider. See [Connect an identity provider](authentication/connect-an-identity-provider.md). + +## Related + +- [AI Workspace configuration](configuration.md) — how interpolation tokens deliver values into `config.toml` +- [Get started with AI Workspace](../getting-started.md) — the quickstart these defaults come from diff --git a/en/docs/ai-workspace/1.0.0/sync-gateway-created-artifacts.md b/en/docs/ai-workspace/1.0.0/sync-gateway-created-artifacts.md new file mode 100644 index 000000000..ee706e67e --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/sync-gateway-created-artifacts.md @@ -0,0 +1,359 @@ +--- +title: "Manage gateway-deployed AI artifacts in AI Workspace" +description: "Create an LLM provider template, LLM provider, LLM proxy, or MCP proxy on the AI Gateway and view the read-only copy that syncs up to AI Workspace." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/sync-gateway-created-artifacts/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/sync-gateway-created-artifacts.md +tags: + - cloud + - ai-workspace + - ai-gateway +author: WSO2 API Platform Documentation Team +last_updated: 2026-07-31 +content_type: "how-to" +--- + +# Manage gateway-deployed AI artifacts in AI Workspace + +You can create four kinds of AI artifact directly on the AI Gateway: + +- **Large language model (LLM) Provider Template** +- **LLM Provider** +- **LLM Proxy** +- **Model Context Protocol (MCP) Proxy** + +Each one syncs up to **AI Workspace** automatically, where it appears as a copy the gateway owns. The deployment fields the gateway runs the artifact from are **read-only**. Runtime-neutral details stay editable, such as the description, documentation, OpenAPI definitions, and template connection details. For the full breakdown, see [What you can and can't change in AI Workspace](#what-you-can-and-cant-change-in-ai-workspace). + +This is the reverse of the usual top-down flow: + +| | You create it in… | The AI Workspace copy is… | +|---|---|---| +| **Top-down** | AI Workspace, then it's pushed to the gateway | editable—you own it | +| **This guide (bottom-up)** | the gateway, then it's synced up to AI Workspace | read-only for deployment fields—the gateway owns them; runtime-neutral details stay editable | + +Because the gateway owns these artifacts, they keep serving traffic even if AI Workspace is temporarily unavailable, and any change you make on the gateway is synced up automatically. + +## Prerequisites + +- A gateway that's registered with, and can reach, your AI Workspace. +- Syncing enabled on the gateway. See [Enable the sync](#enable-the-sync); it's on by default. +- For **LLM Proxies** and **MCP Proxies**, which belong to a project: the project they reference must already exist in your organization in AI Workspace. + +## Enable the sync + +Syncing is controlled by a single gateway setting, `deployment_sync_enabled`, which is **on by default**. It controls syncing in both directions between the gateway and AI Workspace. + +**File:** `config.toml` + +```toml +[controller.controlplane] +gateway_name = "default" +insecure_skip_verify = false + +# Sync artifacts with AI Workspace (on by default). +deployment_sync_enabled = true +``` + +Set `insecure_skip_verify = true` only for local development against a known self-signed certificate. It disables verification of the AI Workspace TLS certificate, so never enable it in production. + +Restart the gateway after changing the setting. When it's turned off, the gateway neither syncs its artifacts up nor receives artifacts from AI Workspace. + +## How the sync works + +When you create or update an artifact on the gateway: + +```text +Create on the gateway ─┬─▶ takes effect immediately (starts serving traffic) + └─▶ synced to AI Workspace ─▶ appears as a read-only copy +``` + +The sync happens automatically in the background—you don't trigger it. A few things to know: + +- **Matched by name.** Each artifact is identified by the name you give it (`metadata.name`). Re-creating an artifact with the same name on the gateway updates the same AI Workspace copy instead of creating a duplicate. +- **References use names.** An LLM provider names its template, and an LLM proxy names its provider. Create them in order — the template, then the provider, then the proxy — so each reference resolves. MCP proxies stand on their own. +- **Most recent deployment wins.** If the same artifact is deployed on more than one gateway, AI Workspace shows the version from the most recent deployment. + +## Supported artifacts + +Four AI artifact kinds sync from the gateway to AI Workspace. You create them through the gateway's management API, under the base path `/api/management/v1` (default port `9090`): + +| Kind | Management API endpoint | Belongs to a project? | +|------|-------------------------|-----------------------| +| `LlmProviderTemplate` | `/api/management/v1/llm-provider-templates` | No (organization level) | +| `LlmProvider` | `/api/management/v1/llm-providers` | No (organization level) | +| `LlmProxy` | `/api/management/v1/llm-proxies` | Yes | +| `Mcp` | `/api/management/v1/mcp-proxies` | Yes | + +All manifests use `apiVersion: gateway.api-platform.wso2.com/v1`. Project-scoped kinds name their project in an annotation using the project handle: + +```yaml +metadata: + annotations: + "gateway.api-platform.wso2.com/project-id": "default" +``` + +### Call the management API + +The examples below use these conventions: + +- **Base URL:** `http://localhost:9090/api/management/v1` +- **Content type:** `Content-Type: text/yaml` (the API also accepts JSON) +- **Auth:** HTTP Basic, using a user configured under `[[controller.auth.basic.users]]` in `config.toml`. Pass your own credentials rather than hard-coding them: + + ```bash + export GW_USER='' GW_PASSWORD='' + ``` + +- **Body:** `--data-binary '@.yaml'` uploads a manifest file as-is. `--data-binary '@-'` reads the manifest from standard input instead, which lets you substitute placeholder values as you upload. + +## Create the artifacts on the gateway + +This walkthrough builds a complete **LLM Proxy** together with the two artifacts it depends on: an **LLM Provider Template** and an **LLM Provider**. It then adds a standalone **MCP Proxy**. Each `curl` creates the artifact on the gateway. The artifact starts serving immediately and syncs to AI Workspace. + +Create them in dependency order so each reference resolves: + +```text +LlmProviderTemplate ──(spec.template)──▶ LlmProvider ──(spec.provider.id)──▶ LlmProxy +``` + +!!! note + The LLM proxy and MCP proxy reference the project `default`. Make sure that project exists in your organization in AI Workspace first. + +### Step 1: Create the LLM provider template + +The provider references a template by name, so create the template first. `llm-provider-template.yaml`: + +```yaml +apiVersion: gateway.api-platform.wso2.com/v1 +kind: LlmProviderTemplate +metadata: + name: my-llm-provider-template +spec: + displayName: Custom OpenAI Template + promptTokens: { location: payload, identifier: $.usage.inputTokens } + completionTokens: { location: payload, identifier: $.usage.outputTokens } + totalTokens: { location: payload, identifier: $.usage.totalTokens } + # ... see gateway/examples/llm-provider-template.yaml for the full manifest +``` + +```bash +curl --location 'http://localhost:9090/api/management/v1/llm-provider-templates' \ + --header 'Content-Type: text/yaml' \ + --user "$GW_USER:$GW_PASSWORD" \ + --data-binary '@llm-provider-template.yaml' +``` + +### Step 2: Create the LLM provider + +The provider links to the LLM provider template above via `spec.template`. `llm-provider.yaml`: + +```yaml +apiVersion: gateway.api-platform.wso2.com/v1 +kind: LlmProvider +metadata: + name: my-llm-provider +spec: + displayName: WSO2 My LLM Provider + version: v1.0 + context: /openai-dp-1 + template: my-llm-provider-template # ← must match the template's metadata.name + vhost: api.my-llm-provider.local + upstream: + url: + auth: { type: api-key, header: Authorization, value: } + # ... accessControl + policies omitted; see gateway/examples/llm-provider.yaml +``` + +!!! warning "Substitute the placeholders, and keep credentials out of the file" + The manifests in this guide carry placeholders such as ``. The gateway stores whatever you send, so substitute your own values before each request. The commands below pipe the manifest through [`yq`](https://github.com/mikefarah/yq) and post the result. The placeholders stay in the file on disk, which keeps real credentials out of source control. To avoid holding the value in your shell as well, reference a secret. See [Secrets management](secrets-management.md). + + `yq` edits the manifest as YAML, and its `strenv` function assigns each value as a string. A URL or API key that contains `:`, `#`, `{`, or a quotation mark is preserved as a literal value. + +Set your upstream values, then substitute them as you post the manifest: + +```bash +export UPSTREAM_URL='' UPSTREAM_API_KEY='' + +yq '.spec.upstream.url = strenv(UPSTREAM_URL) | + .spec.upstream.auth.value = strenv(UPSTREAM_API_KEY)' llm-provider.yaml | + curl --location 'http://localhost:9090/api/management/v1/llm-providers' \ + --header 'Content-Type: text/yaml' \ + --user "$GW_USER:$GW_PASSWORD" \ + --data-binary '@-' +``` + +### Step 3: Create the LLM proxy + +The proxy belongs to a project (the `project-id` annotation) and links to the LLM provider via `spec.provider.id`. `llm-proxy.yaml`: + +```yaml +apiVersion: gateway.api-platform.wso2.com/v1 +kind: LlmProxy +metadata: + name: wso2con-assistant + annotations: + "gateway.api-platform.wso2.com/project-id": "default" # ← must be an existing project +spec: + displayName: WSO2 Con Assistant + version: v1.0 + context: "/project-1/assistant" + provider: + id: my-llm-provider # ← must match the provider's metadata.name + auth: { header: X-API-Key, type: api-key, value: } + # ... policies omitted; see gateway/examples/llm-proxy.yaml +``` + +```bash +export PROVIDER_API_KEY='' + +yq '.spec.provider.auth.value = strenv(PROVIDER_API_KEY)' llm-proxy.yaml | + curl --location 'http://localhost:9090/api/management/v1/llm-proxies' \ + --header 'Content-Type: text/yaml' \ + --user "$GW_USER:$GW_PASSWORD" \ + --data-binary '@-' +``` + +### Step 4: Create an MCP proxy + +An MCP proxy also belongs to a project but stands on its own (no template or provider prerequisite). `mcp-proxy.yaml`: + +```yaml +apiVersion: gateway.api-platform.wso2.com/v1 +kind: Mcp +metadata: + name: everything-mcp-v1.0 + annotations: + "gateway.api-platform.wso2.com/project-id": "default" +spec: + displayName: Everything + version: v1.0 + context: "/project-1/everything" + specVersion: "2025-06-18" + upstream: + url: + auth: { header: X-Api-Key, type: header, value: } + # ... policies omitted; see gateway/examples/mcp-proxy.yaml +``` + +```bash +export MCP_SERVER_URL='' MCP_SERVER_API_KEY='' + +yq '.spec.upstream.url = strenv(MCP_SERVER_URL) | + .spec.upstream.auth.value = strenv(MCP_SERVER_API_KEY)' mcp-proxy.yaml | + curl --location 'http://localhost:9090/api/management/v1/mcp-proxies' \ + --header 'Content-Type: text/yaml' \ + --user "$GW_USER:$GW_PASSWORD" \ + --data-binary '@-' +``` + +## View them in AI Workspace + +The gateway syncs each artifact up automatically. Shortly after you create them, all four appear in AI Workspace as **read-only** copies, each keeping the name you gave it on the gateway. + +Open **AI Workspace** for the organization your gateway is registered with, then find each artifact in its section of the sidebar. + +### LLM provider template + +The template appears under **Settings** > **LLM Provider Templates**: + +![LLM provider template synced from the gateway to AI Workspace](../../assets/img/ai-workspace/bottom-up/dp-to-cp-llm-provider-template.png) + +### LLM provider + +The provider appears under **LLM** > **LLM Providers**: + +![LLM provider synced from the gateway to AI Workspace](../../assets/img/ai-workspace/bottom-up/dp-to-cp-llm-provider.png) + +### LLM proxy + +The proxy appears under **LLM** > **App LLM Proxies**, in the **Default** project: + +![LLM proxy synced from the gateway to AI Workspace](../../assets/img/ai-workspace/bottom-up/dp-to-cp-llm-proxy.png) + +### MCP proxy + +The MCP proxy appears under **MCP** > **MCP Proxies**, in the **Default** project: + +![MCP proxy synced from the gateway to AI Workspace](../../assets/img/ai-workspace/bottom-up/dp-to-cp-mcp-proxy.png) + +Open any of them to browse the full configuration. It opens in a read-only view — the edit and deploy actions are unavailable because the gateway owns the artifact. + +If an artifact hasn't appeared after a short wait, see [Troubleshooting](#troubleshooting). + +## What you can and can't change in AI Workspace + +A gateway-created artifact is **read-only** in AI Workspace because the gateway owns it. "Read-only" applies to anything the gateway uses to run the artifact — everything else stays editable. + +**You _can_ change things that don't affect how the gateway runs the artifact** (these stay in AI Workspace only): + +- Its description and display name +- Documentation and API (OpenAPI) definitions +- For an **LLM Provider Template**: its connection details (endpoint URL, auth type/header), logo, and OpenAPI spec + +**You _can't_ change what the gateway uses to run the artifact.** Make those changes on the gateway instead — they sync up automatically. This includes: + +- Upstreams, the auth/routing used to serve traffic, and policies + + ![Read-only access control settings of a gateway-origin LLM provider](../../assets/img/ai-workspace/bottom-up/dp-origin-llm-provider-access-control.png) + +- An LLM provider template's token-tracking settings + + ![Read-only token mapping settings of a gateway-origin LLM provider template](../../assets/img/ai-workspace/bottom-up/dp-origin-llm-provider-template-token-mapping.png) + +- Deploying, redeploying, or undeploying the artifact + + ![Deployment view of a gateway-origin LLM proxy, with the deploy actions unavailable](../../assets/img/ai-workspace/bottom-up/dp-origin-llm-proxy-deployment.png) + +- Deleting it while it's still deployed on a gateway (undeploy it from all gateways first) + +AI Workspace doesn't offer the actions it can't perform, and it declines an edit that would change how the gateway runs the artifact. + +## Update and delete artifacts + +| On the gateway you… | In AI Workspace… | +|---------------------|----------------------| +| **Update** the artifact | the read-only copy refreshes automatically | +| **Delete** the artifact | the copy is kept (not removed) and shown as no longer deployed on that gateway, preserving a record of it | + +To re-sync an artifact after a hiccup, re-apply it on the gateway with the same definition. + +## If AI Workspace is temporarily unavailable + +Syncing is resilient. If AI Workspace can't be reached when you create or change an artifact: + +- The artifact still takes effect on the gateway and keeps serving traffic. +- The gateway retries the sync automatically. +- When the connection is restored, everything that hasn't synced yet is pushed up — no manual action needed. + +You can create artifacts on a gateway while it's disconnected, and they reconcile up on their own once it reconnects. This applies to all four artifact kinds. + +## Immutable gateways + +Some gateways run in **immutable** mode, where artifacts are loaded from on-disk configuration at startup rather than created through the management API (see [Immutable Gateway](../../ai-gateway/next/deployment-modes/immutable-gateway.md)). + +The sync behaves exactly the same for these gateways: artifacts loaded from files are synced up to AI Workspace just like ones created through the management API, with the same read-only copies and the same automatic reconciliation — no extra configuration. An immutable, file-driven gateway is still fully visible in AI Workspace. + +## Troubleshooting + +### An artifact I created on the gateway doesn't appear in AI Workspace + +- **Syncing is turned off.** Set `deployment_sync_enabled = true` in the gateway's `config.toml` and restart the gateway. +- **The AI Workspace can't be reached.** The artifact still works on the gateway; the sync retries automatically and catches up once the connection is restored. Check that the gateway is connected to AI Workspace. +- **The project doesn't exist** (LLM proxy or MCP proxy). These belong to a project. Create the project named in the artifact's `project-id` annotation in your organization, then re-apply the artifact on the gateway: + + ```yaml + metadata: + annotations: + "gateway.api-platform.wso2.com/project-id": "default" + ``` + +- **A referenced artifact isn't there yet.** An LLM provider needs its template, and an LLM proxy needs its provider. Create them in order: the template, then the provider, then the proxy. The dependent artifact catches up on its own once the artifact it references has synced. + +### I can't edit, deploy, or delete a gateway-created artifact in AI Workspace + +This is expected—the gateway owns the artifact, so its deployment fields are read-only in AI Workspace. + +- Make configuration and deployment changes on the **gateway**; they sync up automatically. +- You can still edit runtime-neutral details: the description, display name, documentation, OpenAPI definitions, and, for an LLM provider template, its connection details and logo. +- To delete it from AI Workspace, first undeploy it from **all** gateways it was deployed to, then delete. + +See [What you can and can't change](#what-you-can-and-cant-change-in-ai-workspace) for the full list. diff --git a/en/docs/ai-workspace/1.0.0/using-sdks.md b/en/docs/ai-workspace/1.0.0/using-sdks.md new file mode 100644 index 000000000..c408f0289 --- /dev/null +++ b/en/docs/ai-workspace/1.0.0/using-sdks.md @@ -0,0 +1,556 @@ +--- +title: "Invoke providers and proxies via SDKs" +description: "Call a deployed LLM provider or App LLM proxy using the OpenAI, Anthropic, Gemini, Mistral, Azure OpenAI, or LangChain SDKs." +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/using-sdks/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/using-sdks.md +tags: + - cloud + - ai-workspace + - sdks +author: WSO2 API Platform Documentation Team +last_updated: 2026-06-22 +content_type: "how-to" +--- + +# Invoke providers and proxies via SDKs + +Deploy a large language model (LLM) provider or App LLM proxy in the AI Workspace first. You can then invoke it with any supported AI software development kit (SDK). Point the SDK at the gateway's Invoke URL, and authenticate with your generated API key. + +The examples below apply to both providers and proxies. The only difference between the two is the Invoke URL you supply. + +## Prerequisites + +- An [LLM provider](llm-providers/configure-provider.md) or [App LLM proxy](llm-proxies/configure-proxy.md) deployed to a gateway +- The **Invoke URL** for the deployed endpoint +- A generated **API key** + +## Authentication + +All requests to the gateway must include your API key in the request header named in the **Security** tab of your provider or proxy. That header is `X-API-Key` by default, and the code examples below use that default. + +!!! note + Depending on the SDK or provider you use, choose the header name that works best and set it in the **Security** tab. See [Configure inbound authentication](configure-inbound-auth.md). The examples below use the default `X-API-Key` header. + +## OpenAI + +!!! info "Invoke URL format" + Append `/v1` to the Invoke URL shown in the console: + ```text + https://{gateway-host}/{context}/v1 + ``` + +=== "OpenAI SDK" + + **Install:** `pip install openai` + + **Basic chat completion:** + + ```python + from openai import OpenAI + + INVOKE_URL = "https:////v1" + API_KEY = "" + + client = OpenAI( + api_key=API_KEY, + base_url=INVOKE_URL, + default_headers={"X-API-Key": API_KEY}, + ) + + response = client.chat.completions.create( + model="gpt-4o", + messages=[{"role": "user", "content": "What is WSO2?"}], + ) + + print(response.choices[0].message.content) + ``` + + **Streaming:** + + ```python + stream = client.chat.completions.create( + model="gpt-4o", + messages=[{"role": "user", "content": "Count from 1 to 5."}], + stream=True, + ) + + for chunk in stream: + delta = chunk.choices[0].delta.content if chunk.choices else None + if delta: + print(delta, end="", flush=True) + ``` + +=== "LangChain" + + **Install:** `pip install langchain-openai` + + **Basic invoke:** + + ```python + from langchain_openai import ChatOpenAI + from langchain_core.messages import HumanMessage + + INVOKE_URL = "https:////v1" + API_KEY = "" + + llm = ChatOpenAI( + model="gpt-4o", + api_key=API_KEY, + base_url=INVOKE_URL, + default_headers={"X-API-Key": API_KEY}, + ) + + response = llm.invoke([HumanMessage(content="What is WSO2?")]) + print(response.content) + ``` + + **Streaming:** + + ```python + for chunk in llm.stream([HumanMessage(content="Count from 1 to 5.")]): + if chunk.content: + print(chunk.content, end="", flush=True) + ``` + +## Anthropic + +=== "Anthropic SDK" + + **Install:** `pip install anthropic` + + !!! note + The Anthropic SDK sends the `api_key` parameter as the `x-api-key` header automatically. No additional header configuration is needed. + + **Basic message:** + + ```python + import anthropic + + INVOKE_URL = "https:///" + API_KEY = "" + + client = anthropic.Anthropic( + api_key=API_KEY, + base_url=INVOKE_URL, + ) + + response = client.messages.create( + model="claude-sonnet-4-5", + max_tokens=1024, + messages=[{"role": "user", "content": "What is WSO2?"}], + ) + + print(response.content[0].text) + ``` + + **Streaming:** + + ```python + with client.messages.stream( + model="claude-sonnet-4-5", + max_tokens=1024, + messages=[{"role": "user", "content": "Count from 1 to 5."}], + ) as stream: + for text in stream.text_stream: + print(text, end="", flush=True) + ``` + +=== "LangChain" + + **Install:** `pip install langchain-anthropic` + + **Basic invoke:** + + ```python + from langchain_anthropic import ChatAnthropic + from langchain_core.messages import HumanMessage + + INVOKE_URL = "https:///" + API_KEY = "" + + llm = ChatAnthropic( + model="claude-sonnet-4-5", + api_key=API_KEY, + anthropic_api_url=INVOKE_URL, + default_headers={"X-API-Key": API_KEY}, + max_tokens=1024, + ) + + response = llm.invoke([HumanMessage(content="What is WSO2?")]) + print(response.content) + ``` + + **Streaming:** + + ```python + for chunk in llm.stream([HumanMessage(content="Count from 1 to 5.")]): + if chunk.content: + print(chunk.content, end="", flush=True) + ``` + +## Gemini + +=== "Google GenAI SDK" + + **Install:** `pip install google-genai` + + !!! note + The Gemini SDK normally sends its key as `x-goog-api-key`, which the gateway doesn't use for authentication. Pass `api_key="placeholder"` to satisfy the SDK and supply the real gateway key via `X-API-Key` in `HttpOptions`. + + **Basic content generation:** + + ```python + from google import genai + from google.genai import types as genai_types + + INVOKE_URL = "https:///" + API_KEY = "" + + http_options = genai_types.HttpOptions( + base_url=INVOKE_URL, + headers={"X-API-Key": API_KEY}, + ) + + client = genai.Client(api_key="placeholder", http_options=http_options) + + response = client.models.generate_content( + model="gemini-2.5-flash", + contents="What is WSO2?", + ) + + print(response.text) + ``` + + **Streaming:** + + ```python + for chunk in client.models.generate_content_stream( + model="gemini-2.5-flash", + contents="Count from 1 to 5.", + ): + if chunk.text: + print(chunk.text, end="", flush=True) + ``` + +=== "LangChain" + + **Install:** `pip install langchain-google-genai` + + **Basic invoke:** + + ```python + from langchain_google_genai import ChatGoogleGenerativeAI + from langchain_core.messages import HumanMessage + + INVOKE_URL = "https:///" + API_KEY = "" + + llm = ChatGoogleGenerativeAI( + model="gemini-2.5-flash", + google_api_key=API_KEY, + client_options={"api_endpoint": INVOKE_URL}, + additional_headers={"X-API-Key": API_KEY}, + ) + + response = llm.invoke([HumanMessage(content="What is WSO2?")]) + print(response.content) + ``` + + **Streaming:** + + ```python + for chunk in llm.stream([HumanMessage(content="Count from 1 to 5.")]): + if chunk.content: + print(chunk.content, end="", flush=True) + ``` + +## Mistral AI + +Mistral exposes both a native SDK and an OpenAI-compatible API at `/v1`. + +=== "Mistral SDK" + + **Install:** `pip install mistralai==2.4.5 httpx` + + !!! note + The Mistral SDK sends its API key as a `Bearer` token. Since the gateway requires `X-API-Key`, an httpx event hook injects this header on every outgoing request. + + **Basic chat completion:** + + ```python + import httpx + from mistralai import Mistral + + INVOKE_URL = "https:///" + API_KEY = "" + + def _inject_api_key(request): + request.headers["X-API-Key"] = API_KEY + + http_client = httpx.Client( + event_hooks={"request": [_inject_api_key]}, + ) + + client = Mistral( + api_key=API_KEY, + server_url=INVOKE_URL, + client=http_client, + ) + + response = client.chat.complete( + model="mistral-small-latest", + messages=[{"role": "user", "content": "What is WSO2?"}], + ) + + print(response.choices[0].message.content) + ``` + + **Streaming:** + + ```python + with client.chat.stream( + model="mistral-small-latest", + messages=[{"role": "user", "content": "Count from 1 to 5."}], + ) as stream: + for event in stream: + if event.data.choices and event.data.choices[0].delta.content: + print(event.data.choices[0].delta.content, end="", flush=True) + ``` + +=== "OpenAI SDK" + + **Install:** `pip install openai` + + Mistral's API is OpenAI-compatible. Append `/v1` to the Invoke URL. + + **Basic chat completion:** + + ```python + from openai import OpenAI + + INVOKE_URL = "https:////v1" + API_KEY = "" + + client = OpenAI( + api_key=API_KEY, + base_url=INVOKE_URL, + default_headers={"X-API-Key": API_KEY}, + ) + + response = client.chat.completions.create( + model="mistral-small-latest", + messages=[{"role": "user", "content": "What is WSO2?"}], + ) + + print(response.choices[0].message.content) + ``` + + **Streaming:** + + ```python + stream = client.chat.completions.create( + model="mistral-small-latest", + messages=[{"role": "user", "content": "Count from 1 to 5."}], + stream=True, + ) + + for chunk in stream: + delta = chunk.choices[0].delta.content if chunk.choices else None + if delta: + print(delta, end="", flush=True) + ``` + +=== "LangChain" + + **Install:** `pip install langchain-openai` + + LangChain's `ChatOpenAI` works with Mistral's OpenAI-compatible endpoint. Append `/v1` to the Invoke URL. + + **Basic invoke:** + + ```python + from langchain_openai import ChatOpenAI + from langchain_core.messages import HumanMessage + + INVOKE_URL = "https:////v1" + API_KEY = "" + + llm = ChatOpenAI( + model="mistral-small-latest", + api_key=API_KEY, + base_url=INVOKE_URL, + default_headers={"X-API-Key": API_KEY}, + ) + + response = llm.invoke([HumanMessage(content="What is WSO2?")]) + print(response.content) + ``` + + **Streaming:** + + ```python + for chunk in llm.stream([HumanMessage(content="Count from 1 to 5.")]): + if chunk.content: + print(chunk.content, end="", flush=True) + ``` + +## Azure OpenAI + +!!! note + The `model` / `azure_deployment` parameter must be your **Azure deployment name**, not the underlying model name. + + +=== "Azure OpenAI SDK" + + **Install:** `pip install openai` + + **Basic chat completion:** + + ```python + from openai import AzureOpenAI + + INVOKE_URL = "https:///" + API_KEY = "" + + client = AzureOpenAI( + api_key=API_KEY, + azure_endpoint=INVOKE_URL, + api_version="2024-10-21", + default_headers={"X-API-Key": API_KEY}, + ) + + response = client.chat.completions.create( + model="", + messages=[{"role": "user", "content": "What is WSO2?"}], + ) + + print(response.choices[0].message.content) + ``` + + **Streaming:** + + ```python + stream = client.chat.completions.create( + model="", + messages=[{"role": "user", "content": "Count from 1 to 5."}], + stream=True, + ) + + for chunk in stream: + delta = chunk.choices[0].delta.content if chunk.choices else None + if delta: + print(delta, end="", flush=True) + ``` + +=== "LangChain" + + **Install:** `pip install langchain-openai` + + **Basic invoke:** + + ```python + from langchain_openai import AzureChatOpenAI + from langchain_core.messages import HumanMessage + + INVOKE_URL = "https:///" + API_KEY = "" + + llm = AzureChatOpenAI( + azure_deployment="", + api_version="2024-10-21", + azure_endpoint=INVOKE_URL, + api_key=API_KEY, + default_headers={"X-API-Key": API_KEY}, + ) + + response = llm.invoke([HumanMessage(content="What is WSO2?")]) + print(response.content) + ``` + + **Streaming:** + + ```python + for chunk in llm.stream([HumanMessage(content="Count from 1 to 5.")]): + if chunk.content: + print(chunk.content, end="", flush=True) + ``` + +## Azure AI Foundry + +!!! note + The `model` / `azure_deployment` parameter must be your **Azure deployment name**. + +=== "Azure OpenAI SDK" + + **Install:** `pip install openai` + + **Basic chat completion:** + + ```python + from openai import AzureOpenAI + + INVOKE_URL = "https:///" + API_KEY = "" + + client = AzureOpenAI( + api_key=API_KEY, + azure_endpoint=INVOKE_URL, + api_version="2024-05-01-preview", + default_headers={"X-API-Key": API_KEY}, + ) + + response = client.chat.completions.create( + model="", + messages=[{"role": "user", "content": "What is WSO2?"}], + ) + + print(response.choices[0].message.content) + ``` + + **Streaming:** + + ```python + stream = client.chat.completions.create( + model="", + messages=[{"role": "user", "content": "Count from 1 to 5."}], + stream=True, + ) + + for chunk in stream: + delta = chunk.choices[0].delta.content if chunk.choices else None + if delta: + print(delta, end="", flush=True) + ``` + +=== "LangChain" + + **Install:** `pip install langchain-openai` + + **Basic invoke:** + + ```python + from langchain_openai import AzureChatOpenAI + from langchain_core.messages import HumanMessage + + INVOKE_URL = "https:///" + API_KEY = "" + + llm = AzureChatOpenAI( + azure_deployment="", + api_version="2024-05-01-preview", + azure_endpoint=INVOKE_URL, + api_key=API_KEY, + default_headers={"X-API-Key": API_KEY}, + ) + + response = llm.invoke([HumanMessage(content="What is WSO2?")]) + print(response.content) + ``` + + **Streaming:** + + ```python + for chunk in llm.stream([HumanMessage(content="Count from 1 to 5.")]): + if chunk.content: + print(chunk.content, end="", flush=True) + ``` diff --git a/en/docs/next/ai-workspace/about-this-release.md b/en/docs/ai-workspace/next/about-this-release.md similarity index 99% rename from en/docs/next/ai-workspace/about-this-release.md rename to en/docs/ai-workspace/next/about-this-release.md index bf99c786f..9d6ac5ca4 100644 --- a/en/docs/next/ai-workspace/about-this-release.md +++ b/en/docs/ai-workspace/next/about-this-release.md @@ -1,8 +1,8 @@ --- title: "About this release" description: "Features, compatible product versions, key considerations, and known limitations of WSO2 AI Workspace 1.0.0." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/about-this-release/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/about-this-release.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/about-this-release/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/about-this-release.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/ai-gateways/setting-up.md b/en/docs/ai-workspace/next/ai-gateways/setting-up.md similarity index 98% rename from en/docs/next/ai-workspace/ai-gateways/setting-up.md rename to en/docs/ai-workspace/next/ai-gateways/setting-up.md index 30f16edfa..6dd132505 100644 --- a/en/docs/next/ai-workspace/ai-gateways/setting-up.md +++ b/en/docs/ai-workspace/next/ai-gateways/setting-up.md @@ -1,8 +1,8 @@ --- title: "Set up an AI Gateway in AI Workspace" description: "Register an AI Gateway in AI Workspace, connect the runtime with a registration token, and manage existing gateways." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/ai-gateways/setting-up/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/ai-gateways/setting-up.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/ai-gateways/setting-up/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/ai-gateways/setting-up.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/ci-cd/configure-ci-cd-workflow.md b/en/docs/ai-workspace/next/ci-cd/configure-ci-cd-workflow.md similarity index 98% rename from en/docs/next/ai-workspace/ci-cd/configure-ci-cd-workflow.md rename to en/docs/ai-workspace/next/ci-cd/configure-ci-cd-workflow.md index c73d7d33e..01eec7cf0 100644 --- a/en/docs/next/ai-workspace/ci-cd/configure-ci-cd-workflow.md +++ b/en/docs/ai-workspace/next/ci-cd/configure-ci-cd-workflow.md @@ -1,8 +1,8 @@ --- title: "Configure an AI Workspace CI/CD workflow" description: "Use the ap CLI to validate and apply AI Workspace artifacts such as LLM providers, App LLM proxies, and MCP proxies from project files." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/ci-cd/configure-ci-cd-workflow/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/ci-cd/configure-ci-cd-workflow.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/ci-cd/configure-ci-cd-workflow/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/ci-cd/configure-ci-cd-workflow.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/ci-cd/overview.md b/en/docs/ai-workspace/next/ci-cd/overview.md similarity index 97% rename from en/docs/next/ai-workspace/ci-cd/overview.md rename to en/docs/ai-workspace/next/ci-cd/overview.md index 44885ad84..b9e9b3c82 100644 --- a/en/docs/next/ai-workspace/ci-cd/overview.md +++ b/en/docs/ai-workspace/next/ci-cd/overview.md @@ -1,8 +1,8 @@ --- title: "AI Workspace CI/CD overview" description: "Manage AI Workspace artifacts such as LLM providers, App LLM proxies, and MCP proxies through a Git-based CI/CD workflow." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/ci-cd/overview/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/ci-cd/overview.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/ci-cd/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/ci-cd/overview.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/configure-inbound-auth.md b/en/docs/ai-workspace/next/configure-inbound-auth.md similarity index 94% rename from en/docs/next/ai-workspace/configure-inbound-auth.md rename to en/docs/ai-workspace/next/configure-inbound-auth.md index 263479629..3be729771 100644 --- a/en/docs/next/ai-workspace/configure-inbound-auth.md +++ b/en/docs/ai-workspace/next/configure-inbound-auth.md @@ -1,8 +1,8 @@ --- title: "Configure inbound authentication" description: "Control the header name client applications use to send their API key when calling a deployed LLM provider or App LLM proxy." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/configure-inbound-auth/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/configure-inbound-auth.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/configure-inbound-auth/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/configure-inbound-auth.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/genai-applications.md b/en/docs/ai-workspace/next/genai-applications.md similarity index 97% rename from en/docs/next/ai-workspace/genai-applications.md rename to en/docs/ai-workspace/next/genai-applications.md index d17ce3b76..c87d996d0 100644 --- a/en/docs/next/ai-workspace/genai-applications.md +++ b/en/docs/ai-workspace/next/genai-applications.md @@ -1,8 +1,8 @@ --- title: "GenAI applications" description: "Group API keys under a named GenAI application for application-level usage visibility, analytics, and governance." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/genai-applications/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/genai-applications.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/genai-applications/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/genai-applications.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/getting-started.md b/en/docs/ai-workspace/next/getting-started.md similarity index 97% rename from en/docs/next/ai-workspace/getting-started.md rename to en/docs/ai-workspace/next/getting-started.md index aeaffb0f8..4120e3f7e 100644 --- a/en/docs/next/ai-workspace/getting-started.md +++ b/en/docs/ai-workspace/next/getting-started.md @@ -1,8 +1,8 @@ --- title: "Get started with AI Workspace" description: "Run AI Workspace locally with Docker Compose, create an AI Gateway, configure an LLM provider, and deploy it through the AI Workspace control plane." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/getting-started/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/getting-started.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/getting-started/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/getting-started.md tags: - cloud - ai-workspace @@ -29,8 +29,8 @@ This guide shows commands with `docker compose`. If you use Podman or another Co Run this command in your terminal to download and unzip AI Workspace: ```bash -curl -sLO https://github.com/wso2/api-platform/releases/download/portals/ai-workspace/v1.0.0-rc3/wso2apip-ai-workspace-1.0.0-rc3.zip && \ -unzip wso2apip-ai-workspace-1.0.0-rc3.zip +curl -sLO https://github.com/wso2/api-platform/releases/download/portals/ai-workspace/v1.0.0/wso2apip-ai-workspace-1.0.0.zip && \ +unzip wso2apip-ai-workspace-1.0.0.zip ``` ## Step 2: Run the setup script diff --git a/en/docs/next/ai-workspace/insights.md b/en/docs/ai-workspace/next/insights.md similarity index 96% rename from en/docs/next/ai-workspace/insights.md rename to en/docs/ai-workspace/next/insights.md index 6d606a296..9dce11283 100644 --- a/en/docs/next/ai-workspace/insights.md +++ b/en/docs/ai-workspace/next/insights.md @@ -1,8 +1,8 @@ --- title: "Insights" description: "Open your Moesif analytics workspace from the AI Workspace to view traffic, token usage, latency, and consumer behavior for your AI Gateways." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/insights/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/insights.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/insights/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/insights.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/llm-provider-templates/configure-template.md b/en/docs/ai-workspace/next/llm-provider-templates/configure-template.md similarity index 96% rename from en/docs/next/ai-workspace/llm-provider-templates/configure-template.md rename to en/docs/ai-workspace/next/llm-provider-templates/configure-template.md index 9a3671a33..240accdbd 100644 --- a/en/docs/next/ai-workspace/llm-provider-templates/configure-template.md +++ b/en/docs/ai-workspace/next/llm-provider-templates/configure-template.md @@ -1,8 +1,8 @@ --- title: "Configure an LLM provider template" description: "Create a custom LLM provider template in AI Workspace, configure its connection and token mappings, version it, and deploy it to a gateway." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-provider-templates/configure-template/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-provider-templates/configure-template.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-provider-templates/configure-template/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/llm-provider-templates/configure-template.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/llm-provider-templates/manage-template.md b/en/docs/ai-workspace/next/llm-provider-templates/manage-template.md similarity index 95% rename from en/docs/next/ai-workspace/llm-provider-templates/manage-template.md rename to en/docs/ai-workspace/next/llm-provider-templates/manage-template.md index 1260712c6..e0d2b7eec 100644 --- a/en/docs/next/ai-workspace/llm-provider-templates/manage-template.md +++ b/en/docs/ai-workspace/next/llm-provider-templates/manage-template.md @@ -1,8 +1,8 @@ --- title: "Manage an LLM provider template" description: "Use an LLM provider template to create a provider, and edit, enable/disable, or delete template versions in AI Workspace." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-provider-templates/manage-template/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-provider-templates/manage-template.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-provider-templates/manage-template/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/llm-provider-templates/manage-template.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/llm-provider-templates/overview.md b/en/docs/ai-workspace/next/llm-provider-templates/overview.md similarity index 92% rename from en/docs/next/ai-workspace/llm-provider-templates/overview.md rename to en/docs/ai-workspace/next/llm-provider-templates/overview.md index 3c469ce11..9878f0dfc 100644 --- a/en/docs/next/ai-workspace/llm-provider-templates/overview.md +++ b/en/docs/ai-workspace/next/llm-provider-templates/overview.md @@ -1,8 +1,8 @@ --- title: "LLM provider templates overview" description: "Learn how LLM provider templates capture endpoint, authentication, OpenAPI, and token mapping configuration as reusable blueprints for creating LLM providers in AI Workspace." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-provider-templates/overview/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-provider-templates/overview.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-provider-templates/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/llm-provider-templates/overview.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/llm-providers/configure-provider.md b/en/docs/ai-workspace/next/llm-providers/configure-provider.md similarity index 98% rename from en/docs/next/ai-workspace/llm-providers/configure-provider.md rename to en/docs/ai-workspace/next/llm-providers/configure-provider.md index e4f6de1fa..27957f5aa 100644 --- a/en/docs/next/ai-workspace/llm-providers/configure-provider.md +++ b/en/docs/ai-workspace/next/llm-providers/configure-provider.md @@ -1,8 +1,8 @@ --- title: "Configure an LLM provider" description: "Add an LLM provider in AI Workspace, configure authentication and guardrails, and deploy it to an AI Gateway." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-providers/configure-provider/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-providers/configure-provider.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-providers/configure-provider/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/llm-providers/configure-provider.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/llm-providers/manage-provider.md b/en/docs/ai-workspace/next/llm-providers/manage-provider.md similarity index 98% rename from en/docs/next/ai-workspace/llm-providers/manage-provider.md rename to en/docs/ai-workspace/next/llm-providers/manage-provider.md index 7429b1a08..56c766f1b 100644 --- a/en/docs/next/ai-workspace/llm-providers/manage-provider.md +++ b/en/docs/ai-workspace/next/llm-providers/manage-provider.md @@ -1,8 +1,8 @@ --- title: "Manage an LLM provider" description: "Configure connection, access control, security, rate limiting, guardrails, and models for a deployed LLM provider, then redeploy or delete it." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-providers/manage-provider/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-providers/manage-provider.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-providers/manage-provider/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/llm-providers/manage-provider.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/llm-providers/overview.md b/en/docs/ai-workspace/next/llm-providers/overview.md similarity index 95% rename from en/docs/next/ai-workspace/llm-providers/overview.md rename to en/docs/ai-workspace/next/llm-providers/overview.md index 86d296bb9..3b16641d3 100644 --- a/en/docs/next/ai-workspace/llm-providers/overview.md +++ b/en/docs/ai-workspace/next/llm-providers/overview.md @@ -1,8 +1,8 @@ --- title: "LLM providers overview" description: "Connect AI service platforms such as OpenAI, Anthropic, Azure OpenAI, Gemini, and Mistral AI as reusable LLM providers in AI Workspace." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-providers/overview/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-providers/overview.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-providers/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/llm-providers/overview.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/llm-proxies/configure-proxy.md b/en/docs/ai-workspace/next/llm-proxies/configure-proxy.md similarity index 97% rename from en/docs/next/ai-workspace/llm-proxies/configure-proxy.md rename to en/docs/ai-workspace/next/llm-proxies/configure-proxy.md index 516124fe9..e39046274 100644 --- a/en/docs/next/ai-workspace/llm-proxies/configure-proxy.md +++ b/en/docs/ai-workspace/next/llm-proxies/configure-proxy.md @@ -1,8 +1,8 @@ --- title: "Configure an App LLM proxy" description: "Create an App LLM proxy, optionally attach guardrails, and deploy it to a gateway." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-proxies/configure-proxy/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-proxies/configure-proxy.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-proxies/configure-proxy/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/llm-proxies/configure-proxy.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/llm-proxies/manage-proxy.md b/en/docs/ai-workspace/next/llm-proxies/manage-proxy.md similarity index 97% rename from en/docs/next/ai-workspace/llm-proxies/manage-proxy.md rename to en/docs/ai-workspace/next/llm-proxies/manage-proxy.md index 10a76421d..93b8b3110 100644 --- a/en/docs/next/ai-workspace/llm-proxies/manage-proxy.md +++ b/en/docs/ai-workspace/next/llm-proxies/manage-proxy.md @@ -1,8 +1,8 @@ --- title: "Manage an App LLM proxy" description: "Configure provider settings, resources, security, and guardrails for a deployed App LLM proxy, then save, redeploy, or delete it." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-proxies/manage-proxy/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-proxies/manage-proxy.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-proxies/manage-proxy/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/llm-proxies/manage-proxy.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/llm-proxies/overview.md b/en/docs/ai-workspace/next/llm-proxies/overview.md similarity index 95% rename from en/docs/next/ai-workspace/llm-proxies/overview.md rename to en/docs/ai-workspace/next/llm-proxies/overview.md index fa517b188..cf9c16bb4 100644 --- a/en/docs/next/ai-workspace/llm-proxies/overview.md +++ b/en/docs/ai-workspace/next/llm-proxies/overview.md @@ -1,8 +1,8 @@ --- title: "App LLM proxies overview" description: "Add an application-facing endpoint on top of an LLM provider for app- or agent-specific authentication, guardrails, and access controls." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-proxies/overview/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/llm-proxies/overview.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-proxies/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/llm-proxies/overview.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/mcp-proxies/apply-policies.md b/en/docs/ai-workspace/next/mcp-proxies/apply-policies.md similarity index 93% rename from en/docs/next/ai-workspace/mcp-proxies/apply-policies.md rename to en/docs/ai-workspace/next/mcp-proxies/apply-policies.md index e258d5560..89550f2bb 100644 --- a/en/docs/next/ai-workspace/mcp-proxies/apply-policies.md +++ b/en/docs/ai-workspace/next/mcp-proxies/apply-policies.md @@ -1,8 +1,8 @@ --- title: "Apply policies to an MCP proxy" description: "Apply access control, authorization, and rewrite policies to MCP proxies using the Policy Hub." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/mcp-proxies/apply-policies/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/mcp-proxies/apply-policies.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/mcp-proxies/apply-policies/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/mcp-proxies/apply-policies.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/mcp-proxies/configure-proxy.md b/en/docs/ai-workspace/next/mcp-proxies/configure-proxy.md similarity index 96% rename from en/docs/next/ai-workspace/mcp-proxies/configure-proxy.md rename to en/docs/ai-workspace/next/mcp-proxies/configure-proxy.md index a7290abf4..3d97c977b 100644 --- a/en/docs/next/ai-workspace/mcp-proxies/configure-proxy.md +++ b/en/docs/ai-workspace/next/mcp-proxies/configure-proxy.md @@ -1,8 +1,8 @@ --- title: "Configure an MCP proxy" description: "Create an MCP proxy from an upstream MCP server URL and deploy it to a gateway." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/mcp-proxies/configure-proxy/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/mcp-proxies/configure-proxy.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/mcp-proxies/configure-proxy/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/mcp-proxies/configure-proxy.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/mcp-proxies/overview.md b/en/docs/ai-workspace/next/mcp-proxies/overview.md similarity index 93% rename from en/docs/next/ai-workspace/mcp-proxies/overview.md rename to en/docs/ai-workspace/next/mcp-proxies/overview.md index ceaef52ae..7aff196a5 100644 --- a/en/docs/next/ai-workspace/mcp-proxies/overview.md +++ b/en/docs/ai-workspace/next/mcp-proxies/overview.md @@ -1,8 +1,8 @@ --- title: "MCP proxies overview" description: "Connect the AI Gateway to upstream MCP servers and apply security, policy, and observability controls through an MCP proxy." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/mcp-proxies/overview/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/mcp-proxies/overview.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/mcp-proxies/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/mcp-proxies/overview.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/overview.md b/en/docs/ai-workspace/next/overview.md similarity index 98% rename from en/docs/next/ai-workspace/overview.md rename to en/docs/ai-workspace/next/overview.md index 6b1a24e5c..8029cbc74 100644 --- a/en/docs/next/ai-workspace/overview.md +++ b/en/docs/ai-workspace/next/overview.md @@ -1,8 +1,8 @@ --- title: "AI Workspace overview" description: "Centrally manage AI gateways, LLM providers, proxies, policies, and secrets from a single control plane." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/overview/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/overview.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/overview.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/policies/apply-ai-policies-to-proxies.md b/en/docs/ai-workspace/next/policies/apply-ai-policies-to-proxies.md similarity index 97% rename from en/docs/next/ai-workspace/policies/apply-ai-policies-to-proxies.md rename to en/docs/ai-workspace/next/policies/apply-ai-policies-to-proxies.md index bc4e3b275..842d49412 100644 --- a/en/docs/next/ai-workspace/policies/apply-ai-policies-to-proxies.md +++ b/en/docs/ai-workspace/next/policies/apply-ai-policies-to-proxies.md @@ -1,8 +1,8 @@ --- title: "Apply AI policies to LLM providers, App LLM proxies, and MCP proxies" description: "Sync a custom AI policy from your AI Gateway to your organization and apply it to LLM providers, App LLM proxies, and MCP proxies." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/policies/apply-ai-policies-to-proxies/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/policies/apply-ai-policies-to-proxies.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/apply-ai-policies-to-proxies/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/policies/apply-ai-policies-to-proxies.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/policies/build-gateway-with-ai-policies.md b/en/docs/ai-workspace/next/policies/build-gateway-with-ai-policies.md similarity index 97% rename from en/docs/next/ai-workspace/policies/build-gateway-with-ai-policies.md rename to en/docs/ai-workspace/next/policies/build-gateway-with-ai-policies.md index f6bb56fbb..688464200 100644 --- a/en/docs/next/ai-workspace/policies/build-gateway-with-ai-policies.md +++ b/en/docs/ai-workspace/next/policies/build-gateway-with-ai-policies.md @@ -1,8 +1,8 @@ --- title: "Build the AI Gateway with custom AI policies" description: "Install the AP CLI, configure build.yaml, and build a custom AI Gateway image that includes your custom AI policies." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/policies/build-gateway-with-ai-policies/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/policies/build-gateway-with-ai-policies.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/build-gateway-with-ai-policies/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/policies/build-gateway-with-ai-policies.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/policies/overview.md b/en/docs/ai-workspace/next/policies/overview.md similarity index 99% rename from en/docs/next/ai-workspace/policies/overview.md rename to en/docs/ai-workspace/next/policies/overview.md index a675c2a1e..ea71300de 100644 --- a/en/docs/next/ai-workspace/policies/overview.md +++ b/en/docs/ai-workspace/next/policies/overview.md @@ -1,8 +1,8 @@ --- title: "AI Workspace policies overview" description: "What AI and MCP policies you can attach to LLM providers and proxies in AI Workspace, where each one is applied, and how the rate limit policies cap requests, tokens, and spend." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/policies/overview/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/policies/overview.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/policies/overview.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/policies/writing-an-ai-policy.md b/en/docs/ai-workspace/next/policies/writing-an-ai-policy.md similarity index 98% rename from en/docs/next/ai-workspace/policies/writing-an-ai-policy.md rename to en/docs/ai-workspace/next/policies/writing-an-ai-policy.md index 1685ab9aa..79c2679bd 100644 --- a/en/docs/next/ai-workspace/policies/writing-an-ai-policy.md +++ b/en/docs/ai-workspace/next/policies/writing-an-ai-policy.md @@ -1,8 +1,8 @@ --- title: "Write an AI policy for the AI Gateway" description: "Build a custom AI policy using the gateway SDK, including support for buffered and streaming (SSE) LLM request and response bodies." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/policies/writing-an-ai-policy/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/policies/writing-an-ai-policy.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/writing-an-ai-policy/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/policies/writing-an-ai-policy.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/secrets-management.md b/en/docs/ai-workspace/next/secrets-management.md similarity index 98% rename from en/docs/next/ai-workspace/secrets-management.md rename to en/docs/ai-workspace/next/secrets-management.md index ed5576f05..b1de567a1 100644 --- a/en/docs/next/ai-workspace/secrets-management.md +++ b/en/docs/ai-workspace/next/secrets-management.md @@ -1,8 +1,8 @@ --- title: "Secrets management" description: "Store and manage encrypted secrets in AI Workspace and reference them securely in artifact configurations without exposing plaintext credentials." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/secrets-management/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/secrets-management.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/secrets-management/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/secrets-management.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/setting-up/authentication/asgardeo-setup.md b/en/docs/ai-workspace/next/setting-up/authentication/asgardeo-setup.md similarity index 98% rename from en/docs/next/ai-workspace/setting-up/authentication/asgardeo-setup.md rename to en/docs/ai-workspace/next/setting-up/authentication/asgardeo-setup.md index 38ee4a434..6fc9e59f9 100644 --- a/en/docs/next/ai-workspace/setting-up/authentication/asgardeo-setup.md +++ b/en/docs/ai-workspace/next/setting-up/authentication/asgardeo-setup.md @@ -1,8 +1,8 @@ --- title: "Set up Asgardeo as your identity provider" description: "Configure Asgardeo as the identity provider for a production AI Workspace deployment, from application registration to Platform API configuration." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/authentication/asgardeo-setup/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/authentication/asgardeo-setup.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/asgardeo-setup/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/setting-up/authentication/asgardeo-setup.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/setting-up/authentication/connect-an-identity-provider.md b/en/docs/ai-workspace/next/setting-up/authentication/connect-an-identity-provider.md similarity index 98% rename from en/docs/next/ai-workspace/setting-up/authentication/connect-an-identity-provider.md rename to en/docs/ai-workspace/next/setting-up/authentication/connect-an-identity-provider.md index 3a44b9af5..01d158ba6 100644 --- a/en/docs/next/ai-workspace/setting-up/authentication/connect-an-identity-provider.md +++ b/en/docs/ai-workspace/next/setting-up/authentication/connect-an-identity-provider.md @@ -1,8 +1,8 @@ --- title: "Connect an identity provider to AI Workspace" description: "Configure AI Workspace and the Platform API to delegate login to any OIDC-compliant identity provider: client registration, claim mappings, scope or role authorization, and the config.toml tables both services read." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/authentication/connect-an-identity-provider/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/authentication/connect-an-identity-provider.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/connect-an-identity-provider/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/setting-up/authentication/connect-an-identity-provider.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/setting-up/authentication/entra-id-setup.md b/en/docs/ai-workspace/next/setting-up/authentication/entra-id-setup.md similarity index 98% rename from en/docs/next/ai-workspace/setting-up/authentication/entra-id-setup.md rename to en/docs/ai-workspace/next/setting-up/authentication/entra-id-setup.md index 3249b7b1c..5ffb34077 100644 --- a/en/docs/next/ai-workspace/setting-up/authentication/entra-id-setup.md +++ b/en/docs/ai-workspace/next/setting-up/authentication/entra-id-setup.md @@ -1,8 +1,8 @@ --- title: "Set up Microsoft Entra ID as your identity provider" description: "Configure Microsoft Entra ID for a production AI Workspace deployment: application registration, app roles, and the config.toml settings both services read." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/authentication/entra-id-setup/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/authentication/entra-id-setup.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/entra-id-setup/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/setting-up/authentication/entra-id-setup.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/setting-up/authentication/overview.md b/en/docs/ai-workspace/next/setting-up/authentication/overview.md similarity index 97% rename from en/docs/next/ai-workspace/setting-up/authentication/overview.md rename to en/docs/ai-workspace/next/setting-up/authentication/overview.md index 2b1587df8..fc11dd75d 100644 --- a/en/docs/next/ai-workspace/setting-up/authentication/overview.md +++ b/en/docs/ai-workspace/next/setting-up/authentication/overview.md @@ -1,8 +1,8 @@ --- title: "Authentication in AI Workspace" description: "Understand the two ways users sign in to AI Workspace: file-based authentication for local use and an identity provider for production." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/authentication/overview/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/authentication/overview.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/authentication/overview/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/setting-up/authentication/overview.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/setting-up/configuration.md b/en/docs/ai-workspace/next/setting-up/configuration.md similarity index 97% rename from en/docs/next/ai-workspace/setting-up/configuration.md rename to en/docs/ai-workspace/next/setting-up/configuration.md index f9afcbc35..58240ad8a 100644 --- a/en/docs/next/ai-workspace/setting-up/configuration.md +++ b/en/docs/ai-workspace/next/setting-up/configuration.md @@ -1,8 +1,8 @@ --- title: "AI Workspace configuration and environment interpolation" description: "How AI Workspace and the Platform API load their config.toml files, inject environment values and mounted files through interpolation tokens, and keep sensitive values out of the config file." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/configuration/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/configuration.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/configuration/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/setting-up/configuration.md tags: - ai-workspace - configuration diff --git a/en/docs/next/ai-workspace/setting-up/database.md b/en/docs/ai-workspace/next/setting-up/database.md similarity index 93% rename from en/docs/next/ai-workspace/setting-up/database.md rename to en/docs/ai-workspace/next/setting-up/database.md index 3bb36c122..c183038b8 100644 --- a/en/docs/next/ai-workspace/setting-up/database.md +++ b/en/docs/ai-workspace/next/setting-up/database.md @@ -1,8 +1,8 @@ --- title: "Connect a database to the Platform API" description: "Move the Platform API off its default SQLite file onto PostgreSQL or SQL Server: create the database, configure the connection, secure it with TLS, and tune the connection pool." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/database/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/database.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/database/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/setting-up/database.md tags: - cloud - ai-workspace @@ -142,7 +142,7 @@ ssl_mode = "require" For SQL Server, set `driver = "sqlserver"` and `port = 1433`. !!! warning "Never write the database password as a literal" - The {% raw %}`{{ file }}`{% endraw %} token above reads it from a mounted file, which is the right choice in production — resolution fails closed, so a missing or unreadable file aborts startup rather than connecting with an empty password. Mount the secret at that path and add the mount to the `platform-api` service in `docker-compose.yaml`. To read it from `api-platform.env` instead, swap the token for {% raw %}`'{{ env "APIP_CP_DATABASE_PASSWORD" }}'`{% endraw %}. + The {% raw %}`{{ file }}`{% endraw %} token above reads it from a mounted file, which is the right choice in production. Resolution fails closed, so a missing or unreadable file aborts startup rather than connecting with an empty password. Mount the secret at that path and add the mount to the `platform-api` service in `docker-compose.yaml`. To read it from `api-platform.env` instead, swap the token for {% raw %}`'{{ env "APIP_CP_DATABASE_PASSWORD" }}'`{% endraw %}. ## Step 4: Secure the connection with TLS diff --git a/en/docs/next/ai-workspace/setting-up/ports.md b/en/docs/ai-workspace/next/setting-up/ports.md similarity index 90% rename from en/docs/next/ai-workspace/setting-up/ports.md rename to en/docs/ai-workspace/next/setting-up/ports.md index 130ea6f4a..16a393b94 100644 --- a/en/docs/next/ai-workspace/setting-up/ports.md +++ b/en/docs/ai-workspace/next/setting-up/ports.md @@ -1,8 +1,8 @@ --- title: "Change the ports AI Workspace uses" description: "Move the AI Workspace and Platform API off their default ports, either by remapping the published host port or by changing the port each service listens on." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/ports/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/setting-up/ports.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/setting-up/ports/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/setting-up/ports.md tags: - cloud - ai-workspace @@ -112,7 +112,13 @@ Both approaches touch `url` and `controlplane_host`, which sit either side of th Nothing in AI Workspace connects to `controlplane_host`. The value is display-only: the workspace substitutes it into the gateway setup commands the **Get Started** section shows an admin, such as the `APIP_GW_CONTROLLER_CONTROLPLANE_HOST` line and the Helm `--set gateway.controller.controlPlane.host` flag. The admin then copies those commands to the machine running the gateway, which is what makes the connection. -So a wrong value leaves AI Workspace working normally and breaks the gateway instead: the printed commands look right, but the gateway they configure can't reach the control plane and never registers. Give the key an address that's reachable **from the gateway's** network — `host.docker.internal` for a gateway in another container on the same machine, or the machine's hostname or IP address from anywhere else. A gateway outside the stack can't resolve `platform-api`. +So a wrong value leaves AI Workspace working normally and breaks the gateway instead. The printed commands look right, but the gateway they configure can't reach the control plane and never registers. + +Set `controlplane_host` to an address that's reachable from the gateway's network: + +- For a gateway in another container on the same machine, use `host.docker.internal`. +- For a gateway anywhere else, use the machine's hostname or IP address. +- Don't use `platform-api`. That name resolves only inside the Compose network, so a gateway outside the stack can't reach it. ## Apply the change diff --git a/en/docs/next/ai-workspace/sync-gateway-created-artifacts.md b/en/docs/ai-workspace/next/sync-gateway-created-artifacts.md similarity index 98% rename from en/docs/next/ai-workspace/sync-gateway-created-artifacts.md rename to en/docs/ai-workspace/next/sync-gateway-created-artifacts.md index af1b13099..9fb244a6c 100644 --- a/en/docs/next/ai-workspace/sync-gateway-created-artifacts.md +++ b/en/docs/ai-workspace/next/sync-gateway-created-artifacts.md @@ -1,8 +1,8 @@ --- title: "Manage Gateway-deployed AI artifacts in AI Workspace" description: "Create an LLM provider template, LLM provider, LLM proxy, or MCP proxy on the AI Gateway and view the read-only copy that syncs up to AI Workspace." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/sync-gateway-created-artifacts/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/sync-gateway-created-artifacts.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/sync-gateway-created-artifacts/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/sync-gateway-created-artifacts.md tags: - cloud - ai-workspace diff --git a/en/docs/next/ai-workspace/using-sdks.md b/en/docs/ai-workspace/next/using-sdks.md similarity index 98% rename from en/docs/next/ai-workspace/using-sdks.md rename to en/docs/ai-workspace/next/using-sdks.md index cca5b68ff..fdd49411d 100644 --- a/en/docs/next/ai-workspace/using-sdks.md +++ b/en/docs/ai-workspace/next/using-sdks.md @@ -1,8 +1,8 @@ --- title: "Invoke providers and proxies via SDKs" description: "Call a deployed LLM provider or App LLM proxy using the OpenAI, Anthropic, Gemini, Mistral, Azure OpenAI, or LangChain SDKs." -canonical_url: https://wso2.com/api-platform/docs/next/ai-workspace/using-sdks/ -md_url: https://wso2.com/api-platform/docs/next/ai-workspace/using-sdks.md +canonical_url: https://wso2.com/api-platform/docs/ai-workspace/1.0.0/using-sdks/ +md_url: https://wso2.com/api-platform/docs/ai-workspace/next/using-sdks.md tags: - cloud - ai-workspace diff --git a/en/docs/next/index.md b/en/docs/next/index.md index 9ca46a031..5eadd466e 100644 --- a/en/docs/next/index.md +++ b/en/docs/next/index.md @@ -138,7 +138,7 @@ The following is a complete view of all components and where to find their docum | Component | What it does | Docs | | :--- | :--- | :--- | | **API Control Plane** | Design, publish, version, and govern APIs through the web UI and configuration files. Enforce policies across all connected gateways. | Part of [Cloud](../cloud/introduction/what-is-bijira.md) and [API Manager](../api-manager/overview.md) | -| **AI Workspace** | The enterprise control plane for AI. Manage LLM providers, MCP servers, and GenAI applications. Configure cost and token-based rate limits, enforce guardrails, and view AI consumption insights at the organizational level. Connected to one or more AI Gateways. | [AI Workspace docs](ai-workspace/overview.md) | +| **AI Workspace** | The enterprise control plane for AI. Manage LLM providers, MCP servers, and GenAI applications. Configure cost and token-based rate limits, enforce guardrails, and view AI consumption insights at the organizational level. Connected to one or more AI Gateways. | [AI Workspace docs](../ai-workspace/next/overview.md) | | **API Portal & MCP Hub** | Developer facing portal for API discovery, subscription management, SDK generation, and theming. Includes agentic consumption capabilities for AI agents, such as the llms.txt endpoint, MCP registry, and Arazzo workflow support. | [API Portal docs](api-portal/overview.md) | | **Analytics and Monetization** | Traffic monitoring, runtime and audit logs, usage tracking, and API monetization with usage-based billing. Powered by Moesif. |
  • [Analytics docs](../analytics/overview.md)
  • [Monetization docs](../monetization/overview.md)
| @@ -161,7 +161,7 @@ The following is a complete view of all components and where to find their docum | Govern LLM traffic (rate limits, guardrails, cost control) | [AI Gateway LLM Proxy](../ai-gateway/next/llm-proxy/quick-start-guide.md) | | Expose my APIs as MCP tools for AI agents | [AI Gateway MCP Proxy](../ai-gateway/next/mcp-proxy/quick-start-guide.md) | | Govern MCP traffic (inbound + outbound) | [AI Gateway MCP Proxy](../ai-gateway/next/mcp-proxy/quick-start-guide.md) | -| Manage LLM providers and AI policies at the organizational level | [AI Workspace](ai-workspace/overview.md) | +| Manage LLM providers and AI policies at the organizational level | [AI Workspace](../ai-workspace/next/overview.md) | | Set up an API Portal & MCP Hub for API discovery | [API Portal](api-portal/overview.md) | | Monitor traffic and monetize my APIs | [Analytics](../analytics/overview.md) and [Monetization](../monetization/overview.md) | | Follow end-to-end scenario walkthroughs | [Guides](../guides/ai-and-mcp/convert-rest-api-to-mcp-server.md) | diff --git a/en/mkdocs.yml b/en/mkdocs.yml index b3eccf065..1922a6f04 100644 --- a/en/mkdocs.yml +++ b/en/mkdocs.yml @@ -765,57 +765,110 @@ nav: - Get a Bearer Token via curl: next/api-portal/references/get-a-bearer-token-via-curl.md - Configurations: next/api-portal/references/configurations.md - AI Workspace: - - Overview: next/ai-workspace/overview.md - - About this Release: next/ai-workspace/about-this-release.md - - Getting Started: next/ai-workspace/getting-started.md - # Set up the workspace itself. - - Setting Up: - - Configuration & Interpolation: next/ai-workspace/setting-up/configuration.md - - Ports: next/ai-workspace/setting-up/ports.md - - Database: next/ai-workspace/setting-up/database.md - - Authentication: - - Overview: next/ai-workspace/setting-up/authentication/overview.md - - Connect an Identity Provider: next/ai-workspace/setting-up/authentication/connect-an-identity-provider.md - - Set up Asgardeo: next/ai-workspace/setting-up/authentication/asgardeo-setup.md - - Set up Microsoft Entra ID: next/ai-workspace/setting-up/authentication/entra-id-setup.md - # Build the artifacts, in the order you create them. - - AI Gateways: - - Setting Up: next/ai-workspace/ai-gateways/setting-up.md - - LLM Providers: - - Overview: next/ai-workspace/llm-providers/overview.md - - Configure Provider: next/ai-workspace/llm-providers/configure-provider.md - - Manage Provider: next/ai-workspace/llm-providers/manage-provider.md - - LLM Provider Templates: - - Overview: next/ai-workspace/llm-provider-templates/overview.md - - Create a Custom Template: next/ai-workspace/llm-provider-templates/configure-template.md - - Manage Templates: next/ai-workspace/llm-provider-templates/manage-template.md - - App LLM Proxies: - - Overview: next/ai-workspace/llm-proxies/overview.md - - Configure App LLM Proxy: next/ai-workspace/llm-proxies/configure-proxy.md - - Manage App LLM Proxy: next/ai-workspace/llm-proxies/manage-proxy.md - - MCP Proxies: - - Overview: next/ai-workspace/mcp-proxies/overview.md - - Configure Proxy: next/ai-workspace/mcp-proxies/configure-proxy.md - - Apply Policies: next/ai-workspace/mcp-proxies/apply-policies.md - # Credentials the artifacts above reference. - - Secrets Management: next/ai-workspace/secrets-management.md - # The other two ways artifacts get created. - - Manage Gateway-deployed AI artifacts: next/ai-workspace/sync-gateway-created-artifacts.md - - CI/CD: - - Overview: next/ai-workspace/ci-cd/overview.md - - Configure CI/CD Workflow: next/ai-workspace/ci-cd/configure-ci-cd-workflow.md - # Govern what you built. - - Policies: - - Overview: next/ai-workspace/policies/overview.md - - Custom Policies: - - Writing an AI Policy: next/ai-workspace/policies/writing-an-ai-policy.md - - Building the Gateway with AI Policies: next/ai-workspace/policies/build-gateway-with-ai-policies.md - - Apply AI Policies to Proxies: next/ai-workspace/policies/apply-ai-policies-to-proxies.md - - Insights: next/ai-workspace/insights.md - # Consume it from an application. - - GenAI Applications: next/ai-workspace/genai-applications.md - - Configure Inbound Auth: next/ai-workspace/configure-inbound-auth.md - - Invoke via SDKs: next/ai-workspace/using-sdks.md + - "next": + - Overview: ai-workspace/next/overview.md + - About this Release: ai-workspace/next/about-this-release.md + - Getting Started: ai-workspace/next/getting-started.md + # Set up the workspace itself. + - Setting Up: + - Configuration & Interpolation: ai-workspace/next/setting-up/configuration.md + - Ports: ai-workspace/next/setting-up/ports.md + - Database: ai-workspace/next/setting-up/database.md + - Authentication: + - Overview: ai-workspace/next/setting-up/authentication/overview.md + - Connect an Identity Provider: ai-workspace/next/setting-up/authentication/connect-an-identity-provider.md + - Set up Asgardeo: ai-workspace/next/setting-up/authentication/asgardeo-setup.md + - Set up Microsoft Entra ID: ai-workspace/next/setting-up/authentication/entra-id-setup.md + # Build the artifacts, in the order you create them. + - AI Gateways: + - Setting Up: ai-workspace/next/ai-gateways/setting-up.md + - LLM Providers: + - Overview: ai-workspace/next/llm-providers/overview.md + - Configure Provider: ai-workspace/next/llm-providers/configure-provider.md + - Manage Provider: ai-workspace/next/llm-providers/manage-provider.md + - LLM Provider Templates: + - Overview: ai-workspace/next/llm-provider-templates/overview.md + - Create a Custom Template: ai-workspace/next/llm-provider-templates/configure-template.md + - Manage Templates: ai-workspace/next/llm-provider-templates/manage-template.md + - App LLM Proxies: + - Overview: ai-workspace/next/llm-proxies/overview.md + - Configure App LLM Proxy: ai-workspace/next/llm-proxies/configure-proxy.md + - Manage App LLM Proxy: ai-workspace/next/llm-proxies/manage-proxy.md + - MCP Proxies: + - Overview: ai-workspace/next/mcp-proxies/overview.md + - Configure Proxy: ai-workspace/next/mcp-proxies/configure-proxy.md + - Apply Policies: ai-workspace/next/mcp-proxies/apply-policies.md + # Credentials the artifacts above reference. + - Secrets Management: ai-workspace/next/secrets-management.md + # The other two ways artifacts get created. + - Manage Gateway-deployed AI artifacts: ai-workspace/next/sync-gateway-created-artifacts.md + - CI/CD: + - Overview: ai-workspace/next/ci-cd/overview.md + - Configure CI/CD Workflow: ai-workspace/next/ci-cd/configure-ci-cd-workflow.md + # Govern what you built. + - Policies: + - Overview: ai-workspace/next/policies/overview.md + - Custom Policies: + - Writing an AI Policy: ai-workspace/next/policies/writing-an-ai-policy.md + - Building the Gateway with AI Policies: ai-workspace/next/policies/build-gateway-with-ai-policies.md + - Apply AI Policies to Proxies: ai-workspace/next/policies/apply-ai-policies-to-proxies.md + - Insights: ai-workspace/next/insights.md + # Consume it from an application. + - GenAI Applications: ai-workspace/next/genai-applications.md + - Configure Inbound Auth: ai-workspace/next/configure-inbound-auth.md + - Invoke via SDKs: ai-workspace/next/using-sdks.md + - "1.0.0": + - Overview: ai-workspace/1.0.0/overview.md + - About this Release: ai-workspace/1.0.0/about-this-release.md + - Getting Started: ai-workspace/1.0.0/getting-started.md + # Set up the workspace itself. + - Setting Up: + - Configuration & Interpolation: ai-workspace/1.0.0/setting-up/configuration.md + - Ports: ai-workspace/1.0.0/setting-up/ports.md + - Database: ai-workspace/1.0.0/setting-up/database.md + - Authentication: + - Overview: ai-workspace/1.0.0/setting-up/authentication/overview.md + - Connect an Identity Provider: ai-workspace/1.0.0/setting-up/authentication/connect-an-identity-provider.md + - Set up Asgardeo: ai-workspace/1.0.0/setting-up/authentication/asgardeo-setup.md + - Set up Microsoft Entra ID: ai-workspace/1.0.0/setting-up/authentication/entra-id-setup.md + # Build the artifacts, in the order you create them. + - AI Gateways: + - Setting Up: ai-workspace/1.0.0/ai-gateways/setting-up.md + - LLM Providers: + - Overview: ai-workspace/1.0.0/llm-providers/overview.md + - Configure Provider: ai-workspace/1.0.0/llm-providers/configure-provider.md + - Manage Provider: ai-workspace/1.0.0/llm-providers/manage-provider.md + - LLM Provider Templates: + - Overview: ai-workspace/1.0.0/llm-provider-templates/overview.md + - Create a Custom Template: ai-workspace/1.0.0/llm-provider-templates/configure-template.md + - Manage Templates: ai-workspace/1.0.0/llm-provider-templates/manage-template.md + - App LLM Proxies: + - Overview: ai-workspace/1.0.0/llm-proxies/overview.md + - Configure App LLM Proxy: ai-workspace/1.0.0/llm-proxies/configure-proxy.md + - Manage App LLM Proxy: ai-workspace/1.0.0/llm-proxies/manage-proxy.md + - MCP Proxies: + - Overview: ai-workspace/1.0.0/mcp-proxies/overview.md + - Configure Proxy: ai-workspace/1.0.0/mcp-proxies/configure-proxy.md + - Apply Policies: ai-workspace/1.0.0/mcp-proxies/apply-policies.md + # Credentials the artifacts above reference. + - Secrets Management: ai-workspace/1.0.0/secrets-management.md + # The other two ways artifacts get created. + - Manage gateway-deployed AI artifacts: ai-workspace/1.0.0/sync-gateway-created-artifacts.md + - CI/CD: + - Overview: ai-workspace/1.0.0/ci-cd/overview.md + - Configure CI/CD Workflow: ai-workspace/1.0.0/ci-cd/configure-ci-cd-workflow.md + # Govern what you built. + - Policies: + - Overview: ai-workspace/1.0.0/policies/overview.md + - Custom Policies: + - Writing an AI Policy: ai-workspace/1.0.0/policies/writing-an-ai-policy.md + - Building the Gateway with AI Policies: ai-workspace/1.0.0/policies/build-gateway-with-ai-policies.md + - Apply AI Policies to Proxies: ai-workspace/1.0.0/policies/apply-ai-policies-to-proxies.md + - Insights: ai-workspace/1.0.0/insights.md + # Consume it from an application. + - GenAI Applications: ai-workspace/1.0.0/genai-applications.md + - Configure Inbound Auth: ai-workspace/1.0.0/configure-inbound-auth.md + - Invoke via SDKs: ai-workspace/1.0.0/using-sdks.md - Analytics: - Overview: analytics/overview.md - Monetization: @@ -989,6 +1042,44 @@ plugins: api-gateway/setup/artifact-templating.md: api-gateway/1.1.0/setup/artifact-templating.md api-gateway/setup/storage-and-backends.md: api-gateway/1.1.0/setup/storage-and-backends.md api-gateway/setup/upstream-timeouts.md: api-gateway/1.1.0/setup/upstream-timeouts.md + # AI Workspace self-hosted docs moved from next/ai-workspace to the + # versioned ai-workspace/ layout; 1.0.0 is the latest release. + next/ai-workspace/index.md: ai-workspace/1.0.0/overview.md + next/ai-workspace/about-this-release.md: ai-workspace/1.0.0/about-this-release.md + next/ai-workspace/ai-gateways/setting-up.md: ai-workspace/1.0.0/ai-gateways/setting-up.md + next/ai-workspace/ci-cd/configure-ci-cd-workflow.md: ai-workspace/1.0.0/ci-cd/configure-ci-cd-workflow.md + next/ai-workspace/ci-cd/overview.md: ai-workspace/1.0.0/ci-cd/overview.md + next/ai-workspace/configure-inbound-auth.md: ai-workspace/1.0.0/configure-inbound-auth.md + next/ai-workspace/genai-applications.md: ai-workspace/1.0.0/genai-applications.md + next/ai-workspace/getting-started.md: ai-workspace/1.0.0/getting-started.md + next/ai-workspace/insights.md: ai-workspace/1.0.0/insights.md + next/ai-workspace/llm-provider-templates/configure-template.md: ai-workspace/1.0.0/llm-provider-templates/configure-template.md + next/ai-workspace/llm-provider-templates/manage-template.md: ai-workspace/1.0.0/llm-provider-templates/manage-template.md + next/ai-workspace/llm-provider-templates/overview.md: ai-workspace/1.0.0/llm-provider-templates/overview.md + next/ai-workspace/llm-providers/configure-provider.md: ai-workspace/1.0.0/llm-providers/configure-provider.md + next/ai-workspace/llm-providers/manage-provider.md: ai-workspace/1.0.0/llm-providers/manage-provider.md + next/ai-workspace/llm-providers/overview.md: ai-workspace/1.0.0/llm-providers/overview.md + next/ai-workspace/llm-proxies/configure-proxy.md: ai-workspace/1.0.0/llm-proxies/configure-proxy.md + next/ai-workspace/llm-proxies/manage-proxy.md: ai-workspace/1.0.0/llm-proxies/manage-proxy.md + next/ai-workspace/llm-proxies/overview.md: ai-workspace/1.0.0/llm-proxies/overview.md + next/ai-workspace/mcp-proxies/apply-policies.md: ai-workspace/1.0.0/mcp-proxies/apply-policies.md + next/ai-workspace/mcp-proxies/configure-proxy.md: ai-workspace/1.0.0/mcp-proxies/configure-proxy.md + next/ai-workspace/mcp-proxies/overview.md: ai-workspace/1.0.0/mcp-proxies/overview.md + next/ai-workspace/overview.md: ai-workspace/1.0.0/overview.md + next/ai-workspace/policies/apply-ai-policies-to-proxies.md: ai-workspace/1.0.0/policies/apply-ai-policies-to-proxies.md + next/ai-workspace/policies/build-gateway-with-ai-policies.md: ai-workspace/1.0.0/policies/build-gateway-with-ai-policies.md + next/ai-workspace/policies/overview.md: ai-workspace/1.0.0/policies/overview.md + next/ai-workspace/policies/writing-an-ai-policy.md: ai-workspace/1.0.0/policies/writing-an-ai-policy.md + next/ai-workspace/secrets-management.md: ai-workspace/1.0.0/secrets-management.md + next/ai-workspace/setting-up/authentication/asgardeo-setup.md: ai-workspace/1.0.0/setting-up/authentication/asgardeo-setup.md + next/ai-workspace/setting-up/authentication/connect-an-identity-provider.md: ai-workspace/1.0.0/setting-up/authentication/connect-an-identity-provider.md + next/ai-workspace/setting-up/authentication/entra-id-setup.md: ai-workspace/1.0.0/setting-up/authentication/entra-id-setup.md + next/ai-workspace/setting-up/authentication/overview.md: ai-workspace/1.0.0/setting-up/authentication/overview.md + next/ai-workspace/setting-up/configuration.md: ai-workspace/1.0.0/setting-up/configuration.md + next/ai-workspace/setting-up/database.md: ai-workspace/1.0.0/setting-up/database.md + next/ai-workspace/setting-up/ports.md: ai-workspace/1.0.0/setting-up/ports.md + next/ai-workspace/sync-gateway-created-artifacts.md: ai-workspace/1.0.0/sync-gateway-created-artifacts.md + next/ai-workspace/using-sdks.md: ai-workspace/1.0.0/using-sdks.md # Administer administer/configure-an-external-idp/configure-asgardeo-as-an-external-idp.md: cloud/administer/configure-an-external-idp/configure-asgardeo-as-an-external-idp.md administer/configure-an-external-idp/configure-azure-ad-as-an-external-idp.md: cloud/administer/configure-an-external-idp/configure-azure-ad-as-an-external-idp.md @@ -1035,7 +1126,7 @@ plugins: cloud/ai-gateway/llm/ai-tools/codex-configuration-with-ai-gateway.md: guides/ai-and-mcp/ai-coding-assistants/codex-configuration-with-ai-gateway.md # AI Workspace policy pages consolidated into the AI Gateway policy reference. # AI Workspace keeps only the overview plus rate limiting, which has no AI Gateway equivalent. - next/ai-workspace/policies/guardrails/overview.md: next/ai-workspace/policies/overview.md + next/ai-workspace/policies/guardrails/overview.md: ai-workspace/1.0.0/policies/overview.md next/ai-workspace/policies/guardrails/semantic-prompt-guard.md: cloud/ai-gateway/llm/guardrails/semantic-prompt-guard.md next/ai-workspace/policies/guardrails/regex-pii-masking.md: cloud/ai-gateway/llm/guardrails/pii-masking-regex.md next/ai-workspace/policies/guardrails/azure-content-safety.md: cloud/ai-gateway/llm/guardrails/azure-content-safety.md @@ -1047,12 +1138,12 @@ plugins: next/ai-workspace/policies/other-policies/semantic-cache.md: cloud/ai-gateway/llm/semantic-caching.md # The rate limit policy pages merged into the Policies overview page. - next/ai-workspace/policies/rate-limit/llm-cost.md: next/ai-workspace/policies/overview.md - next/ai-workspace/policies/rate-limit/token-based-rate-limit.md: next/ai-workspace/policies/overview.md - next/ai-workspace/policies/rate-limit/llm-cost-based-rate-limit.md: next/ai-workspace/policies/overview.md - next/ai-workspace/policies/other-policies/basic-rate-limit.md: next/ai-workspace/policies/overview.md - next/ai-workspace/policies/other-policies/token-based-rate-limit.md: next/ai-workspace/policies/overview.md - next/ai-workspace/policies/rate-limiting.md: next/ai-workspace/policies/overview.md + next/ai-workspace/policies/rate-limit/llm-cost.md: ai-workspace/1.0.0/policies/overview.md + next/ai-workspace/policies/rate-limit/token-based-rate-limit.md: ai-workspace/1.0.0/policies/overview.md + next/ai-workspace/policies/rate-limit/llm-cost-based-rate-limit.md: ai-workspace/1.0.0/policies/overview.md + next/ai-workspace/policies/other-policies/basic-rate-limit.md: ai-workspace/1.0.0/policies/overview.md + next/ai-workspace/policies/other-policies/token-based-rate-limit.md: ai-workspace/1.0.0/policies/overview.md + next/ai-workspace/policies/rate-limiting.md: ai-workspace/1.0.0/policies/overview.md # AI Workspace ai-workspace/ai-gateways/setting-up.md: cloud/ai-workspace/ai-gateways/setting-up.md @@ -1266,15 +1357,18 @@ extra: versions: - "1.1.0" - "1.0.0" + AI Workspace: + slug: ai-workspace + default: "1.0.0" + versions: + - "1.0.0" # Top-level nav sections that only apply to a given doc version. They are # rendered hidden; theme.js reveals them while the current URL contains the - # version as a path segment (e.g. /next/ai-workspace/... or - # /api-gateway/next/...). See partials/nav-item.html and theme.js. + # version as a path segment (e.g. /next/api-portal/...). See + # partials/nav-item.html and theme.js. version_scoped_navs: - title: API Portal & MCP Hub version: "next" - - title: AI Workspace - version: "next" expanded_navs: - title: Cloud options: @@ -1301,12 +1395,13 @@ extra: - title: API Portal & MCP Hub options: - verticle-line - # Root-level (version-scoped) copy only; 'level: 1' keeps the AI Workspace + # Root-level (versioned) copy only; 'level: 1' keeps the AI Workspace # section nested under Cloud rendering as a regular collapsed item. - title: AI Workspace level: 1 options: - verticle-line + - divider - title: API Manager options: - verticle-line