[docs] tests for quickstarts

.github/workflows/doc-tests.yaml

@@ -19,9 +19,26 @@ jobs:
         with:
           python-version: '3.x'
 
+      - name: Set up Go
+        uses: actions/setup-go@v5
+        with:
+          go-version: 'stable'
+          cache: false
+
+      - name: Set up Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: '20'
+
       - name: Install Python dependencies
         run: pip install pyyaml
 
+      - name: Install cloud-provider-kind
+        run: go install sigs.k8s.io/cloud-provider-kind@latest
+
+      - name: Install yamltest
+        run: npm install -g yamltest@latest
+
       - name: Generate test scripts
         run: python3 scripts/doc_test_run.py --repo-root . --generate-only
 

assets/agw-docs/standalone/quickstart/llm.md

@@ -4,7 +4,7 @@ Configure the agentgateway binary to route requests to the [OpenAI](https://open
 
 1. [Install the agentgateway binary]({{< link-hextra path="/deployment/binary" >}}).
 
-   ```sh
+   ```sh,paths="llm"
    curl -sL https://agentgateway.dev/install | bash
    ```
 
@@ -20,15 +20,15 @@ Route to an OpenAI backend through agentgateway.
 
 Store your OpenAI API key in an environment variable so agentgateway can authenticate to the API.
 
-```sh
-export OPENAI_API_KEY=<your-api-key>
+```sh,paths="llm"
+export OPENAI_API_KEY="${OPENAI_API_KEY:-<your-api-key>}"
 ```
 
 ### Step 2: Create the configuration
 
 Create a `config.yaml` that defines an HTTP listener and an AI backend for OpenAI. This configuration listens on port 3000, routes traffic to the OpenAI backend, and attaches your API key to outgoing requests via the `backendAuth` policy.
 
-```yaml
+```yaml,paths="llm"
 cat > config.yaml << 'EOF'
 # yaml-language-server: $schema=https://agentgateway.dev/schema/config
 binds:
@@ -56,6 +56,13 @@ Run agentgateway with the config file.
 agentgateway -f config.yaml
 ```
 
+{{< doc-test paths="llm" >}}
+agentgateway -f config.yaml &
+AGW_PID=$!
+trap 'kill $AGW_PID 2>/dev/null' EXIT
+sleep 3
+{{< /doc-test >}}
+
 Example output:
 
 ```
@@ -68,7 +75,7 @@ info  proxy::gateway started bind  bind="bind/3000"
 
 From another terminal, send a request to the chat completions endpoint.
 
-```sh
+```sh,paths="llm"
 curl -s http://localhost:3000/v1/chat/completions \
   -H "Content-Type: application/json" \
   -d '{

assets/agw-docs/standalone/quickstart/mcp.md

@@ -4,7 +4,7 @@ Use the agentgateway binary to proxy requests to an open source MCP test server,
 
 1. [Install the agentgateway binary]({{< link-hextra path="/deployment/binary" >}}).
 
-   ```sh
+   ```sh,paths="mcp"
    curl -sL https://agentgateway.dev/install | bash
    ```
 
@@ -18,15 +18,15 @@ Use the agentgateway binary to proxy requests to an open source MCP test server,
 
 Download the basic MCP example configuration.
 
-```sh
+```sh,paths="mcp"
 curl -L https://agentgateway.dev/examples/basic/config.yaml -o config.yaml
 ```
 
 ### Step 2: Review the configuration
 
 Inspect the file to see how the listener, route, and MCP backend are defined.
 
-```sh
+```sh,paths="mcp"
 cat config.yaml
 ```
 
@@ -44,6 +44,13 @@ Run agentgateway with the config file.
 agentgateway -f config.yaml
 ```
 
+{{< doc-test paths="mcp" >}}
+agentgateway -f config.yaml &
+AGW_PID=$!
+trap 'kill $AGW_PID 2>/dev/null' EXIT
+sleep 3
+{{< /doc-test >}}
+
 Example output:
 
 ```

assets/agw-docs/standalone/quickstart/non-agentic-http.md

@@ -14,7 +14,7 @@ flowchart LR
 
 1. [Install the agentgateway binary]({{< link-hextra path="/deployment/binary" >}}).
 
-   ```sh
+   ```sh,paths="httpbin"
    curl -sL https://agentgateway.dev/install | bash
    ```
 
@@ -31,21 +31,21 @@ Run the httpbin image so it listens on port 80 inside the container. Map it to a
 {{< tabs items="Linux, macOS (Apple Silicon)" >}}
 {{% tab %}}
 
-```sh
+```sh,paths="httpbin,httpbin-linux"
 docker run --rm -d -p 8000:80 --name httpbin kennethreitz/httpbin
 ```
 {{% /tab %}}
 {{% tab %}}
 
-```sh
+```sh,paths="httpbin-macos"
 docker run --rm -d -p 8000:80 --name httpbin kennethreitz/httpbin --platform linux/amd64
 ```
 {{% /tab %}}
 {{< /tabs >}}
 
 Verify that httpbin responds.
 
-```sh
+```sh,paths="httpbin"
 curl -s http://localhost:8000/headers | head -20
 ```
 
@@ -65,7 +65,7 @@ Example output:
 
 Create a `config.yaml` that listens on port 3000 and routes traffic to the httpbin host. Use a static `host` backend with the address and port where httpbin is reachable, such as `127.0.0.1:8000`.
 
-```yaml
+```yaml,paths="httpbin"
 cat > config.yaml << 'EOF'
 # yaml-language-server: $schema=https://agentgateway.dev/schema/config
 binds:
@@ -86,6 +86,13 @@ In a separate terminal, run agentgateway with the config file.
 agentgateway -f config.yaml
 ```
 
+{{< doc-test paths="httpbin" >}}
+agentgateway -f config.yaml &
+AGW_PID=$!
+trap 'kill $AGW_PID 2>/dev/null' EXIT
+sleep 3
+{{< /doc-test >}}
+
 Example output:
 
 ```
@@ -98,10 +105,23 @@ info  proxy::gateway started bind  bind="bind/3000"
 
 Send a request to agentgateway on port 3000. Agentgateway forwards it to httpbin; the response is returned to you.
 
-```sh
+```sh,paths="httpbin"
 curl -i http://localhost:3000/headers
 ```
 
+{{< doc-test paths="httpbin" >}}
+YAMLTest -f - <<'EOF'
+- name: request through agentgateway to httpbin returns 200
+  http:
+    url: "http://localhost:3000/headers"
+    method: GET
+  source:
+    type: local
+  expect:
+    statusCode: 200
+EOF
+{{< /doc-test >}}
+
 Example response (status and headers):
 
 ```txt
@@ -124,7 +144,7 @@ Example JSON body:
 
 You can try other httpbin endpoints through agentgateway, such as the following.
 
-```sh
+```sh,paths="httpbin"
 curl -s http://localhost:3000/get
 curl -s http://localhost:3000/post -X POST -H "Content-Type: application/json" -d '{"key":"value"}'
 ```
@@ -133,7 +153,7 @@ curl -s http://localhost:3000/post -X POST -H "Content-Type: application/json" -
 
 When you are done, stop and remove the httpbin container.
 
-```sh
+```sh,paths="httpbin"
 docker stop httpbin
 ```
 

content/docs/standalone/main/quickstart/llm.md

@@ -2,6 +2,10 @@
 title: LLM (OpenAI)
 weight: 11
 description: Route requests to OpenAI's chat completions API with the agentgateway binary.
+test:
+  llm-openai:
+  - file: content/docs/standalone/main/quickstart/llm.md
+    path: llm
 ---
 
 {{< reuse "agw-docs/standalone/quickstart/llm.md" >}}

content/docs/standalone/main/quickstart/mcp.md

@@ -2,6 +2,10 @@
 title: MCP servers
 weight: 12
 description: Connect to an MCP server and try tools in the agentgateway playground.
+test:
+  mcp-playground:
+  - file: content/docs/standalone/main/quickstart/mcp.md
+    path: mcp
 ---
 
 {{< reuse "agw-docs/standalone/quickstart/mcp.md" >}}

content/docs/standalone/main/quickstart/non-agentic-http.md

@@ -2,6 +2,10 @@
 title: Non-agentic HTTP traffic
 weight: 13
 description: Route HTTP traffic to a non-agentic backend such as httpbin with the agentgateway binary.
+test:
+  non-agentic-http:
+  - file: content/docs/standalone/main/quickstart/non-agentic-http.md
+    path: httpbin
 ---
 
 {{< reuse "agw-docs/standalone/quickstart/non-agentic-http.md" >}}

scripts/doc_test_extract.py

@@ -429,6 +429,9 @@ def select_blocks(self) -> List[CodeBlock]:
                     continue
                 if selectors and set(block.paths).intersection(selectors):
                     selected.append(block)
+        # Emit blocks in document order (by file, then line) so hidden blocks
+        # (e.g. start server in background) appear before dependent visible blocks.
+        selected.sort(key=lambda b: (b.file_path, b.start_line))
         return selected
 
     def select_test_includes(self) -> List[TestInclude]: