Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions docs/tests/report-grok-build-capability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Grok Build Capability Probe

Date: 2026-05-18T10:45:34Z
Suite: tests/test-grok-build-capability
Runtime target: grok-build-acp / grok-build-cli
Verdict: **Wait**

## Summary

- PASS: 2
- FAIL: 0
- SKIP: 7
- WARN: 0
- grok version: `grok 0.1.211 (2f2cd6d5c)`
- Credential: `GROK_CODE_XAI_API_KEY` missing

## Results

| Probe | Status | Detail |
|---|---|---|

| grok --version | PASS | grok 0.1.211 (2f2cd6d5c) |
| permission default | PASS | --always-approve is explicit; probe did not enable it |
| API-key-only headless | SKIP | GROK_CODE_XAI_API_KEY not provided |
| grok -p final answer | SKIP | requires GROK_CODE_XAI_API_KEY |
| streaming-json schema | SKIP | requires GROK_CODE_XAI_API_KEY |
| session resume | SKIP | requires GROK_CODE_XAI_API_KEY |
| temp repo file edit | SKIP | requires GROK_CODE_XAI_API_KEY |
| ACP stdio | SKIP | requires GROK_CODE_XAI_API_KEY |
| MCP no-op tool | SKIP | requires GROK_CODE_XAI_API_KEY |

## Fixtures

- streaming JSON: `docs/tests/fixtures/grok-build/streaming-json.jsonl`
- JSON final: `docs/tests/fixtures/grok-build/final.json`
- ACP stdio: `docs/tests/fixtures/grok-build/acp-stdio.jsonl`
- MCP no-op: `docs/tests/fixtures/grok-build/mcp-noop.jsonl`

## Notes

- This suite installs Grok Build inside Docker using the official installer URL: `https://x.ai/cli/install.sh`.
- If `GROK_CODE_XAI_API_KEY` is absent, authenticated probes are skipped and the verdict remains Wait.
- The probe does not enable `--always-approve` by default. Permission behavior is inspected separately.
21 changes: 21 additions & 0 deletions tests/test-grok-build-capability/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
FROM node:20-slim

RUN apt-get update && apt-get install -y --no-install-recommends \
bash \
ca-certificates \
coreutils \
curl \
git \
jq \
procps \
sed \
tar \
unzip \
xz-utils \
&& rm -rf /var/lib/apt/lists/*

WORKDIR /probe
COPY tests/test-grok-build-capability/run.sh /probe/run.sh
RUN chmod +x /probe/run.sh

CMD ["/probe/run.sh"]
12 changes: 12 additions & 0 deletions tests/test-grok-build-capability/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Grok Build Capability Probe

This suite validates whether Grok Build can be used as an Agent Network runtime.

Run from the repository root:

```bash
sg docker -c 'docker build -t agent-network-grok-probe -f tests/test-grok-build-capability/Dockerfile .'
sg docker -c 'docker run --rm -e GROK_CODE_XAI_API_KEY="$GROK_CODE_XAI_API_KEY" -v "$PWD/docs/tests:/work/docs/tests" agent-network-grok-probe'
```

Without `GROK_CODE_XAI_API_KEY`, authenticated checks are marked `SKIP` and the verdict remains `Wait`.
Loading
Loading