Integration Test Kit

SDK compatibility, every night.
Interoperability and conformance testing for the Agent2Agent (A2A) SDKs.

Dashboard | Documentation | Specification

---

Two test suites for the SDKs of the A2A protocol live in this repository. They share one agent launcher, one container image and one CI driver, but they answer different questions - the same two questions the dashboard has a tab for:

🔀 Interoperability - ITK	📐 Conformance - ACTS
Every SDK against its peers.	Each SDK against the specification.
Can this SDK talk to the other SDKs, in both directions, over every transport?	Does this SDK, on its own, do what the specification says?
Starts a cluster of agents built from several SDKs and versions, sends one nested instruction through all of them over JSON-RPC, gRPC or HTTP+JSON, and checks the trace that comes back	Starts one agent and runs the ACTS corpus from the A2A repository against it, one report per binding

• 🧱 One agent, both suites. Each SDK repository carries a small agent under itk/; ITK walks it as a peer, ACTS interrogates it as the system under test.
• 🔌 Three transports. JSON-RPC, gRPC and HTTP+JSON, with streaming, push notifications and resubscription covered on the ITK side and every binding reported separately on the ACTS side.
• 🐳 One pipeline everywhere. The same code runs from the local CLI, inside the container and behind the HTTP service CI calls, so a result reproduces.
• 🌙 Every night. Each SDK publishes its nightly results; the dashboard shows the latest night and the trend.

🧭 What it is for

• Gating SDK changes. An SDK's pull request runs a shared ITK scenario set against reference peers and the ACTS corpus against the SDK's own agent, through the same itk/run_itk.sh in every repository. Nightly runs cover the full peer matrix and publish results.
• Checking a local checkout before pushing. Both suites run from this repo against an SDK directory on disk, with the pipeline CI uses.
• Seeing where the ecosystem stands. The dashboard, per SDK and per suite.

🧩 Supported SDKs

SDK	ITK peer lines	ACTS bindings	Dashboard
Python	1.0, 0.3	JSON-RPC, gRPC, REST	ITK / ACTS
Go	1.0, 0.3	JSON-RPC, gRPC, REST	ITK / ACTS
TypeScript	1.0, 0.3	JSON-RPC, gRPC, REST	ITK / ACTS
Java	1.0	JSON-RPC, gRPC, REST	ITK / ACTS
Rust	1.0	JSON-RPC, gRPC, REST	ITK / ACTS
.NET	1.0	JSON-RPC, gRPC, REST	ITK / ACTS

Which repository and ref each peer line means is in matrix.yaml. Last night's results are on the dashboard; what each suite covers, what is frozen and what is still open is in ITK status and ACTS status.

🚀 Quick start

You need uv and, for ITK, the toolchain of every peer you select (builds run on your machine with each SDK's native tools; the bundled smoke set uses only Python and Go peers). The container has every toolchain if you would rather not install them.

ITK

uv run run_tests.py                                           # bundled smoke set
uv run run_tests.py --scenarios scenarios/traversal/pr.yaml   # the shared PR set
uv run run_tests.py --list-sdks                               # peers matrix.yaml can resolve
uv run run_tests.py --dry-run                                 # plan only, no network

# a local SDK checkout as the code under test
uv run run_tests.py --mount ~/Source/a2a-python/itk --sut-sdk python \
    --scenarios scenarios/traversal/pr.yaml

Peers are fetched and built on first use and cached under ~/.cache/a2a-itk, so the first run is slow and the next ones are not. Details: running ITK.

ACTS

uv run run_acts.py --mount ~/Source/a2a-python/itk --sdk a2a-python                   # JSON-RPC
uv run run_acts.py --mount ~/Source/a2a-python/itk --sdk a2a-python --transport all   # every binding
uv run run_acts.py --mount ~/Source/a2a-python/itk --sdk a2a-python -t CORE-SEND-001 -v

# an agent you started yourself, e.g. one the launcher cannot build
uv run run_acts.py --sut-url http://127.0.0.1:9000 --sdk a2a-foo --behaviors ../a2a-foo/acts/sut-behaviors.yaml

Exit code 0 means conformant: every must test passed on every binding that ran. Details: running ACTS, including the two extra instances an external agent needs for the deviation tests.

In the container

docker build -t itk_service .

docker run --rm \
  -v "$HOME/Source/a2a-python:/app/agents/repo" \
  -v "$HOME/.cache/a2a-itk-launcher:/root/.cache/a2a-itk" \
  itk_service uv run run_tests.py \
    --scenarios scenarios/traversal/pr.yaml --sut-sdk python --mount /app/agents/repo/itk

Run with no command and the image starts the HTTP service CI talks to, with POST /run for ITK and POST /run-acts for ACTS:

docker run -d --name itk-service -p 8000:8000 \
  -v "$HOME/Source/a2a-python:/app/agents/repo" \
  -v "$HOME/.cache/a2a-itk-launcher:/root/.cache/a2a-itk" \
  itk_service

curl http://127.0.0.1:8000/health

In an SDK repository

Each SDK's itk/run_itk.sh is a short shim over scripts/run_itk_shared.sh: it builds and starts the container, mounts the SDK into it, posts the request and reports the result. Running it locally is the same as the workflow:

cd ~/Source/a2a-python/itk
A2A_ITK_REVISION=main bash run_itk.sh                                                      # ITK, PR set
A2A_ITK_REVISION=main ITK_ACTS_RUN=1 ITK_ACTS_TRANSPORTS=jsonrpc,grpc,rest bash run_itk.sh  # ACTS

How the workflows are wired, and what the nightly publishes: ITK in CI, ACTS in CI.

Unit tests

uv run pytest

No network and no SDK needed. This is what this repository's own CI runs.

📊 Dashboard

https://a2aproject.github.io/a2a-itk/dashboard/

The latest night per SDK for both suites and the trend over the rolling window, in the same pass / fail / skip vocabulary the reports use. It is a daily snapshot, not live monitoring: each SDK's nightly uploads its results to a nightly-metrics release in its own repository, and a workflow here fetches them at 04:00 UTC and rebuilds the site. Adding an SDK to it is described in ITK in CI.

📚 Documentation

Everything beyond this page is in docs/, split the same way as the suites. Each folder starts with an overview and continues in reading order:

	Interoperability (ITK)	Conformance (ACTS)
Overview	README	README
01 Running	locally, in Docker, via the service	locally, in Docker, via the service
02 Architecture	resolve -> launch -> traverse -> verify	YAML test -> wire calls -> verdict
03 Inputs	scenarios, matrix.yaml, known failures	the corpus and where it comes from
04 CI	PR and nightly workflows	PR and nightly workflows
05 SDK integration	what an SDK's itk/ agent must do	behaviors, contract file, auth, deviations
06 Status	coverage and open items	gating, corpus pin and open items

🤝 Contributing

Contributions are welcome! Please see the CONTRIBUTING.md file for guidelines on how to get involved.

📄 License

This project is licensed under the Apache 2.0 License. See the LICENSE file for more details.