Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 29 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,14 @@ tests/
proxy_node_integration.rs # Integration tests: Node.js proxy capture (HTTP + HTTPS)
proxy_php_integration.rs # Integration test: PHP curl via proxy backend (HTTP + HTTPS)
proxy_php_ldpreload_integration.rs # Integration test: PHP curl via ldpreload backend (Linux only)
proxy_bind_integration.rs # Integration test: --bind 0.0.0.0 + ca.pem export (Docker sidecar mode)
cli_query_integration.rs # Integration tests: exit-code propagation, JSONL purity, list/get/clear, store lock UX
mcp_stdio_integration.rs # Integration test: MCP server via raw JSON-RPC over stdio
apps/node-app/ # Test Node.js app (client.js, client-alts.js, proxy-preload.js)
apps/php-app/ # Test PHP app (client.php, curl extension, PHP 5.3-compatible)
integration/ # Shell-based integration tests
examples/
docker-sidecar/ # phantom as a Docker Compose sidecar tracing a peer container
Cargo.toml # Workspace root + binary crate
plan.md # Japanese-language technical design document
```
Expand Down Expand Up @@ -97,6 +100,7 @@ make check # fmt + clippy + build + test (full CI locally)
| `-b, --backend <BACKEND>` | `proxy` | `proxy` (MITM, cross-platform) or `ldpreload` (Linux only, HTTP + HTTPS) |
| `-o, --output <OUTPUT>` | `tui` | `tui` (interactive) or `jsonl` (stdout stream; exits with the child's exit code) |
| `-p, --port <PORT>` | `8080` | Proxy capture port |
| `--bind <ADDR>` | `127.0.0.1` | IP address the proxy binds to. `0.0.0.0` exposes it to other hosts/containers (Docker sidecar mode) — no auth, trusted networks only |
| `--insecure` | off | Disable TLS verification for backend servers (self-signed certs) |
| `--agent-lib <PATH>` | — | Path to `libphantom_agent.so` (ldpreload backend) |
| `--fault <SPEC>` | — | Fault injection rules (repeatable) |
Expand Down Expand Up @@ -161,6 +165,19 @@ Unlike Node.js, PHP requires **no injected script**: libcurl (the library backin

---

## Docker Sidecar Mode (`--bind`)

phantom can trace an arbitrary web app already running in its own Docker container, without spawning or managing it — run phantom as a **sidecar container** on the same Docker network, and configure the target container's `HTTP_PROXY`/`HTTPS_PROXY` to point at it. This is the same "manual" proxy-configuration mode phantom always supported on a single host (`HTTP_PROXY=http://127.0.0.1:8080 your-app`), extended across a Docker network boundary via `--bind`.

- `--bind 0.0.0.0` makes the proxy reachable from other containers (default is `127.0.0.1`, unreachable from outside the container). **No authentication** — only bind `0.0.0.0` on a trusted/private network.
- The MITM CA certificate is written to **`<data_dir>/ca.pem` on every `phantom run`** (not just when phantom spawns a PHP child) — as soon as the proxy is confirmed listening, regardless of whether a `-- <CMD>` is given. Bind-mount `<data_dir>` into the target container to make the CA available there for HTTPS trust. It is regenerated fresh every run, so sharing one `--data-dir` across concurrent phantom processes will race on `ca.pem` — use one data dir per phantom instance.
- CA trust is client/language-specific (there is no universal env var): curl uses `CURLOPT_CAINFO`/`--cacert`, Node.js uses `NODE_EXTRA_CA_CERTS`, PHP's curl extension uses `-d curl.cainfo=` (same mechanism as automatic PHP child injection above), the JVM needs a truststore import, Python `requests` uses `REQUESTS_CA_BUNDLE`, and Debian/Ubuntu-based images can use `update-ca-certificates`.
- See `examples/docker-sidecar/` for a runnable `compose.yaml` + walkthrough (not verified end-to-end in the environment it was authored in — no Docker daemon was available there; confirm it in your own environment).
- Out of scope: transparent traffic interception (iptables/eBPF, requiring no target container changes) and tracing across container boundaries via the `ldpreload` backend. Possible future work.
- `phantom mcp` sessions always bind loopback only (no `--bind` there) — Docker sidecar mode applies to `phantom run`.

---

## JSONL Output Schema

When `--output jsonl` is used, one JSON object is written per line to stdout. All fields are always present unless marked optional.
Expand Down Expand Up @@ -249,6 +266,9 @@ cargo test --test proxy_php_integration -- --nocapture
cargo build -p phantom-agent
cargo test --test proxy_php_ldpreload_integration -- --nocapture

# --bind 0.0.0.0 + ca.pem export test (Docker sidecar mode; requires curl)
cargo test --test proxy_bind_integration -- --nocapture

# CLI query / exit-code / lock UX integration tests (requires curl for the capture parts)
cargo test --test cli_query_integration -- --nocapture

Expand Down Expand Up @@ -279,6 +299,10 @@ Both tests:

Both auto-skip if `php` (or its curl extension) is not available. `client.php` (shared by both tests, `tests/apps/php-app/client.php`) is written in PHP 5.3-compatible syntax and takes an optional `PHANTOM_TEST_INSECURE` env var to disable curl peer verification for the ldpreload test, where there is no phantom CA to trust against the mock backend's self-signed cert.

### Docker Sidecar Test (`tests/proxy_bind_integration.rs`)

Verifies `--bind 0.0.0.0` still proxies correctly (reachable via loopback, since an unspecified bind always includes it) and that `<data_dir>/ca.pem` is written once the proxy is ready. Drives `phantom run ... -- curl ...` rather than a truly standalone/no-child invocation, since JSONL mode only auto-exits on child completion or ctrl-c — the CA-export code path runs unconditionally before the child-spawn branch either way, so this still fully exercises what a Docker sidecar run would do. The fully standalone case is verified manually via `examples/docker-sidecar/`.

### CLI and MCP Integration Tests

| Test | Description |
Expand Down Expand Up @@ -426,7 +450,7 @@ pub enum StorageError {
|---|---|
| `src/main.rs` | Subcommand dispatch, exit-code mapping, store opening (with lock hint for query commands) |
| `src/cli.rs` | `clap` derive: `Cli`, `Commands`, per-subcommand arg structs, `GlobalOpts` |
| `src/runner.rs` | Child-process spawning: proxy env vars, Node/PHP/Java injection assets, `TempScript`, `wait_for_proxy` |
| `src/runner.rs` | Child-process spawning: proxy env vars, Node/PHP/Java injection assets, `TempScript`, `wait_for_proxy`, `loopback_safe` (bind-address resolution) |
| `src/commands/run.rs` | `phantom run`: proxy/ldpreload capture wiring, JSONL output loop with exit-status propagation |
| `src/commands/query.rs` | `phantom list/get/search/stats/clear`, `--since/--until` parsing, table/json/jsonl output |
| `src/mcp/server.rs` | `PhantomMcp`: rmcp `#[tool_router]` with the 7 MCP tools, `run_mcp` stdio entry |
Expand All @@ -441,14 +465,17 @@ pub enum StorageError {
| `tests/proxy_php_integration.rs` | Integration test: PHP curl HTTP/HTTPS via `proxy` backend (`-d curl.cainfo=` injection) |
| `tests/proxy_php_ldpreload_integration.rs` | Integration test: PHP curl HTTP/HTTPS via `ldpreload` backend (Linux only) |
| `tests/apps/php-app/client.php` | Test client: curl extension, PHP 5.3-compatible syntax; shared by both PHP tests |
| `tests/proxy_bind_integration.rs` | Integration test: `--bind 0.0.0.0` and `<data_dir>/ca.pem` export (Docker sidecar mode) |
| `examples/docker-sidecar/compose.yaml` | Example: phantom as a Docker Compose sidecar tracing a peer `app` container via HTTP_PROXY |
| `examples/docker-sidecar/README.md` | Walkthrough: sidecar pattern, `--bind` security note, per-client CA trust table |
| `crates/phantom-core/src/trace.rs` | `HttpTrace`, `TraceId`, `SpanId`, `HttpMethod` (incl. `FromStr`/`from_hex`) |
| `crates/phantom-core/src/query.rs` | `TraceQuery` filter struct + `matches()` predicate, `StatusRange` parsing |
| `crates/phantom-core/src/view.rs` | `TraceView` agent-facing JSON DTO + `RenderOptions` (truncation, headers-only, redaction) |
| `crates/phantom-core/src/storage.rs` | `TraceStore` trait (incl. `query`/`clear`) |
| `crates/phantom-core/src/capture.rs` | `CaptureBackend` trait |
| `crates/phantom-core/src/error.rs` | `CaptureError`, `StorageError` |
| `crates/phantom-storage/src/fjall_store.rs` | Storage implementation + all storage tests |
| `crates/phantom-capture/src/proxy.rs` | MITM proxy implementation (cross-platform); also exposes `ca_cert_pem()` for PHP `curl.cainfo` injection |
| `crates/phantom-capture/src/proxy.rs` | MITM proxy implementation (cross-platform); `bind_ip` field controls listen address; also exposes `ca_cert_pem()` for PHP `curl.cainfo` injection and Docker sidecar CA export |
| `crates/phantom-capture/src/ldpreload.rs` | LD_PRELOAD capture backend (Linux only) |
| `crates/phantom-agent/src/lib.rs` | LD_PRELOAD dylib: hooks libc `send`/`recv`/`close` and OpenSSL `SSL_write`/`SSL_read` (HTTPS) |
| `crates/phantom-tui/src/app.rs` | TUI state (`App`) and all state mutation |
Expand Down
9 changes: 6 additions & 3 deletions crates/phantom-capture/src/proxy.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use std::collections::HashMap;
use std::net::SocketAddr;
use std::net::{IpAddr, SocketAddr};
use std::sync::Arc;
use std::time::{Duration, Instant, SystemTime};

Expand All @@ -20,6 +20,7 @@ use crate::fault::{FaultConfig, FaultRule};
const MAX_BODY_SIZE: usize = 1024 * 1024;

pub struct ProxyCaptureBackend {
bind_ip: IpAddr,
listen_port: u16,
insecure: bool,
fault_config: FaultConfig,
Expand All @@ -29,8 +30,9 @@ pub struct ProxyCaptureBackend {
}

impl ProxyCaptureBackend {
pub fn new(listen_port: u16, insecure: bool) -> Self {
pub fn new(bind_ip: IpAddr, listen_port: u16, insecure: bool) -> Self {
Self {
bind_ip,
listen_port,
insecure,
fault_config: FaultConfig::default(),
Expand Down Expand Up @@ -67,6 +69,7 @@ impl CaptureBackend for ProxyCaptureBackend {
fault_config: Arc::new(self.fault_config.clone()),
};

let bind_ip = self.bind_ip;
let port = self.listen_port;
let insecure = self.insecure;
let ca_cert_pem = Arc::clone(&self.ca_cert_pem);
Expand All @@ -76,7 +79,7 @@ impl CaptureBackend for ProxyCaptureBackend {
*ca_cert_pem.lock().unwrap() = Some(ca_cert.pem());
let ca = RcgenAuthority::new(key_pair, ca_cert, 1000);

let addr = SocketAddr::from(([127, 0, 0, 1], port));
let addr = SocketAddr::new(bind_ip, port);
info!("Starting proxy on {addr}");

if insecure {
Expand Down
90 changes: 90 additions & 0 deletions examples/docker-sidecar/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Docker Sidecar Example

> **Not verified end-to-end in the environment this was authored in** (no
> working Docker daemon was available there). Please run through the steps
> below yourself and confirm before relying on this pattern. `docker compose
> config` (no daemon required) was used to confirm `compose.yaml`'s syntax
> and structure resolve correctly.

## The pattern

phantom normally spawns and traces a process itself (`phantom run -- node app.js`).
This example instead runs phantom as a **sidecar container** on the same
Docker network as a target app container that phantom does not spawn or
manage — the target container is entirely yours (its own image, its own
`Dockerfile`/compose entry). You only need to:

1. Point the target container's `HTTP_PROXY`/`HTTPS_PROXY` at phantom's
Compose service name (Docker's embedded DNS resolves it), e.g.
`http://phantom:8080`.
2. Trust phantom's MITM CA certificate in the target container for HTTPS
capture (see below — this step is client/language-specific).

This is the same "manual" proxy-configuration mode phantom already supports
on a single host (`HTTP_PROXY=http://127.0.0.1:8080 your-app`), just crossing
a Docker network boundary instead of staying on loopback.

**Scope**: this covers the HTTP_PROXY-sidecar approach only. Transparent
traffic interception (iptables/eBPF, no target container changes needed) and
tracing across container boundaries via the `ldpreload` backend are both out
of scope here — possible future work, not implemented.

## Running it

```sh
cd examples/docker-sidecar
docker compose up --build
docker compose logs -f phantom | jq . # view captured traces (JSONL)
```

`compose.yaml` reuses the repository's existing root `Dockerfile` (a debug
build bundled with test tooling, built for phantom's own CI integration
tests) rather than a dedicated slim release image — deliberate, to keep this
example minimal. A smaller/optimized image is possible future work if
needed.

The `app` service here is a disposable `curlimages/curl` container making
periodic requests to `httpbin.org` (needs outbound internet) purely so the
example is runnable standalone. Swap it out for your own service.

## `--bind 0.0.0.0` and security

By default phantom's proxy only binds `127.0.0.1` (unreachable from other
containers). `--bind 0.0.0.0` is required for a sidecar to be reachable from
other containers on the same Docker network. **The proxy has no
authentication** — only bind `0.0.0.0` on a trusted/private network (an
internal Docker network, never anything internet-facing).

## Trusting the CA certificate

phantom writes its MITM CA certificate to `<data_dir>/ca.pem` (here: `/data/ca.pem`
inside the `phantom` container, shared via the `phantom-data` named volume and
mounted read-only into `app` at `/ca/ca.pem`) every time it starts. **There is
no single environment variable that every HTTP client honors** — this is the
same nuance already documented for phantom's Node.js/PHP auto-injection, just
applied manually here since phantom isn't spawning the target process. Some
common cases:

| Client / language | How to trust `ca.pem` |
|---|---|
| curl / libcurl | `CURLOPT_CAINFO` or `curl --cacert /ca/ca.pem` (used in `compose.yaml`'s example) |
| Node.js | `NODE_EXTRA_CA_CERTS=/ca/ca.pem` |
| PHP curl extension | `-d curl.cainfo=/ca/ca.pem` (same mechanism phantom auto-injects when it spawns a PHP child directly) |
| Java / JVM | Import into a truststore (`keytool -importcert -file /ca/ca.pem ...`) or `-Djavax.net.ssl.trustStore=...` |
| Python `requests` | `REQUESTS_CA_BUNDLE=/ca/ca.pem` |
| Debian/Ubuntu-based images (system-wide) | Copy to `/usr/local/share/ca-certificates/`, then run `update-ca-certificates` |

If your target app doesn't support any of these, it will fail HTTPS requests
with a certificate error once traffic is routed through phantom — that's
expected until CA trust is configured; plain HTTP capture works regardless.

## Viewing traces

- **JSONL (used here, recommended for a non-interactive sidecar)**:
`docker compose logs -f phantom | jq .` streams one JSON trace object per
line as requests happen.
- **TUI**: requires an interactive terminal, so `docker compose up -d` won't
work for it. Use `docker compose run --rm phantom run --bind 0.0.0.0 --port 8080`
instead (drop `--output jsonl` from the command, add `tty: true` /
`stdin_open: true` to the service definition — mirrors the `test-ldpreload`
service in the repository root's `compose.yaml`).
62 changes: 62 additions & 0 deletions examples/docker-sidecar/compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Minimal example: phantom running as a Docker Compose sidecar, tracing a
# separately-managed "app" container via HTTP_PROXY/HTTPS_PROXY.
#
# phantom does NOT spawn or manage the app container here — it is a peer
# service on the same Docker network. The app owns its own image/command;
# only its HTTP_PROXY/HTTPS_PROXY env vars point at phantom, and it trusts
# phantom's MITM CA certificate (written to /data/ca.pem, shared via the
# `phantom-data` volume) for HTTPS.
#
# Usage:
# cd examples/docker-sidecar
# docker compose up --build
# docker compose logs -f phantom # view captured traces (JSONL)
#
# See README.md for the full walkthrough, including how to adapt CA trust
# for other languages/clients, and an explicit disclaimer that this compose
# file was authored without a Docker daemon available to verify it against —
# please confirm it runs end-to-end in your own environment.

services:
phantom:
build:
context: ../..
dockerfile: Dockerfile
command:
- run
- --bind
- "0.0.0.0"
- --port
- "8080"
- --output
- jsonl
- --data-dir
- /data
volumes:
- phantom-data:/data
ports:
- "8080:8080" # optional: lets you also `curl -x http://127.0.0.1:8080 ...` from the host

app:
image: curlimages/curl:8.10.1
depends_on:
- phantom
environment:
HTTP_PROXY: http://phantom:8080
HTTPS_PROXY: http://phantom:8080
NO_PROXY: ""
volumes:
- phantom-data:/ca:ro # phantom writes /data/ca.pem here, shared read-only
entrypoint: ["/bin/sh", "-c"]
command:
- |
while true; do
curl -s -o /dev/null -w 'HTTP %{http_code} %{url_effective}\n' \
http://httpbin.org/get || true
curl -s -o /dev/null --cacert /ca/ca.pem -w 'HTTPS %{http_code} %{url_effective}\n' \
https://httpbin.org/get || true
sleep 5
done

volumes:
phantom-data:
26 changes: 25 additions & 1 deletion src/cli.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
use std::net::IpAddr;
use std::path::PathBuf;

use clap::{Args, Parser, Subcommand, ValueEnum};
Expand Down Expand Up @@ -125,6 +126,11 @@ pub enum Commands {
\n\
• Manual (start phantom alone, then configure your app)\n\
Set HTTP_PROXY=http://127.0.0.1:8080 in the target process yourself.\n\
Same pattern works across a Docker network: run phantom as a sidecar\n\
container with --bind 0.0.0.0, point the target container's\n\
HTTP_PROXY/HTTPS_PROXY at it (e.g. http://phantom:8080), and trust\n\
the MITM CA written to <data-dir>/ca.pem for HTTPS. See\n\
examples/docker-sidecar/.\n\
\n\
ldpreload (Linux only)\n\
Injects libphantom_agent.so via LD_PRELOAD. Hooks send/recv/close at\n\
Expand Down Expand Up @@ -178,7 +184,14 @@ pub enum Commands {
cargo build -p phantom-agent\n\
phantom run --backend ldpreload \\\n\
--agent-lib ./target/debug/libphantom_agent.so \\\n\
-- curl http://api.example.com/v1/users"
-- curl http://api.example.com/v1/users\n\
\n\
# Run as a Docker Compose sidecar, tracing a container you don't spawn:\n\
phantom run --bind 0.0.0.0 --output jsonl --data-dir /data\n\
# then, in the target container:\n\
HTTP_PROXY=http://phantom:8080 HTTPS_PROXY=http://phantom:8080 your-app\n\
# trust /data/ca.pem in the target container for HTTPS. See\n\
# examples/docker-sidecar/."
)]
pub struct RunArgs {
/// Capture backend: 'proxy' (MITM, cross-platform) or 'ldpreload' (Linux, HTTP + HTTPS).
Expand All @@ -194,6 +207,17 @@ pub struct RunArgs {
#[arg(short, long, default_value = "8080")]
pub port: u16,

/// IP address the proxy binds to.
///
/// Defaults to 127.0.0.1 (localhost only, matches today's behavior).
/// Use 0.0.0.0 to expose the proxy to other hosts/containers — e.g. running
/// phantom as a Docker Compose sidecar service (`phantom`) next to a target
/// container configured with HTTP_PROXY=http://phantom:8080. There is no
/// authentication on the proxy port: only bind 0.0.0.0 on a trusted/private
/// network (an internal Docker network, not anything internet-facing).
#[arg(long, default_value = "127.0.0.1", value_name = "ADDR")]
pub bind: IpAddr,

/// Disable TLS certificate verification for connections to backend servers.
/// Use when tracing apps that talk to servers with self-signed certificates.
#[arg(long, default_value = "false")]
Expand Down
Loading
Loading