Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 17 additions & 9 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Build stage
FROM rust:1.88.0-slim-bookworm AS builder

# Build arguments for optional feature control
# Pass extra space-delimited features via FEATURES (e.g. "redact-sensitive")
ARG FEATURES=
# Space-delimited Cargo feature list. `auto` enables Azure support on amd64 and
# disables it on other architectures.
ARG FEATURES=auto

RUN apt-get update && apt-get install -y \
pkg-config clang libclang-dev \
Expand All @@ -14,13 +14,21 @@ RUN apt-get update && apt-get install -y \
WORKDIR /app
COPY . .

# On x86_64: build with requested features
# On ARM: build without azure/TPM features (cross-compilation not supported for TPM libs)
RUN if [ "$(dpkg --print-architecture)" = "amd64" ]; then \
cargo build --release --features "azure${FEATURES:+ }${FEATURES}"; \
# Resolve the architecture-dependent default, then build with exactly the
# requested features. An explicitly empty FEATURES value enables no features.
RUN build_features="$FEATURES"; \
if [ "$build_features" = "auto" ]; then \
if [ "$(dpkg --print-architecture)" = "amd64" ]; then \
build_features="azure"; \
else \
build_features=""; \
echo "WARNING: Building on ARM without Azure/TPM features (cross-compilation not supported)"; \
fi; \
fi; \
if [ -n "$build_features" ]; then \
cargo build --release --no-default-features --features "$build_features"; \
else \
echo "WARNING: Building on ARM without Azure/TPM features (cross-compilation not supported)" && \
cargo build --release; \
cargo build --release --no-default-features; \
fi

# Runtime stage
Expand Down
82 changes: 53 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,15 @@ This is designed to be an alternative to [`cvm-reverse-proxy`](https://github.co

Details of the remote-attested TLS protocol are in [attested-tls/README.md](attested-tls/README.md). This is provided as a separate crate for other uses than HTTP proxying.

The proxy-client, on starting, immediately connects to the proxy-server and an attestation-verification exchange is made. This attested-TLS channel is then re-used for all requests from that proxy-client instance.
The proxy-client, on starting, immediately connects to the proxy-server and an attestation-verification exchange is made. This attested-TLS channel is then re-used for requests from that proxy-client instance. If the channel is lost, the client reconnects automatically and repeats the attestation exchange before forwarding subsequent requests.

It has five subcommands:

It has three subcommands:
- `attested-tls-proxy server` - run a proxy server, which accepts TLS connections from a proxy client, sends an attestation and then forwards traffic to a target CVM service.
- `attested-tls-proxy client` - run a proxy client, which accepts connections from elsewhere, connects to and verifies the attestation from the proxy server, and then forwards traffic to it over TLS.
- `attested-tls-proxy get-tls-cert` - connects to a proxy-server, verify the attestation, and if successful write the server's PEM-encoded TLS certificate chain to standard out. This can be used to make subsequent connections to services using this certificate over regular TLS.
- `attested-tls-proxy get-tls-cert` - connect to a proxy server, verify its attestation, and, if successful, write its PEM-encoded TLS certificate chain to standard output. This can be used to make subsequent connections to services using this certificate over regular TLS.
- `attested-tls-proxy attested-file-server` - serve files from a local filesystem path over an attested TLS channel.
- `attested-tls-proxy attested-get` - connect to a proxy server, verify its attestation, make a single HTTP GET request, and write the response body to standard output.

### How it works

Expand All @@ -26,13 +29,13 @@ One or both of the proxy-client and proxy-server may be running in a confidentia

### Measurements File

Accepted measurements for the remote party can be specified in a JSON file containing an array of objects, each of which specifies an accepted attestation type and set of measurement values or OS image hashes.
Accepted measurements for the remote party can be specified in a JSON document loaded from a local file or URL. It contains an array of objects, each of which specifies an accepted attestation type and set of measurement values or OS image hashes.

This aims to be compatible with the formatting used by `cvm-reverse-proxy`.

Details and examples of the measurements file format are [in the `attestation` crate documentation](https://github.com/flashbots/attested-tls/tree/main/crates/attestation#measurements-file).

If a measurements file is not provided, a single allowed attestation type **must** be specified using the `--allowed-remote-attestation-type` option. This may be `none` for cases where the remote party is not running in a CVM, but that must be explicitly specified.
Exactly one verification policy must be provided: either `--measurements-file` or `--allowed-remote-attestation-type`. The latter may be `none` for cases where the remote party is not running in a CVM, but that must be explicitly specified.

As an alternative to specifying measurement values, OS image hashes can be specified. See [portable measurement policies](https://github.com/flashbots/attested-tls/tree/main/crates/attestation#portable-measurement-policies) for details.

Expand All @@ -51,7 +54,7 @@ Header value:
"1": "48 byte RTMR0 value encoded as hex",
"2": "48 byte RTMR1 value encoded as hex",
"3": "48 byte RTMR2 value encoded as hex",
"4": "48 byte RTMR3 value encoded as hex",
"4": "48 byte RTMR3 value encoded as hex"
}
```

Expand All @@ -69,41 +72,49 @@ These are the attestation type names used in the HTTP headers, and the measureme
- `azure-tdx` - TDX on Azure, with vTPM attestation
- `dcap-tdx` - DCAP TDX (platform not specified)

### Other CLI Options

- `--pccs-url` selects the PCCS used to retrieve collateral when verifying DCAP attestations. It defaults to Intel PCS.
- `client`, `get-tls-cert`, and `attested-get` accept `--allow-self-signed` to permit a self-signed remote TLS certificate.
- `client` and `server` accept `--listen-addr-healthcheck` to start a separate HTTP health-check listener.
- `get-tls-cert --out-measurements <PATH>` writes the verified remote measurements as JSON in addition to writing the certificate chain to standard output.
- If `server` is started without `--tls-private-key-path` and `--tls-certificate-path`, it generates a self-signed certificate for its listening IP address.

## Protocol Specification

A proxy-client client will immediately attempt to connect to the given proxy-server.
A proxy-client will immediately attempt to connect to the given proxy-server.

Proxy-client to proxy-server connections use TLS 1.3.

The protocol name `flashbots-ratls/1` must be given in the TLS configuration for ALPN protocol negotiation during the TLS handshake. Future versions of this protocol will use incrementing version numbers, eg: `flashbots-ratls/2`.

Immediately after the TLS handshake, an attestation exchange is made. Details of how this works are in the [attested-tls protocol spepcification](attested-tls/README.md#protocol-specification).
Immediately after the TLS handshake, an attestation exchange is made. Details of how this works are in the [attested-tls protocol specification](attested-tls/README.md#protocol-specification).

Following a successful attestation exchange, the client can make HTTP requests, and the server will forward them to the target service.

As described above, the server will inject measurement data into the request headers before forwarding them to the target service, and the client will inject measurement data into the response headers before forwarding them to the source client.

<!-- TODO describe HTTP version negotiation details -->
The proxy client and proxy server support HTTP/2 and HTTP/1.1 over their attested-TLS channel, with HTTP/2 preferred. The HTTP protocol is combined with the attested-TLS protocol version in ALPN, producing `flashbots-ratls/1+h2` or `flashbots-ratls/1+http/1.1`. A negotiated `flashbots-ratls/1` value without an HTTP suffix falls back to HTTP/1.1.

## Dependencies and feature flags

The `azure` feature, for Microsoft Azure attestation requires [tpm2](https://tpm2-software.github.io) to be installed. On Debian-based systems this is provided by [`libtss2-dev`](https://packages.debian.org/trixie/libtss2-dev), and on nix `tpm2-tss`. This dependency is currently not packaged for MacOS, meaning currently it is not possible to compile or run with the `azure` feature on MacOS.
The `azure` feature, for Microsoft Azure attestation requires [tpm2](https://tpm2-software.github.io) to be installed. On Debian-based systems this is provided by [`libtss2-dev`](https://packages.debian.org/trixie/libtss2-dev), and on nix `tpm2-tss`. This dependency is currently not packaged for MacOS, meaning currently it is not possible to compile or run with the `azure` feature on MacOS.

This feature is disabled by default. Note that without this feature, verification of azure attestations is not possible and azure attestations will be rejected with an error.

## Trying it out locally (without CVM attestation)

This might help give an understanding of how it works.

1. Run the helper script to generate a mock certifcate authority and a TLS certificate for localhost signed by it.
1. Run the helper script to generate a mock certificate authority and a TLS certificate for localhost signed by it.

This requires `openssl` to be installed.

```
./scripts/generate-cert.sh localhost 127.0.0.1
```

2. Start a http server to try this out with, on 127.0.01:8000
2. Start an HTTP server to try this out with, on 127.0.0.1:8000.

This requires `python3` to be installed.

Expand All @@ -123,7 +134,7 @@ cargo run -- server \
127.0.0.1:8000
```

The final positional argument is the target address - in this case the python server we started in step 3.
The final positional argument is the target address - in this case the Python server we started in step 2.
Note that you must specify that you accept 'none' as the remote attestation type.

4. Start a proxy-client:
Expand All @@ -148,17 +159,17 @@ curl 127.0.0.1:6000/README.md

Assuming you started the python http server in the directory of this repository, this should print the contents of this README.

Since we just wanted to make a single GET request here, we can make this process simpler but using the `attested-get` command:
Since we just wanted to make a single GET request here, we can make this process simpler by using the `attested-get` command:

```
cargo run -- attested-get \
--url-path README.md
--url-path README.md \
--tls-ca-certificate ca.crt \
--allowed-remote-attestation-type none \
localhost:7000
```

This should also print the README file. This should work even if the proxy-client from step 5 is not running.
This should also print the README file. This works even if the proxy-client from step 4 is not running.

## CLI differences from `cvm-reverse-proxy`

Expand All @@ -176,11 +187,18 @@ This aims to have a similar command line interface to `cvm-reverse-proxy` but th
```bash
docker build -t attested-tls-proxy .

# With custom features (e.g., without azure/TPM):
# Without the Azure feature:
docker build --build-arg FEATURES="" -t attested-tls-proxy .

# With an explicit space-delimited feature list:
docker build --build-arg FEATURES="azure" -t attested-tls-proxy .
```

**Note for Apple Silicon (M1-M4) Mac users:** When building on ARM Macs, the Docker build will automatically compile without Azure/TPM features (`--no-default-features`) because the TPM libraries cannot be cross-compiled. For production builds with full Azure support, use an x86_64 system.
`FEATURES` specifies the complete Cargo feature list; it does not add to a separate implicit list.
When omitted, it defaults to `auto`, which enables `azure` on amd64 and disables it on other
architectures. In particular, Docker builds on ARM Macs automatically compile without Azure/TPM
support because the TPM libraries cannot be cross-compiled. For production builds with full Azure
support, use an x86_64 system.

### Running

Expand All @@ -190,13 +208,18 @@ The same image supports all subcommands (server, client, get-tls-cert, etc.):
# Show help
docker run --rm attested-tls-proxy --help

# Run as server
docker run --rm attested-tls-proxy server \
# Run as server, forwarding to a service on port 8080 of the Docker host
docker run --rm \
--mount type=bind,source=/path/to/certs,target=/certs,readonly \
--add-host host.docker.internal:host-gateway \
-p 8443:443 \
attested-tls-proxy server \
--listen-addr 0.0.0.0:443 \
--target-addr 127.0.0.1:8080 \
--tls-private-key-path /path/to/key.pem \
--tls-certificate-path /path/to/cert.pem \
--allowed-remote-attestation-type none
--tls-private-key-path /certs/server.key \
--tls-certificate-path /certs/server.crt \
--server-attestation-type none \
--allowed-remote-attestation-type none \
host.docker.internal:8080

# Run as client
docker run --rm attested-tls-proxy client \
Expand All @@ -205,17 +228,18 @@ docker run --rm attested-tls-proxy client \
--allowed-remote-attestation-type none
```

Replace `/path/to/certs` with the host directory containing `server.key` and `server.crt`.
When the target service is another container, attach both containers to the same Docker network and
use the target container's name instead of `host.docker.internal`.

### Testing with Docker Compose

A `docker-compose.yml` is provided to test the full proxy chain:

1. **Generate test certificates:**
```bash
mkdir -p certs && cd certs
../scripts/generate-cert.sh proxy-server 127.0.0.1
# Convert key to PKCS#8 format (required by the proxy)
openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in server.key -out server.pkcs8.key
mv server.pkcs8.key server.key
mkdir -p certs
(cd certs && ../scripts/generate-cert.sh proxy-server 127.0.0.1)
```

2. **Start all services:**
Expand Down
4 changes: 2 additions & 2 deletions attested-tls/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ The first 32 bytes are the SHA256 hash of the encoded public key from the TLS le

The remaining 32 bytes are exported key material ([RFC5705](https://www.rfc-editor.org/rfc/rfc5705)) from the TLS session. This must have the exporter label `EXPORTER-Channel-Binding` and no context data.

In the case of attestation types `dcap-tdx`, `gcp-tdx`, and `qemu-tdx`, a standard DCAP attestation is generated using the `configfs-tsm` linux filesystem interface. This means that this binary must be run with access to `/sys/kernel/config/tsm/report` which on many systems requires sudo.
In the case of attestation types `dcap-tdx` and `gcp-tdx`, a standard DCAP attestation is generated using the `configfs-tsm` Linux filesystem interface. This means that this binary must be run with access to `/sys/kernel/config/tsm/report`, which on many systems requires elevated privileges.

When verifying DCAP attestations, the Intel PCS is used to retrieve collateral unless a PCCS url is provided via a command line argument. If expired TCB collateral is provided, the quote will fail to verify.

Expand All @@ -47,5 +47,5 @@ When verifying DCAP attestations, the Intel PCS is used to retrieve collateral u
These are specified in the `attestation` crate documentation:

- [Attestation types](https://github.com/flashbots/attested-tls/tree/main/crates/attestation#attestation-types)
- [Measuremets file format](https://github.com/flashbots/attested-tls/tree/main/crates/attestation#measurements-file)
- [Measurements file format](https://github.com/flashbots/attested-tls/tree/main/crates/attestation#measurements-file)
- [Portable measurement policies](https://github.com/flashbots/attested-tls/tree/main/crates/attestation#portable-measurement-policies)
Loading