Skip to content

Coder boundary: audit records in the final flush window are dropped from the coderd forward on shutdown (local log keeps them) #1019

Description

@AdrianD97

Environment: Coder v2.34.2, agent-firewall via built-in coder boundary (module v0.0.3, use_agent_firewall_directly=false), nsjail, Kubernetes (3 coderd replicas)

Summary: When a boundary-wrapped command exits shortly after a request, that request's audit record reaches the local --log-dir file but is not forwarded to coderd (coderd.agentrpc: boundary_request). The coderd forward is an async periodic (~5s) batch with no flush/drain on shutdown, so any record in the final interval before the wrapped command exits is silently lost centrally. Independent of decision (allow/deny).

Steps to reproduce
Allowlist = domain=www.docker.com only.

# Variant A — trailing record has a flush window (sleep 5)
boundary --jail-type=nsjail --config=<cfg> -- bash -c '
  curl -sk -o /dev/null -w "kubernetes(deny): %{http_code}\n" https://kubernetes.io/docs/tutorials/ || true   # DENY
  sleep 5
  curl -sk -o /dev/null -w "docker(allow): %{http_code}\n" https://www.docker.com/get-started/ || true    # ALLOW (trailing)
  echo done'
Image
# Variant B — no flush window (back-to-back, immediate exit)
boundary --jail-type=nsjail --config=<cfg> -- bash -c '
  curl -sk -o /dev/null -w "kubernetes(deny): %{http_code}\n" https://kubernetes.io/docs/tutorials/ || true   # DENY
  curl -sk -o /dev/null -w "docker(allow): %{http_code}\n" https://www.docker.com/get-started/ || true    # ALLOW
  echo done'
Image

Compare the local --log-dir file against the coderd boundary_request logs on all replicas.
Variant A — trailing ALLOW lands 40 ms before shutdown (DENY had a ~5s window):

Record Local log coderd
DENY kubernetes.io @ 14:23:51.600Z ✅ present
ALLOW www.docker.com @ 14:23:56.664Z ❌ missing (all 3 replicas)

Variant B — both records within ~110 ms of shutdown, no window:

Record Local log coderd
DENY kubernetes.io @ 14:42:18.273Z ❌ missing
ALLOW www.docker.com @ 14:42:18.321Z ❌ missing

Impact
Short-lived invocations are the canonical use case for Agent Firewall (boundary -- ). For any such command the trailing audit record — and for very short commands all records — never reach the central coderd audit stream, while the local file log is complete. Centralized audit is therefore silently incomplete for exactly the workloads Agent Firewall is designed to wrap.

Suggested fix: on the wrapped command's exit, flush/drain the pending audit buffer to the log-proxy socket and await ack before boundary exits (optionally with a bounded drain timeout).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions