From dffb3222718d239cbeb179961903a556165a3b39 Mon Sep 17 00:00:00 2001 From: Laura Huysamen Date: Thu, 5 Mar 2026 12:31:18 +0000 Subject: [PATCH] Add project documentation (CLAUDE.md, ARCHITECTURE.md, CONTRIBUTING.md) Initialize comprehensive project documentation for the chisel fork: - ARCHITECTURE.md with system diagrams and design decisions - CONTRIBUTING.md with build, test, and PR guidelines - CLAUDE.md with AI agent guidance and quick-reference --- ARCHITECTURE.md | 198 ++++++++++++++++++++++++++++++++++++++++++++++++ CLAUDE.md | 94 +++++++++++++++++++++++ CONTRIBUTING.md | 125 ++++++++++++++++++++++++++++++ 3 files changed, 417 insertions(+) create mode 100644 ARCHITECTURE.md create mode 100644 CLAUDE.md create mode 100644 CONTRIBUTING.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 00000000..bdffd7ad --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,198 @@ +# Architecture + +Chisel is a TCP/UDP tunneling tool transported over HTTP and secured via SSH. A single Go binary acts as both client and server. It is a fork of [jpillora/chisel](https://github.com/jpillora/chisel), maintained by OutSystems with custom build/release pipelines. + +## High-Level Overview + +```mermaid +graph LR + subgraph Client Side + App["Application"] + CP["Chisel Client
(Proxy Listener)"] + end + + subgraph Transport + WS["WebSocket over HTTP/S"] + SSH["SSH Session
(encrypted)"] + end + + subgraph Server Side + SP["Chisel Server
(HTTP + WS Handler)"] + Target["Target Service"] + end + + App -->|"TCP/UDP"| CP + CP -->|"SSH channel
inside WebSocket"| WS + WS --- SSH + SSH -->|"SSH channel
inside WebSocket"| SP + SP -->|"TCP/UDP"| Target +``` + +**Forward tunnel:** Client listens locally, traffic flows through the server to the target. + +**Reverse tunnel:** Server listens, traffic flows back through the client to a target behind the client. + +## Project Structure + +``` +chisel/ +├── main.go # CLI entrypoint — parses flags, delegates to client or server +├── client/ # Client package +│ ├── client.go # Client struct, config, initialization, TLS, SSH setup +│ └── client_connect.go# Connection loop with backoff and reconnection +├── server/ # Server package +│ ├── server.go # Server struct, config, SSH key management, auth +│ ├── server_handler.go# HTTP/WebSocket handler, SSH handshake, tunnel setup +│ └── server_listen.go # TCP listener with TLS (manual certs or LetsEncrypt) +├── share/ # Shared library packages +│ ├── version.go # Protocol version ("chisel-v3") and build version +│ ├── compat.go # Backwards-compatible type aliases +│ ├── ccrypto/ # Key generation, fingerprinting (ECDSA/SHA256) +│ ├── cio/ # Bidirectional pipe, logger, stdio +│ ├── cnet/ # WebSocket-to-net.Conn adapter, HTTP server with graceful shutdown +│ ├── cos/ # OS signals (SIGUSR2 stats, SIGHUP reconnect), context helpers +│ ├── settings/ # Config encoding, remote parsing, user/auth management, env helpers +│ └── tunnel/ # Core tunnel engine — proxy, SSH channel handling, keepalive, UDP +├── test/ +│ ├── e2e/ # End-to-end tests (auth, TLS, SOCKS, UDP, proxy) +│ └── bench/ # Benchmarking tool +├── Dockerfile # Alpine-based container image +├── goreleaser.yml # GoReleaser config — builds Linux binary, pushes to GHCR +└── Makefile # Build targets for cross-compilation, lint, test, release +``` + +## Component Responsibilities + +### `main.go` -- CLI + +Parses the top-level command (`server` or `client`), then delegates to dedicated flag parsers. Handles environment variable fallbacks for host, port, auth, and key configuration. + +### `client/` -- Tunnel Client + +- Establishes a WebSocket connection to the server (optionally through an HTTP CONNECT or SOCKS5 proxy). +- Upgrades the WebSocket to an SSH connection and performs host key fingerprint verification. +- Sends its tunnel configuration (list of remotes) to the server for validation. +- Runs a **connection loop** with exponential backoff for automatic reconnection. +- Creates local **Proxy** listeners for forward tunnels. For reverse tunnels, the server side creates the listeners. + +### `server/` -- Tunnel Server + +- Listens on HTTP(S) with optional TLS (manual key/cert or automatic LetsEncrypt via `autocert`). +- Upgrades incoming WebSocket connections (matching the `chisel-v3` protocol) to SSH server sessions. +- Authenticates users via SSH password auth, checked against an auth file (`users.json`) that hot-reloads on changes via `fsnotify`. +- Validates client-requested remotes against user permissions (regex-based address matching). +- Falls back to a reverse proxy for non-WebSocket HTTP requests, or serves `/health` and `/version` endpoints. + +### `share/tunnel/` -- Tunnel Engine + +The core data-plane, shared by both client and server: + +- **`Tunnel`** -- Binds an SSH connection and manages its lifecycle (keepalive pings, channel handling). +- **`Proxy`** -- Listens on a local address (TCP, UDP, or stdio) and pipes traffic through SSH channels. +- **Outbound handler** -- Receives SSH channel open requests and connects to the target host (TCP dial, UDP dial, or SOCKS5 proxy). +- **UDP multiplexing** -- Encodes/decodes UDP packets with source address metadata over a single SSH channel using `gob` encoding. + +### `share/settings/` -- Configuration + +- **`Remote`** -- Parses the `local:remote` format (e.g., `3000:google.com:80`, `R:2222:localhost:22`, `socks`, `stdio:host:port`). +- **`Config`** -- JSON-serialized handshake payload exchanged between client and server over SSH. +- **`UserIndex`** -- Loads and watches the auth file, manages user permissions with regex-based address ACLs. +- **`Env`** -- Reads `CHISEL_*` environment variables for tuning (timeouts, buffer sizes, UDP settings). + +### `share/ccrypto/` -- Cryptography + +Generates ECDSA P256 keys (deterministic from seed or random), converts between PEM and chisel key formats, and computes SHA256 fingerprints for host key verification. + +### `share/cnet/` -- Network Adapters + +- **`wsConn`** -- Wraps `gorilla/websocket.Conn` as a standard `net.Conn` with buffered reads. +- **`HTTPServer`** -- Extends `net/http.Server` with context-aware graceful shutdown. + +## Connection Lifecycle + +```mermaid +sequenceDiagram + participant C as Client + participant WS as WebSocket + participant S as Server + participant T as Target + + C->>S: HTTP Upgrade (Sec-WebSocket-Protocol: chisel-v3) + S->>C: 101 Switching Protocols + + Note over C,S: SSH handshake over WebSocket + C->>S: SSH ClientConn (with user/pass if auth enabled) + S->>C: SSH ServerConn (host key for fingerprint verification) + + C->>S: SSH Request "config" (JSON: version + remotes) + S->>S: Validate remotes against user permissions + S->>C: Reply (ok / denied) + + Note over C,S: Tunnel active — keepalive pings every 25s + + rect rgb(230, 240, 255) + Note over C,T: Forward tunnel data flow + C->>S: SSH Channel Open "chisel" (target host:port) + S->>T: TCP/UDP Dial + C-->>T: Bidirectional data pipe + end + + rect rgb(255, 240, 230) + Note over C,T: Reverse tunnel data flow + S->>S: Listen on server-side port + S->>C: SSH Channel Open "chisel" (target host:port) + C->>T: TCP/UDP Dial + T-->>S: Bidirectional data pipe + end +``` + +## Protocol Layers + +```mermaid +graph TB + subgraph "Protocol Stack" + L1["Application Data (TCP/UDP payload)"] + L2["SSH Channel (encrypted, multiplexed)"] + L3["SSH Connection (auth, keepalive)"] + L4["WebSocket (framing)"] + L5["HTTP/S (transport, TLS optional)"] + L6["TCP"] + end + + L1 --> L2 --> L3 --> L4 --> L5 --> L6 +``` + +This layering means chisel traffic looks like regular HTTP/WebSocket traffic to firewalls and proxies, while the SSH layer provides encryption and authentication. + +## External Dependencies + +| Dependency | Purpose | +|---|---| +| `gorilla/websocket` | WebSocket transport layer | +| `golang.org/x/crypto/ssh` | SSH protocol implementation (encryption, auth, channels) | +| `golang.org/x/crypto/acme/autocert` | Automatic TLS certificates via LetsEncrypt | +| `armon/go-socks5` | Built-in SOCKS5 proxy server | +| `jpillora/backoff` | Exponential backoff for client reconnection | +| `fsnotify/fsnotify` | Hot-reload of the users auth file | +| `golang.org/x/net/proxy` | SOCKS5 outbound proxy dialer (client side) | +| `golang.org/x/sync/errgroup` | Concurrent goroutine lifecycle management | + +## Key Design Decisions + +1. **SSH over WebSocket over HTTP.** This combination lets chisel traverse corporate proxies and firewalls that only allow HTTP traffic, while still providing authenticated, encrypted tunnels. + +2. **Single binary, dual mode.** The same executable runs as either client or server. The `share/` packages contain all the tunnel logic used by both sides. + +3. **Multiplexed SSH channels.** Multiple tunnel remotes share a single WebSocket/SSH connection. Each remote gets its own SSH channel, avoiding the overhead of multiple TCP connections. + +4. **UDP over SSH.** UDP packets are serialized with `gob` encoding (including source address metadata) and multiplexed through SSH channels, enabling UDP tunneling over the inherently stream-oriented SSH protocol. + +5. **Automatic reconnection.** The client uses exponential backoff with a configurable max retry count and interval. SIGHUP can short-circuit the backoff timer for immediate reconnection. + +6. **Hot-reloadable auth.** The server watches its `users.json` file for changes and reloads permissions without restart, using `fsnotify`. + +7. **Graceful context propagation.** Both client and server use `context.Context` throughout, enabling clean shutdown on OS signals (SIGINT/SIGTERM) via `cos.InterruptContext()`. + +## Build and Release + +The project uses **GoReleaser** to build Linux binaries and push Docker images to `ghcr.io/outsystems/chisel`. The build version is injected via `-ldflags` at compile time into `share.BuildVersion`. The Makefile provides cross-compilation targets for FreeBSD, Linux, Windows, and macOS. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..ecc7f7c4 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,94 @@ +# CLAUDE.md + +Chisel is a TCP/UDP tunneling tool transported over HTTP and secured via SSH. A single Go binary acts as both client and server. This is an OutSystems fork of [jpillora/chisel](https://github.com/jpillora/chisel). + +## Key Commands + +```sh +make all # Build for current platform (uses goreleaser) +make test # Unit tests with race detection and coverage (excludes test/) +make lint # go fmt + go vet (excludes test/) +go test ./... # All tests including e2e +go test ./test/e2e/ # Only e2e tests +make release # Lint + test + goreleaser release +``` + +See [CONTRIBUTING.md](./CONTRIBUTING.md) for full build, test, and release details. + +## Directory Overview + +| Directory | Contents | +|---|---| +| `main.go` | CLI entrypoint -- parses `server`/`client` subcommand, delegates | +| `client/` | Client: WebSocket connection, SSH setup, reconnection with backoff | +| `server/` | Server: HTTP/TLS listener, WebSocket upgrade, SSH auth, user permissions | +| `share/` | Shared libraries used by both client and server | +| `share/tunnel/` | Core tunnel engine -- proxy, SSH channels, keepalive, UDP mux | +| `share/settings/` | Config parsing, remote format, user/auth, `CHISEL_*` env vars | +| `share/ccrypto/` | ECDSA key generation, SSH fingerprinting | +| `share/cnet/` | WebSocket-to-net.Conn adapter, HTTP server with graceful shutdown | +| `share/cio/` | Bidirectional pipe, logging, stdio | +| `share/cos/` | OS signals (SIGUSR2 stats, SIGHUP reconnect), context helpers | +| `test/e2e/` | End-to-end tests (auth, TLS, SOCKS, UDP, proxy) | +| `test/bench/` | Benchmarking tool | +| `example/` | Example configs (users.json, Fly.io deployment) | + +See [ARCHITECTURE.md](./ARCHITECTURE.md) for component responsibilities and data flow diagrams. + +## Important Patterns + +### Module path and import aliases + +The Go module path is `github.com/jpillora/chisel` (upstream path, kept for compatibility). Standard import aliases used throughout: + +```go +chclient "github.com/jpillora/chisel/client" +chserver "github.com/jpillora/chisel/server" +chshare "github.com/jpillora/chisel/share" +``` + +### Backwards compatibility layer + +`share/compat.go` re-exports types and functions from sub-packages under the `chshare` package. This exists for backwards compatibility with code that imports `chshare` directly. When adding new public APIs to `share/` sub-packages, do NOT add new aliases to `compat.go` unless maintaining an existing external API contract. + +### Build version injection + +`share.BuildVersion` is set at compile time via `-ldflags`. Default is `"0.0.0-src"`. The protocol version (`"chisel-v3"`) is in `share/version.go` -- changing it breaks client/server compatibility. + +### Environment variables + +All chisel-specific env vars use the `CHISEL_` prefix, accessed via `settings.Env("SUFFIX")` (which reads `CHISEL_SUFFIX`). Helper functions `EnvInt`, `EnvDuration`, `EnvBool` are in `share/settings/env.go`. + +### Test structure + +- Unit tests live alongside source files (e.g., `client/client_test.go`) +- E2e tests are in `test/e2e/` using `package e2e_test` +- E2e tests share a `testLayout` setup helper defined in `test/e2e/setup_test.go` +- `make test` excludes the `test/` directory; use `go test ./...` to include e2e + +### CGO and cross-compilation + +Most build targets use `CGO_ENABLED=0` except Linux and Windows which use `CGO_ENABLED=1`. All builds use `-trimpath` for reproducibility. + +### Context and shutdown + +Both client and server use `cos.InterruptContext()` for graceful shutdown on SIGINT/SIGTERM. Always propagate `context.Context` through new code paths. + +### go.mod replace directive + +`go.mod` contains `replace github.com/jpillora/chisel => ../chisel`. This is a local development override -- do not remove it, but be aware it means `go mod tidy` expects a sibling directory. + +## Domain Glossary + +| Term | Meaning | +|---|---| +| **Remote** | A tunnel endpoint specification in the format `local:remote` (e.g., `3000:google.com:80`). Parsed by `settings.DecodeRemote`. | +| **Forward tunnel** | Client listens locally, traffic flows through server to the target. | +| **Reverse tunnel** | Server listens, traffic flows back through client. Remotes prefixed with `R:`. | +| **Fingerprint** | SHA256 hash of the server's ECDSA public key, base64-encoded (44 chars). Used for host-key verification. | +| **SOCKS remote** | A special remote using `socks` as the target, routing through the built-in SOCKS5 proxy. | +| **stdio remote** | A special remote using `stdio` as the local host, connecting stdin/stdout to the tunnel (useful with SSH ProxyCommand). | +| **Config handshake** | JSON payload (`settings.Config`) sent from client to server over SSH after connection, containing version and requested remotes. | +| **UserIndex** | Server-side auth structure loaded from `users.json`. Maps `user:pass` to regex-based address ACLs. Hot-reloads via `fsnotify`. | +| **Keepalive** | SSH-level ping sent every 25s (default) to prevent proxy idle timeouts. | +| **Proxy (tunnel)** | A `tunnel.Proxy` instance that listens on a local address and pipes traffic through an SSH channel. | diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..730e6fab --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,125 @@ +# Contributing to Chisel + +## Prerequisites + +- **Go 1.25.1+** (see `go.mod` for exact version) +- **goreleaser** (for builds and releases) +- **gocover-cobertura** (for coverage reports) + +Install build dependencies: + +```sh +make dep +``` + +## Project Structure + +``` +main.go # CLI entrypoint (server + client subcommands) +client/ # Client package +server/ # Server package +share/ # Shared libraries + ccrypto/ # Cryptographic utilities (key generation, SSH keys) + cio/ # I/O helpers (logging, pipes, stdio) + cnet/ # Networking (WebSocket connections, HTTP server, metering) + cos/ # OS utilities (signals, pprof) + settings/ # Configuration (remotes, users, environment) + tunnel/ # Tunnel implementation (TCP/UDP proxying) +test/ + e2e/ # End-to-end tests (auth, TLS, SOCKS, UDP, proxy) + bench/ # Benchmarks +example/ # Example config files (users.json, deployment configs) +``` + +## Building + +Build for the current platform: + +```sh +make all +``` + +Cross-compile for a specific OS: + +```sh +make linux +make darwin +make windows +make freebsd +``` + +Build a Docker image: + +```sh +make docker +``` + +## Running Tests + +Unit tests (excludes the `test/` package): + +```sh +make test +``` + +This runs tests with race detection, generates a coverage report in `./build/`, and produces a Cobertura XML for CI. + +Run all tests directly (including e2e): + +```sh +go test ./... +``` + +Run only e2e tests: + +```sh +go test ./test/e2e/ +``` + +## Linting + +Format and vet the code: + +```sh +make lint +``` + +This runs `go fmt` and `go vet` across all packages (excluding `test/`). + +## Code Conventions + +- Standard Go formatting (`go fmt`) +- No external linting tools beyond `go vet` +- Tests use the `_test` package suffix for e2e tests (e.g., `package e2e_test`) +- Unit tests live alongside their source files (e.g., `client/client_test.go`) +- E2e tests use a shared `testLayout` setup helper in `test/e2e/setup_test.go` + +## Release Process + +Releases are handled via goreleaser. The configuration lives in two places: + +- `.github/goreleaser.yml` -- upstream multi-platform release config +- `goreleaser.yml` -- OutSystems-specific config (Linux only, with Docker image to `ghcr.io`) + +To do a dry-run release: + +```sh +goreleaser release --config .github/goreleaser.yml --clean --snapshot +``` + +To create an actual release: + +```sh +make release +``` + +This runs lint, tests, and then `goreleaser release`. + +## Pull Request Guidelines + +1. Fork or branch from `master` +2. Keep changes focused -- one concern per PR +3. Run `make lint` and `make test` before pushing +4. Ensure your changes compile across platforms (`CGO_ENABLED=0` is used for most builds) +5. Add or update tests for new functionality, especially in `test/e2e/` for tunnel behavior changes +6. Dependabot manages dependency updates on a monthly cycle -- don't batch unrelated dep bumps with feature work