Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,4 @@ docs/

# OS
.DS_Store
.wrangler/
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,4 @@ cover.out
/bench
/fanout
/fanout-docgen
.wrangler/
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ client, built-in agent, MCP tools, dashboards, and alerts are preview capabiliti
[documentation](https://fanout.run) or run it below.

- OTLP ingest over gRPC and HTTP, with local Parquet storage and DuckDB analytics.
- Explore and FanQL for querying telemetry without an AI provider.
- An investigation agent that answers questions about your telemetry in plain language.
- An MCP server for external agents using the same typed query tools.
- Dashboards across traces, logs, and metrics, and alerts on service error rate, latency,
Expand Down Expand Up @@ -59,7 +60,7 @@ export FANOUT_AUTH_CODE_SECRET=$(openssl rand -hex 32)
fanout
```

Set `FANOUT_VERSION` to a [release tag](https://github.com/labstack/fanout/releases), such as `v2026.10.23-alpha`, to pin a release, and `FANOUT_PREFIX` to choose the installation
Set `FANOUT_VERSION` to a [release tag](https://github.com/labstack/fanout/releases), such as `v2026.11.0-alpha`, to pin a release, and `FANOUT_PREFIX` to choose the installation
directory. Native installs bind `127.0.0.1:7520`; set `FANOUT_ADDR=:7520` for remote access, then
follow the same first-boot setup.

Expand Down Expand Up @@ -211,17 +212,17 @@ generated assets, hooks, and project layout.
## Releases

Versions are CalVer — `vYYYY.M.N[-alpha|-beta|-rc]`, numbered from 0 within each month across all
maturity suffixes. Fanout is alpha (for example, `v2026.10.23-alpha`); GA has no suffix.
maturity suffixes. Fanout is alpha (for example, `v2026.11.0-alpha`); GA has no suffix.
Container tags omit the leading `v`, and every release moves `latest`. Tagged releases publish
identical manifests to Docker Hub (`labstack/fanout`) and GHCR (`ghcr.io/labstack/fanout`); GHCR is
canonical and also carries development images.

| Image tag | Points at |
| --- | --- |
| `labstack/fanout:latest` | the newest release, mirrored on Docker Hub |
| `labstack/fanout:2026.10.23-alpha` | that exact release, mirrored on Docker Hub |
| `labstack/fanout:2026.11.0-alpha` | that exact release, mirrored on Docker Hub |
| `ghcr.io/labstack/fanout:latest` | the newest release |
| `ghcr.io/labstack/fanout:2026.10.23-alpha` | that exact release |
| `ghcr.io/labstack/fanout:2026.11.0-alpha` | that exact release |
| `ghcr.io/labstack/fanout:main` | the tip of `main` |
| `ghcr.io/labstack/fanout:sha-<commit>` | one specific commit |

Expand Down
2 changes: 1 addition & 1 deletion cmd/fanout-docgen/routes.go
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ func renderRoutes(routes []api.RouteDoc) []byte {
b.WriteString("pattern Echo matches on, not a literal URL.\n")
b.WriteString("\n")

b.WriteString("## Interface stability\n\nThe `/api/*` routes serve the browser client and can change without migration\npaths. [MCP](/docs/guides/mcp) is the agent interface. OTLP ingest and\n[operational endpoints](/docs/reference/endpoints) have their own contracts.\n\nManage alert rules through the [HTTP API](/docs/guides/alerts). Administrators\nrotate ingest tokens on **Connect telemetry** or with\n`POST /api/settings/ingest/token/rotate`.\n\n")
b.WriteString("## Interface stability\n\nThe `/api/*` routes serve the browser client and can change without migration\npaths. [MCP](/docs/guides/mcp) is the agent interface. OTLP ingest and\n[operational endpoints](/docs/reference/endpoints) have their own contracts.\n\nManage alert rules on the [Alerts page](/docs/guides/alerts) or through the HTTP API.\nAdministrators rotate ingest tokens on **Connect telemetry** or with\n`POST /api/settings/ingest/token/rotate`.\n\n")

b.WriteString("## Errors\n\nApplication API errors, including unknown routes and wrong methods, return objects shaped like\n`{\"code\":\"bad_request\",\"message\":\"Safe explanation\"}`. Codes are stable\nsnake_case identifiers; clients branch on `code` rather than message text.\nInvalid dashboard or panel specs also include `problems` with field paths,\nmessages and optional hints. Every error response uses `Cache-Control: no-store`;\nHEAD errors have no body. Server failures return a short, safe message; the\nunderlying error is logged and never returned.\n\nDefault codes by HTTP status:\n\n| Status | Code |\n|---|---|\n| 400 | `bad_request` |\n| 401 | `unauthorized` |\n| 403 | `forbidden` |\n| 404 | `not_found` |\n| 405 | `method_not_allowed` |\n| 409 | `conflict` |\n| 413 | `payload_too_large` |\n| 415 | `unsupported_media_type` |\n| 422 | `unprocessable` |\n| 429 | `rate_limited` |\n| 500 | `internal` |\n| 502 | `bad_gateway` |\n| 503 | `unavailable` |\n| 504 | `timeout` |\n\nOther 4xx statuses use `bad_request`; other 5xx statuses use `internal`.\nSpecific conditions use `already_current`, `dashboard_version_not_found`,\n`dashboard_version_conflict`, `invalid_spec` and `last_active_administrator`.\n`GET /api/telemetry/schema` returns 503 `schema_busy` with `Retry-After` when a\nnew window and namespace cannot start a discovery yet, or when the discovery is\nstill running after 15 seconds; it keeps running, so a retry finds it cached.\nCached ones are always served. An unknown window is 400 `bad_request`.\nTool failures carry `error: {code, message}` inside the tool result.\nPanel execution results keep their own status and error fields.\nOAuth, OTLP/gRPC, MCP JSON-RPC and AG-UI run errors keep their protocol spelling.\nOperational responses at `/healthz`, `/readyz` and `/metrics` keep their bodies.\n\n")

Expand Down
13 changes: 11 additions & 2 deletions site/astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -82,15 +82,21 @@ export default defineConfig({
{ label: "Install", slug: "docs/get-started/install" },
{ label: "First boot", slug: "docs/get-started/first-boot" },
{ label: "Send your first telemetry", slug: "docs/get-started/send-telemetry" },
{ label: "Get help and contribute", slug: "docs/help" },
],
},
{
label: "Guides",
items: [
{ label: "Terminate TLS", slug: "docs/guides/tls" },
{ label: "Use the built-in agent", slug: "docs/guides/chat" },
{ label: "Query with Explore", slug: "docs/guides/explore" },
{ label: "Build and refine dashboards", slug: "docs/guides/dashboards" },
{ label: "Set up alerts", slug: "docs/guides/alerts" },
{ label: "Connect an agent over MCP", slug: "docs/guides/mcp" },
{ label: "Build and refine dashboards", slug: "docs/guides/dashboards" },
{ label: "Run with Compose or systemd", slug: "docs/guides/deploy" },
{ label: "Terminate TLS", slug: "docs/guides/tls" },
{ label: "Upgrade", slug: "docs/guides/upgrade" },
{ label: "Harden an instance", slug: "docs/guides/hardening" },
{ label: "Back up and restore", slug: "docs/guides/backup-and-restore" },
{ label: "Tune retention", slug: "docs/guides/retention" },
{ label: "Troubleshoot", slug: "docs/guides/troubleshooting" },
Expand All @@ -110,6 +116,7 @@ export default defineConfig({
{ autogenerate: { directory: "docs/reference/configuration" } },
],
},
{ label: "CLI", slug: "docs/reference/cli" },
{ label: "HTTP routes", slug: "docs/reference/api" },
{ label: "MCP tools", slug: "docs/reference/mcp" },
{ label: "FanQL", slug: "docs/reference/fanql" },
Expand All @@ -126,6 +133,8 @@ export default defineConfig({
{ label: "Why one binary", slug: "docs/concepts/architecture" },
{ label: "How storage works", slug: "docs/concepts/storage" },
{ label: "Authentication modes", slug: "docs/concepts/authentication" },
{ label: "Insights and anomalies", slug: "docs/concepts/insights" },
{ label: "How Fanout compares", slug: "docs/concepts/comparison" },
{ label: "Performance", slug: "docs/concepts/performance" },
],
},
Expand Down
84 changes: 79 additions & 5 deletions site/src/content/docs/docs/concepts/authentication.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,10 @@ and account suspension rules.
Active users sign in with an emailed code when SMTP is configured. Without
SMTP, an operator with access to the control database mints a single-use link
with `fanout login-link <email>`. Administrators manage accounts and roles
inside Fanout. See [account profiles and suspension](#account-profiles-and-suspension)
for account APIs and [authentication operations](https://github.com/labstack/fanout/blob/main/docs/authentication.md)
for invitation delivery and recovery.
through the HTTP API. There is no user-management browser page. See
[account profiles and suspension](#account-profiles-and-suspension)
for account APIs and [local recovery](#recover-local-sign-in) for invitation delivery
and recovery.

`FANOUT_SELF_SIGNUP` allows verified email addresses to create viewer accounts.
It defaults to `false` and requires complete SMTP configuration. Self-signup is
Expand Down Expand Up @@ -48,8 +49,8 @@ from the configured group mapping, falling back to `FANOUT_OIDC_DEFAULT_ROLE`

`FANOUT_OIDC_EMAIL_VERIFICATION=required` requires a verified email. The `issuer`
policy trusts the configured issuer and requires an allowlist. See
[OIDC operations](https://github.com/labstack/fanout/blob/main/docs/authentication.md#oidc)
for public access and Microsoft Entra ID group-overage handling.
[Microsoft Entra ID group overage](#microsoft-entra-id-group-overage) if group
claims are omitted.

### Role ownership

Expand All @@ -62,6 +63,14 @@ Without a role-group mapping, administrators manage roles inside Fanout and
OIDC sign-in preserves them. Reconciliation cannot demote the last active
administrator; promote a second administrator before applying that mapping.

### Microsoft Entra ID group overage

Entra can omit the `groups` claim when an account has too many memberships.
If the token signals group overage and any group-based policy is configured,
Fanout denies login. It does not fetch group membership from Graph.
Configure the application registration to emit only groups assigned to the
application. Domain-only allow policies are unaffected.

## Account profiles and suspension

Administrators create active accounts with `POST /api/users`, supplying `email`,
Expand All @@ -73,6 +82,71 @@ Email, role and status changes revoke browser sessions. Suspension blocks local
and OIDC sign-in and OAuth token use. Reactivation does not restore revoked
sessions. The last active administrator cannot be demoted, suspended or deleted.

`GET /api/users` returns a `users` array. `DELETE /api/users/{id}` removes an
account and cascades to its sessions, OAuth grants and owned dashboards. Use
suspension when the account and its data should remain.

`POST /api/users/{id}/access/revoke` revokes all browser sessions and OAuth
tokens for that user. It preserves the account. A user who remains eligible
can sign in again.

### Call account APIs from a script

These routes accept only a browser session with the admin role. Fanout has no
API key for user administration. Ingest tokens, metrics tokens and MCP OAuth
tokens cannot substitute for that session.

Sign in as an administrator. In browser developer tools, find an authenticated
request and copy its session cookie into a temporary Netscape-format cookie
jar supported by curl. Keep it private and outside source control. A script
can use that jar while the session remains valid:

```sh
curl --fail-with-body --cookie "$COOKIE_JAR" \
https://fanout.example.com/api/users

curl --fail-with-body --cookie "$COOKIE_JAR" \
-H 'Fanout-Request: 1' -H 'Content-Type: application/json' \
-X POST https://fanout.example.com/api/users \
--data '{"email":"user@example.com","display_name":"New user","role":"viewer"}'
```

Replace the origin and set `COOKIE_JAR` to that private jar. Mutating requests
need `Fanout-Request: 1` or a matching Origin/Referer; cross-site browser requests
are rejected. Session expiry or revocation requires signing in again. Delete
the temporary jar when finished and sign out to revoke that session.

## Recover local sign-in

Run with the server's environment and data directory:

```sh
fanout login-link admin@example.com
# For Docker:
docker exec fanout fanout login-link admin@example.com
```

The URL is printed to stderr, expires in 15 minutes and works once. Issuance
and redemption are audited. The command supports an online server but refuses
missing or suspended users, a missing control database and OIDC mode.

Creating a local user without SMTP returns `201`,
`invite_delivery: "not_configured"` and `login_link_required: true`. Mint a link
for that active user and deliver it through a trusted channel. If configured
SMTP fails delivery, the API reports an error. Repair SMTP after regaining access.

If the only administrator is suspended or has an unusable address, stop Fanout
and back up the control database. Repair that existing account's email or status
in SQLite, preserving its ID and owned data. Restart and mint a login link for
the active account. OIDC recovery belongs at the provider or configuration layer.

## Review MCP connections

Open **Connect telemetry → Connected agents** in the account menu to review and disconnect one OAuth client.
Disconnecting revokes that client's access and refresh tokens for your account.
Reconnect by authorizing it again. Other clients and browser sessions remain
connected. See [MCP connections](/docs/guides/mcp#disconnect-an-agent) for API routes.

## Access boundaries

See [namespace access limits](/docs/reference/capabilities#operating-limits).
Expand Down
20 changes: 20 additions & 0 deletions site/src/content/docs/docs/concepts/comparison.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: How Fanout compares
description: Compare the deployment shape and scope of Fanout with other telemetry systems.
summary: The comparison table from the project README.
read_when:
- You are deciding whether one Fanout instance fits your workload.
status: preview
---

| If you use | Where Fanout differs |
| --- | --- |
| **Grafana with Loki, Tempo, and Mimir** | That stack keeps a service and a query language per signal, plus object storage underneath. Fanout keeps one process, one data directory, and one typed contract across all three signals, at the cost of the horizontal scale those components are built for. |
| **SigNoz** | Both are OTLP-native and self-hosted. SigNoz composes a collector, ClickHouse, and query services; Fanout compiles ingest, authoritative Parquet storage, indexed trace reads, DuckDB analytics, alerting, and the browser client into one binary. |
| **Jaeger** | Jaeger covers traces and expects a storage backend you run separately. Fanout ingests traces, logs, and metrics into the same store, with nothing else to deploy. |
| **Prometheus with Grafana** | Prometheus pulls metrics and is excellent at them. Fanout accepts pushed OTLP for all three signals and is built around investigating a specific incident rather than maintaining long-range metric series. |
| **Datadog**, **Honeycomb**, **Grafana Cloud** | Those are managed services: someone else runs the storage, the scaling, and the upgrades, and your telemetry leaves your network to get there. Fanout is a binary you run, on data that stays on your disk. |
| **An OpenTelemetry Collector piped into ClickHouse** | The same shape, assembled by hand: collector, database, dashboards, and the glue between them. Fanout is that assembly as one program, with an agent and an MCP server already wired to the same query contract. |

Fanout is a single node. It has no clustering, no replication, and no object tier; a deployment that
outgrows one machine's disk and CPU has outgrown Fanout.
64 changes: 64 additions & 0 deletions site/src/content/docs/docs/concepts/insights.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
title: Insights and anomalies
description: Understand detector snapshots and the values used by alert rules.
summary: Service changes, repeated log patterns, health scores and alert variables.
read_when:
- You need to interpret an insight or detector-based alert.
status: preview
---

## Read a snapshot

`GET /api/insights` returns the latest completed detector snapshot and requires
`telemetry:read`. It returns 503 before a snapshot is ready. MCP clients use
`get_insights`. Neither path requires an AI provider.

The detector runs once at startup and every 60 seconds. Its lookback is
15 minutes. These are built-in detector settings, not environment options.
Snapshots contain anomalies, repeated log patterns, derived insights, a summary,
`generated_at`, `timeframe` and `health_score`.

## What detection measures

The detector checks service error-rate changes, latency degradation and span
volume changes across all namespaces. Service checks use completed rollup
minutes, ending before the service watermark. The baseline is the preceding
window of equal length. Latency uses the span-weighted mean of minute p95
values. See [rollup timing](/docs/concepts/storage#rollups-follow-durable-publication).

Scores compare current values with the baseline and apply signal-specific
variance floors and change thresholds. The z-score thresholds are 2.0 for
error rate and latency, and 2.5 for volume. Treat a finding as a reason to
inspect its current value, baseline and service evidence.

Log detection reads the current wall-clock lookback even when the rollup
watermark has not advanced. It groups `WARN`, `ERROR` and `FATAL` logs by body
template, severity and service. A pattern needs at least three occurrences;
the snapshot keeps at most 50, ordered by count. Repeated `ERROR` patterns
with at least 20 occurrences also produce warning insights.

## Health score

The score starts at 100 and has a floor of zero:

| Finding | Deduction |
|---|---|
| Absolute anomaly z-score at least 3.0 | 15 |
| Absolute anomaly z-score at least 2.5 but below 3.0 | 10 |
| Absolute anomaly z-score at least 2.0 but below 2.5 | 5 |
| Each `ERROR` pattern with at least 10 occurrences | 5 |
| Each `WARN` pattern with at least 20 occurrences | 2 |

This is a summary of detected findings, not an uptime percentage.

## Alert variables

Alert evaluation takes `health_score` from the latest detector snapshot and
uses the same score for every service. For each service, `z_score` is the
signed anomaly score with the largest absolute value across that service's
anomalies. It is zero when that service has no anomaly in the snapshot.
If no snapshot exists, both values default to zero.

Use `z_score > 2.5 || z_score < -2.5` when either direction matters. Combine these values with
service measurements in [alert rules](/docs/guides/alerts). Detector timing
is separate from the alert evaluation interval.
13 changes: 12 additions & 1 deletion site/src/content/docs/docs/get-started/first-boot.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,11 @@ Open the URL, create the administrator, and **save the ingest token shown after
setup**. The raw token is displayed once; Fanout stores only its hash and cannot
show it again.

An administrator can issue a replacement in **Connect telemetry → Ingest token** in the account menu, or
with `POST /api/settings/ingest/token/rotate`. Save the new token when it is
shown. Rotation invalidates the old token immediately and cuts off existing
exporters until you update their credentials.

The setup form sends `email`, `display_name`, and `setup_token` to
`POST /api/auth/setup`. The first administrator starts with status `active`.

Expand All @@ -57,7 +62,13 @@ installation and rebuild with a fresh data directory before repeating setup.
Local mode has no passwords. Once SMTP is configured, active users receive
short-lived codes by email. Without SMTP, run `fanout login-link <email>` with
the server's environment and data directory to mint a 15-minute, single-use
link for an existing active user. OIDC deployments use the provider's flow instead.
link for an existing active user. For Docker, use:

```sh
docker exec fanout fanout login-link admin@example.com
```

OIDC deployments use the provider's flow instead.

## Next

Expand Down
Loading
Loading