cli.md

Command line reference

tail-watch [flags]
tail-watch check-config [--config <file>]
tail-watch notify-test [flags]

Without a subcommand, tail-watch runs the collector.

Collector flags

Flag	Type	Default	Meaning
-config	string	see below	Config file path. Overrides TAIL_WATCH_CONFIG.
-listen	string	collector.listen, else 127.0.0.1:9109	HTTP listen address.
-socket	string	platform default	tailscaled LocalAPI socket. Ignored when collector.local_only is true.
-log-level	string	info	debug, info, warn, or error. Overrides TAIL_WATCH_LOG_LEVEL.
-once	bool	false	Run every check once, print one line each, and exit.
-ui	bool	collector.ui, else false	Serve the monitor UI and API. --ui=false turns off a config that enables it.
-version	bool	false	Print the version and exit.

Go's flag syntax applies: -flag and --flag are equivalent, and a value is given as --flag value or --flag=value. -h and --help print usage and exit 0.

The default socket is /var/run/tailscale/tailscaled.sock on Linux.

Precedence

Setting	Order, first match wins
Config path	--config, TAIL_WATCH_CONFIG, /etc/tail-watch/tail-watch.yaml
Listen address	--listen, collector.listen, 127.0.0.1:9109
UI	--ui or --ui=false, collector.ui, off
Log level	--log-level, TAIL_WATCH_LOG_LEVEL, info

Environment variables

Variable	Meaning
TAIL_WATCH_CONFIG	Config file path when --config is not given.
TAIL_WATCH_LOG_LEVEL	Log level when --log-level is not given.

Log levels are case-insensitive. warning is accepted as warn. Logs are JSON, one object per line, on stderr. At debug, every check result is logged.

--once

Runs the preflight, runs every check of every configured target once, prints one line per result, and exits. It starts no HTTP server, builds no notifier, resolves no secrets, does not contact a setec server, sends no heartbeat, and ignores the UI.

<target> <check> ok <latency>
<target> <check> fail <error>

Startup checks

Before serving, the collector fails with exit code 1 when:

• the config file cannot be read or is invalid;
• the UI is on and collector.state_dir is unset, or the UI is on with collector.local_only: true;
• tailscaled is unreachable, or its state is not Running;
• tailscaled uses userspace networking (kernel mode is required);
• a target is a remote subnet address on Linux and the host has not accepted routes (tailscale set --accept-routes);
• a sink or heartbeat secret cannot be read or is empty, including a setec secret that is missing or a setec server that is unreachable (server mode only);
• the notification state file cannot be decoded (server mode only).

With collector.local_only: true, every tailscaled check above is skipped: the collector makes no LocalAPI call, logs its hostname and local-only mode, and schedules its checks.

It logs a warning, and continues, when the node lacks tag:monitor, when an exit node is active, when a webhook sink has no secret, and when the UI is reachable beyond loopback.

check-config

tail-watch check-config [--config <file>] [--ui[=false]]

Loads and validates the config. It does not contact tailscaled, does not resolve secrets, and does not contact a setec server, so it needs no environment variables and no network. It does check collector.setec and every *_setec key. On success it prints one line per route, one per target, one per sink, and one for the heartbeat if configured, then exits 0. A route line is route <name> severity=<page|warn> sinks=<names>. Each target line ends with route=<name|none> and sinks=<candidates>. An unrouted target lists every configured sink. none means the target or the route has no candidate sink.

With collector.authorization set, check-config exits 1 when the UI would be off. --ui and --ui=false override collector.ui as they do for the collector.

nyc-router interval=15s timeout=5s network=tailnet checks=tsping route=none sinks=ops-webhook,ops-slack
nyc-nas interval=30s timeout=5s network=tailnet checks=icmp,tcp:445,http:https://10.20.0.50:5001/|DSM route=none sinks=ops-webhook,ops-slack
sink ops-webhook type=webhook min_severity=page events=target.down,target.recovered queue=disk match=sites:nyc targets:nyc-* tags:tag:router
sink ops-slack type=slack min_severity=warn events=all queue=memory
heartbeat interval=1m0s

url=setec:<name> is printed only for a sink whose URL is a url_setec key. The name is the secret's name. The value is not printed.

A target that names a route prints that route and only its sinks:

route db-team severity=page sinks=db-hook
pg-primary interval=30s timeout=5s network=tailnet checks=icmp route=db-team sinks=db-hook

On failure it prints every problem to stderr and exits 1. The only flag is -config.

notify-test

tail-watch notify-test [--config <file>] [--sink <name>] [--timeout <duration>] [--socket <path>]

Sends one test event through each selected sink's real signing, rendering, and retry code. It bypasses filters, queues, and state_dir. It needs the secrets that the sinks reference. For *_setec keys it opens the same in-memory store a normal start does, before each sink is checked. If that server is unreachable, each selected setec sink prints a line naming the sink and the secret and the command exits 1. The value is not printed. Sinks that use *_env or *_file still send. It works when tailscaled is unreachable, in which case the collector name in the payload is the OS hostname and tailscale_ip is omitted.

Flag	Type	Default	Meaning
-config	string	see Precedence	Config file path.
-sink	string	every sink	Send only to the sink with this name.
-timeout	duration	30s	Give up on a sink after this long, including retries.
-socket	string	platform default	tailscaled LocalAPI socket, used to read the collector identity. Ignored when collector.local_only is true.

Sinks are tried in parallel. One line is printed per sink, in config order:

Line	Meaning
<name> <type> <status>	Delivered. <status> is the 2xx response code.
<name> <type> <status-or-error>	Failed. A permanent rejection such as 404 shows the status code.
<name> <type> error: timed out after retries (last: <result>)	Timeout reached while retrying.
<name> <type> error: <detail>	A secret could not be read, or the request could not be built.

Retryable failures (408, 429, 5xx, network errors) are retried with backoff until --timeout. See Event payload for the request format.

Exit codes

Code	Meaning
0	Success. For the collector: clean shutdown, or --once with every check passing. For notify-test: every selected sink answered 2xx. --help also exits 0.
1	Failure. Config or startup error, HTTP server error, a check failed under --once, or a sink failed under notify-test.
2	Usage error. Unknown flag, invalid --log-level or TAIL_WATCH_LOG_LEVEL, or notify-test with no sinks configured or an unknown --sink.

HTTP endpoints

Served on the listen address by the collector, not under --once.

Path	Response
/	Prober status page: each check's recent results, latency, and state.
/metrics	Prometheus text exposition. See Metrics.
/healthz	200 with body ok. Says the process is serving, not that targets are up.
/ui/	Monitor UI. 404 unless the UI is on. Not covered by the role check.
/api/v1/me	Monitor API: GET the caller's login, node, tags, role and identified. 200 with role: "" when no role was granted, and identified: false when tailscaled could not identify the caller. With authorization off, authorization is false and role is editor.
/api/v1/targets	Monitor API: GET list, POST create. 404 unless the UI is on.
/api/v1/targets/{name}	Monitor API: PUT replace, DELETE remove.
/api/v1/validate	Monitor API: POST validate a monitor without saving it.
/api/v1/probe	Monitor API: POST validate a monitor, then run each of its checks once without saving it. Authorized and audited like a write.
/api/v1/peers	Monitor API: GET the tailnet peers the collector can currently see, for the address picker. 404 unless the UI is on.
/api/v1/routes	Monitor API: GET every route from the config file and the UI, POST create a UI route. See Read routes and sinks.
/api/v1/routes/{name}	Monitor API: PUT replace, DELETE remove a UI route. See Change a route.
/api/v1/sinks	Monitor API: GET the configured sinks without their secrets.
/api/v1/route-preview	Monitor API: POST a monitor definition and get the sinks each event type would reach, without saving it. See Preview where an event would go.

POST /api/v1/probe takes the same body as POST /api/v1/targets. An invalid definition gets 422 with an errors list and runs no check. A valid one gets 200 with {"results": [...]}, one entry per check in definition order, each carrying check, ok, latency_ms and, for a failed check, error. A failing check is data, not an error of the call. Each check is bounded by the definition's timeout, and the whole call by 15 seconds.

GET /api/v1/peers answers {"seeded": bool, "peers": [...]}. Each peer carries name (MagicDNS short name, else host name), dns_name, host_name, ips (its Tailscale IPs), tags, online (true, false or null when unknown) and routes (the subnet routes it is primary for). seeded is false, with an empty list, until the first status arrives from tailscaled and whenever the connection to it is down.

Write requests to /api/v1/ must be application/json and must not carry an Origin for another host. Those checks run before the role check. A request that fails them is 415 or 403 and changes nothing.

With collector.authorization unset, every request that reaches the listener is permitted. With it set, GET /api/v1/targets, /peers, /routes and /sinks need viewer. POST, PUT and DELETE on targets and routes, POST /api/v1/validate, POST /api/v1/probe and POST /api/v1/route-preview need editor. A missing role is 403 with an errors list that names the role or says that no role was granted, and nothing changes. GET /api/v1/me does not need a role. /metrics, /healthz, / and /ui/ are not checked.

See Manage monitors in the UI, Roles and Monitor UI risks.

Signals

SIGINT and SIGTERM stop check scheduling and cancel running checks. The collector then stops the notifier and shuts down the HTTP server, waiting up to the longest target timeout plus 5 seconds. It exits 0. Under --once, a signal stops the run and exits 0.