diff --git a/content/run/operations/troubleshooting/index.md b/content/run/operations/troubleshooting/index.md index 5911301..3ead67c 100644 --- a/content/run/operations/troubleshooting/index.md +++ b/content/run/operations/troubleshooting/index.md @@ -99,6 +99,24 @@ docker-compose -f ~/docker-compose.yml start The Generator client falls back to HTTP polling. Check that `GETH_WS_URL` is correct and the port is reachable. HTTP-only mode works but is slightly slower. +### My Generator appears offline in the dashboard but is still running + +The dashboard determines if a node is online/offline by probing `http://:8080/health`, using the IP it reads from your registered connection string (the `/ip4//...` part of your libp2p multiaddr). If port `8080` isn't reachable from the internet, the probe fails and your Generator shows as offline even though `docker ps` reports the container as healthy and the node is up and committing blocks normally. + +This is the most common reason a running Generator appears offline, and it almost always comes down to `8080` not being published or opened: + +- **Docker:** the default `docker run` in [Install](/run/run-a-generator/install/) publishes `8080` with `-p 8080:8080`. If you removed that flag, or you run a Compose file that doesn't map `8080`, add it back. +- **Firewall / security group:** open inbound TCP `8080` on your server's firewall or cloud security group (Hetzner, DigitalOcean, GCP, AWS, etc.). This is separate from the P2P port `9171`. The dashboard online check only looks at `8080`. +- **Reverse proxy:** if you put nginx in front (as in the [Nginx with snapshots](/run/run-a-generator/deployment-examples/nginx-with-snapshots/) example), make sure the dashboard's plaintext probe to `http://:8080/health` reaches the Generator's health server on port `8080`. If your proxy only listens on a TLS port such as `443`, also publish `8080` directly so the probe can reach it. + +Confirm the fix by running the same probe the dashboard runs, from outside your server: + +```shell +curl -s http://:8080/health | jq -r '.status' +``` + +A healthy Generator returns `healthy`. A connection refused or timeout means `8080` still isn't reachable from outside. The endpoint returns `"unhealthy"` (HTTP 503) only while the Generator's DefraDB node or processing pipeline is still initializing. That clears once startup finishes. + ## Host client ### What is the GitHub link for the Shinzo Host? diff --git a/content/run/run-a-generator/deployment-examples/managed-gcp-node/index.md b/content/run/run-a-generator/deployment-examples/managed-gcp-node/index.md index 6b8955d..2688050 100644 --- a/content/run/run-a-generator/deployment-examples/managed-gcp-node/index.md +++ b/content/run/run-a-generator/deployment-examples/managed-gcp-node/index.md @@ -119,6 +119,8 @@ Verify health. The health server listens on port 8080 inside the container but i docker compose exec shinzo-generator curl -f http://localhost:8080/health ``` +Because this compose doesn't publish `8080`, the public Generators dashboard can't probe `/health`, so your Generator will show as offline on the dashboard even while it's running fine. To have it appear online, publish `8080:8080` on the `shinzo-generator` service and open inbound TCP `8080` in your firewall. See [Troubleshooting](/run/operations/troubleshooting/#my-generator-appears-offline-in-the-dashboard-but-is-still-running). + ## Registration Once the Generator is running, register it with the Shinzo Network. See [Registration](/run/run-a-generator/register/). diff --git a/content/run/run-a-generator/deployment-examples/nginx-with-snapshots/index.md b/content/run/run-a-generator/deployment-examples/nginx-with-snapshots/index.md index 7cae364..05a8ce7 100644 --- a/content/run/run-a-generator/deployment-examples/nginx-with-snapshots/index.md +++ b/content/run/run-a-generator/deployment-examples/nginx-with-snapshots/index.md @@ -241,6 +241,7 @@ Once the Generator is running, register it with the Shinzo Network. See [Registr - `LOG_LEVEL`, `LOG_SOURCE`, and `LOG_STACKTRACE` appear in the original `docker-compose-prod.yml` but are not read by the Generator client, so they are omitted here. `SCHEMA_AUTH_MODE=none` is kept because the Generator client does read it (it controls auth on the `/api/v1/schema` endpoints Nginx proxies). See the [env vars table](/run/run-a-generator/config-reference#environment-variables) for details. - The snapshot directory defaults to `./snapshots` inside the container. Snapshots are written to the DefraDB data directory at `~/shinzo-data/defradb/snapshots` on the host because of the volume mount. Hosts download them through Nginx, not directly from the filesystem. - The `proxy_read_timeout` and `proxy_send_timeout` are set to 300 seconds for snapshot downloads. Large snapshot files can take time to transfer. If Hosts time out downloading, increase these values. +- **Dashboard online check:** the public Generators dashboard probes `http://:8080/health` over plaintext to decide whether your Generator shows as online. This scenario terminates TLS on `443` and only opens `443` and `9171`, so that probe can't reach Nginx and your Generator will appear offline on the dashboard even though it's healthy. If you want it to show as online, also publish `8080` (for example map `8080:8080` on the `shinzo-generator` service, or add a plaintext `listen 8080` server block to Nginx that proxies `/health`) and open inbound TCP `8080` in your firewall. See [Troubleshooting](/run/operations/troubleshooting/#my-generator-appears-offline-in-the-dashboard-but-is-still-running). ## Need help diff --git a/content/run/run-a-generator/install/index.md b/content/run/run-a-generator/install/index.md index 09f40ed..ebfca31 100644 --- a/content/run/run-a-generator/install/index.md +++ b/content/run/run-a-generator/install/index.md @@ -176,11 +176,11 @@ Set `GETH_API_KEY_TYPE` to the header name your provider expects. The following ports must be exposed and available on the machine. -| Port | Service | -| --- | --- | -| `8080` | Health endpoint (`/health`), metrics (`/metrics`), and registration (`/registration`). | -| `9171` | DefraDB P2P. | -| `9181` | DefraDB GraphQL API. | +| Port | Service | Notes | +| --- | --- | --- | +| `8080` | Health (`/health`), metrics (`/metrics`), and registration (`/registration`). | Published by the `docker run` above (`-p 8080:8080`). Open inbound TCP `8080` in your firewall so the public Generators dashboard can probe `/health` and show your Generator as online. If `8080` isn't reachable from the internet, your Generator shows as offline on the dashboard even while it's running fine. | +| `9171` | DefraDB P2P. | Must be reachable from the internet. Open or forward this port. | +| `9181` | DefraDB GraphQL API. | | ## Troubleshooting diff --git a/content/run/run-a-generator/register/index.md b/content/run/run-a-generator/register/index.md index 9b1657b..e265632 100644 --- a/content/run/run-a-generator/register/index.md +++ b/content/run/run-a-generator/register/index.md @@ -18,7 +18,7 @@ Before you start, have the following ready: ## Register your Generator -1. Start your Generator Client. +1. Start your Generator client with the health/registration port published (`-p 8080:8080`) and open inbound TCP `8080` in your firewall. The registration app is served on port `8080`, and the public Generators dashboard also probes `http://:8080/health` to decide whether your Generator shows as online. If `8080` isn't reachable from the internet, your Generator shows as offline on the dashboard even while it's running fine. The default `docker run` in [Install](../install) already publishes `8080`. 1. Add the Shinzo Testnet to your browser wallet with the following values: - Network name: `Shinzo` - Default RPC URL: `http://testnet.shinzo.network:8545`