Repository navigation
Grafana: GitHub App credentials and fixes found on live data - #111
Merged
Merged
Conversation
App bulbashenko-com-grafana (read-only: Actions, Pull requests), installed on all org repos. App ID, installation ID and private key go into the Secret grafana; the secret revision bump restarts Grafana so the env vars are picked up. Refs #74 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The "primary only" filter multiplied by (in_recovery == 0), whose value is 0, so WAL age, streaming replicas and Postgres connections always read 0: PostgresNoStreamingReplica fired falsely and PostgresWALArchivingStalled could never fire. Use `and on (pod)`. - FluxResourceNotReady: Flux Operator writes suspended="True"/"False". - "Pods not ready" counted completed Job pods. - "Typical deploy duration" from Flux sync times was meaningless (syncs take seconds; the time goes into CI). Replaced by "last deploy, ago" from ReplicaSet creation. - Grafana was OOMKilled at 384Mi (it sits at ~380Mi with the GitHub plugin): request 256Mi, limit 768Mi. Refs #74 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bulbashenko
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GitHub App
The
grafanaSecret gets the credentials of the GitHub Appbulbashenko-com-grafana(read-only Actions and Pull requests, all org repos):github-app-id,github-app-installation-id,github-app-private-key.secret-revision2 → 3 restarts Grafana so it reads the new env vars.Fixes after checking every dashboard and rule expression against VictoriaMetrics (thanks to #109)
Bug in the "primary only" filter: it multiplied by
(in_recovery == 0), whose value is 0. As a result:PostgresNoStreamingReplicafired falsely (in fact pg-2 is primary and 1 replica streams);PostgresWALArchivingStalledcould never fire.Now
and on (pod); live values: WAL 4 min ago, 1 replica, connections are visible.FluxResourceNotReady: Flux Operator writessuspended="True"/"False", with a capital letter."Pods not ready" no longer counts completed Job pods (was 9, now 0).
"Typical deploy duration" is replaced by "Last deploy, ago". Flux syncs take seconds; the real time goes into CI, which the CI/CD tables already show.
Grafana was OOMKilled at a 384Mi limit (it uses ~380Mi): now request 256Mi, limit 768Mi.
The other expressions return data. The Stalwart target is down until Prometheus is switched on in its WebUI; that is also why
TargetDown,TooManyScrapeErrorsandTooManyLogs(vmagent) are firing.🤖 Generated with Claude Code