Skip to content

Tear down fluent-bit cluster in observability #50

@ryanjjung

Description

@ryanjjung

Rationale

  • This was originally built to accomodate the shipping of some Stalwart events to Posthog. Those events are less useful than we anticipated, and can be shipped by Accounts instead (@Sancus can provide the detail on this, as I think he's working on rerouting this now/soon, and that's a prerequisite for working this ticket).
  • Sancus has also encountered another use case recently where this cluster might have been useful, but he found that adding adequate context to the events was overly cumbersome with the fluent-bit solution. These events are also going to be routed through Accounts, which can enrich the events without as much complexity.
  • I have recently worked on shipping logs from Stalwart to CloudWatch Logs, and in that case also, this cluster turned out to be more trouble than it was worth.

So the evidence is adding up that this fluent-bit cluster has not provided the value we originally anticipated, and we should therefore tear down this infrastrucutre.

How To Do That

  1. Wait for @Sancus to give the all-clear on the work to get the Stalwart events out of fluent-bit.
  2. Disable the webhooks on all Stalwart servers to stop the flow of traffic to fluent-bit and prevent errors in Stalwart that would happen if we tore down the cluster first.
  3. Update our Pulumi code to not build this, or the CW log group built for its output.
  4. Destroy related resources.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions