Skip to content

Restart Pod of the control plane on a panic? #6610

Description

@aenshin-pp

Describe the bug
Once panicked, would it be better to restart Pod?

thread 'tokio-runtime-worker' panicked at /usr/local/cargo/git/checkouts/chitchat-xxx/dxxx/chitchat/src/delta.rs:413:9:
assertion failed: mtu >= 100

we use quickwit/quickwit:v0.8.2

current liveness probe is:

livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /health/livez
              port: rest
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1

which leaves pods in a partially-failed state.

Possible solution is to use RUST_PANIC=abort.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions