Skip to content

[Bug]: Lack of logrotate configuration causes disk exhaustion and unit crashes for high-traffic ingresses #638

Description

@alejdg

Bug Description

We are experiencing disk exhaustion issues on units running the haproxy charm when handling high-traffic ingresses. Because the charm currently does not expose any configuration for logrotate, it falls back to the default log rotation policy, which only rotates logs once a week.

For high-volume environments, this weekly rotation is far too slow, causing the HAProxy logs to fill up the entire disk before the rotation is ever triggered. When the disk is filled, the charm goes into error and stops serving.

We already expanded the disk size but this is not a sustainable solution, we have and application that stops every 2 days. We need the ability to configure log rotation (e.g., size-based rotation, daily/hourly frequency, or capping retention) directly through the charm.

Impact

Medium (functionality degraded, workaround exists)

Impact Rationale

No response

To Reproduce

  1. Deploy the haproxy charm and relate it to workloads as an ingress.
  2. Direct a high and continuous volume of traffic to the ingress.
  3. Monitor the disk usage on the HAProxy unit.
  4. Observe that the HAProxy log files grow unbounded over the course of the week because log rotation only runs weekly.
  5. Once the disk fills up, observe the unit going into error.

Environment

channel=2.8/edge
revision=537
running on canonical-k8s

Relevant log output

2026-08-14 05:44:13 INFO juju.worker.uniter.operation runhook.go:186 ran "haproxy-route-relation-created" hook (via hook dispatching script: dispatch)
2026-08-14 05:44:14 ERROR unit.ingress-ps7-certification/15.juju-log server.go:405 haproxy-route:1220: 2 validation errors for RequirerApplicationData
service
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.13/v/missing
ports
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.13/v/missing
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-ingress-ps7-certification-15/charm/lib/charms/haproxy/v2/haproxy_route.py", line 272, in load
    return cls.model_validate_json(json.dumps(data))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/juju/agents/unit-ingress-ps7-certification-15/charm/venv/lib/python3.12/site-packages/pydantic/main.py", line 782, in model_validate_json
    return cls.__pydantic_validator__.validate_json(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 2 validation errors for RequirerApplicationData
service
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.13/v/missing
ports
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.13/v/missing
2026-08-14 05:44:14 ERROR unit.ingress-ps7-certification/15.juju-log server.go:405 haproxy-route:1220: Invalid requirer application data for remote-dcefa3f1833d4dc18e0cd06dcb274f63
2026-08-14 05:44:14 ERROR unit.ingress-ps7-certification/15.juju-log server.go:405 haproxy-route:1220: 2 validation errors for RequirerApplicationData
service
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.13/v/missing
ports
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.13/v/missing
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-ingress-ps7-certification-15/charm/lib/charms/haproxy/v2/haproxy_route.py", line 272, in load
    return cls.model_validate_json(json.dumps(data))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/juju/agents/unit-ingress-ps7-certification-15/charm/venv/lib/python3.12/site-packages/pydantic/main.py", line 782, in model_validate_json
    return cls.__pydantic_validator__.validate_json(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 2 validation errors for RequirerApplicationData
service
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.13/v/missing
ports
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.13/v/missing
2026-08-14 05:44:14 ERROR unit.ingress-ps7-certification/15.juju-log server.go:405 haproxy-route:1220: Invalid requirer application data for remote-dcefa3f1833d4dc18e0cd06dcb274f63
2026-08-14 05:44:16 INFO unit.ingress-ps7-certification/15.juju-log server.go:405 haproxy-route:1220: ha integration is not ready, skipping.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions