Skip to content

dpll: phase_offset noise triggers continuous dpll_pin_change_ntf() causing ~8GB/day kernel slab memory leak #68

Description

@babu916

=== System ===
Kernel: 6.8.1-1015-realtime
Driver: ice 2.2.9 (out-of-tree)
/lib/modules/6.8.1-1015-realtime/updates/drivers/net/ethernet/intel/ice/ice.ko
Hardware: Intel E810-C for SFP x4 PFs (01:00.0 - 01:00.3)
Platform: AMD (Gigabyte), 32 CPUs
IOMMU: iommu=pt iommu.strict=0 amd_iommu=on

=== DPLL State During Leak (stable/locked, state=3) ===
dpll_0_state: 3 (LOCKED)
dpll_1_state: 3 (LOCKED)
— DPLL is fully locked and stable. No genuine state changes occurring.
— Notifications are firing purely due to phase_offset measurement noise.

=== Slab Memory Leak ===
kmalloc-rnd-08-8k: 3,681,000 objects = ~28.5 GB pinned kernel memory
Leak rate: ~12 objects/sec = ~96 KB/sec = ~8 GB/day
Only recoverable via reboot (SUnreclaim — not freed by drop_caches)

=== ftrace: ice_dpll_notify_changes Firing Rate ===
Fires TWICE per poll cycle (once for EEC, once for PPS dpll):

ice-dplls-0000:-584 285996.823291: ice_dpll_notify_changes <-ice_dpll_periodic_work
ice-dplls-0000:-584 285996.842341: ice_dpll_notify_changes <-ice_dpll_periodic_work +19ms (EEC+PPS pair)
ice-dplls-0000:-584 285997.382425: ice_dpll_notify_changes <-ice_dpll_periodic_work +540ms (next poll)
ice-dplls-0000:-584 285997.382426: ice_dpll_notify_changes <-ice_dpll_periodic_work +0ms (EEC+PPS pair)
ice-dplls-0000:-584 285997.901076: ice_dpll_notify_changes <-ice_dpll_periodic_work +519ms (next poll)
ice-dplls-0000:-584 285997.916758: ice_dpll_notify_changes <-ice_dpll_periodic_work +16ms (EEC+PPS pair)
ice-dplls-0000:-584 285998.450170: ice_dpll_notify_changes <-ice_dpll_periodic_work +533ms (next poll)
ice-dplls-0000:-584 285998.450171: ice_dpll_notify_changes <-ice_dpll_periodic_work +0ms (EEC+PPS pair)

Poll interval: ~530ms (matches msecs_to_jiffies(500))
Rate: 2 notifications/530ms = ~3.8/sec per PF
With 4 PFs: ~15 notifications/sec → ~15 SKB alloc+drop/sec

=== perf: Full Call Stack ===
Primary leak path:
ice_dpll_periodic_work
└─ ice_dpll_notify_changes ← fires every 530ms due to phase_offset noise
└─ dpll_pin_change_ntf
└─ dpll_pin_event_send
└─ dpll_cmd_pin_get_one ← builds full netlink message
└─ dpll_msg_add_pin_dplls
└─ ice_dpll_input_state_get ← callback INTO ice driver
└─ ice_dpll_pin_state_update ← issues AQ command to FW!

Secondary path (hardware polling):
ice_dpll_periodic_work
└─ ice_dpll_update_state
└─ ice_get_cgu_state
└─ ice_aq_get_cgu_dpll_status ← AdminQueue to firmware

ice-dplls CPU: 0.33% average, bursts to 0.99% per poll cycle

=== Root Cause ===
In ice_dpll_notify_changes() [ice_dpll.c]:

#if defined(HAVE_DPLL_PHASE_OFFSET)
if (d->prev_phase_offset != d->phase_offset) { // ← PROBLEM
d->prev_phase_offset = d->phase_offset;
if (!pin_notified && d->active_input)
dpll_pin_change_ntf(d->active_input); // ← SKB allocated here
}
#endif

The CGU phase_offset register oscillates by small amounts (~10-50ps) even when DPLL is fully locked (state=3). This causes prev != current on nearly every 500ms poll, triggering dpll_pin_change_ntf() for both EEC and PPS dplls continuously.

With no DPLL netlink subscriber, dpll_pin_event_send() allocates an SKB and immediately drops it — but the 8K slab allocation is not freed.

=== Proposed Fix ===

diff --git a/src/ice_dpll.c b/src/ice_dpll.c
index e27118b..c2550cb 100644
--- a/src/ice_dpll.c
+++ b/src/ice_dpll.c
@@ -11,6 +11,7 @@
 #define ICE_CGU_STATE_ACQ_ERR_THRESHOLD                50
 #define ICE_DPLL_PIN_IDX_INVALID               0xff
 #define ICE_DPLL_RCLK_NUM_PER_PF               1
+#define ICE_DPLL_PHASE_OFFSET_THRESHOLD                500 /* picoseconds */
 #define ICE_DPLL_PIN_ESYNC_PULSE_HIGH_PERCENT  25
 #define ICE_DPLL_PIN_GEN_RCLK_FREQ             1953125
 #define ICE_DPLL_PIN_PRIO_OUTPUT               0xff
@@ -2389,9 +2390,12 @@ static void ice_dpll_notify_changes(struct ice_dpll *d)
        }
 #if defined(HAVE_DPLL_PHASE_OFFSET)
        if (d->prev_phase_offset != d->phase_offset) {
-               d->prev_phase_offset = d->phase_offset;
-               if (!pin_notified && d->active_input)
-                       dpll_pin_change_ntf(d->active_input);
+               if (abs(d->prev_phase_offset - d->phase_offset) >
+                   ICE_DPLL_PHASE_OFFSET_THRESHOLD) {
+                       d->prev_phase_offset = d->phase_offset;
+                       if (!pin_notified && d->active_input)
+                               dpll_pin_change_ntf(d->active_input);
+               }
        }
 #endif /* HAVE_DPLL_PHASE_OFFSET */
 }

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions