feat(logs): collect kernel crash dumps via systemd journal (systemd-pstore.service)#1808
Closed
timojohlo wants to merge 1 commit into
Closed
feat(logs): collect kernel crash dumps via systemd journal (systemd-pstore.service)#1808timojohlo wants to merge 1 commit into
timojohlo wants to merge 1 commit into
Conversation
timojohlo
force-pushed
the
feat/logs-pstore-receiver
branch
2 times, most recently
from
July 10, 2026 13:08
770b1c1 to
6887a63
Compare
…store.service) systemd-pstore.service always logs pstore contents to the journal via sd_journal_sendv() regardless of the Storage= setting (see systemd's src/pstore/pstore.c: /* Always log to the journal */). Each entry carries MESSAGE=PStore <filename> ... and FILE=<raw dmesg payload>. We can therefore pick up kernel panic/MCE dumps directly from journald and skip the hostPath mount on /var/lib/systemd/pstore. Changes (gated by kvmConfig.enabled): - journald/pstore receiver with units=[systemd-pstore.service] and merge=true (merge=true is required because the collector container has /var/log/journal mounted but not /run/log/journal; without --merge journalctl finds nothing) - transform/pstore extracts MESSAGE, FILE, _BOOT_ID, _HOSTNAME, _SYSTEMD_UNIT from log.body into attributes/resource - dedicated logs/kvm_pstore pipeline (no k8s_attributes; source is host-level) - removed file_log/pstore receiver, transform, and /var/lib/systemd/pstore hostPath volume mount Works with both Storage=external (default) and Storage=journal in /etc/systemd/pstore.conf. Verified end-to-end on cc-b0-qa-de-1: - helm install rolled out 13/13 daemonset pods clean - receiver command emits journalctl ... --unit systemd-pstore.service --merge - synthetic pstore entry (via systemd-pstore.service drop-in override) flowed through: otelcol_receiver_accepted_log_records_total went 13 -> 18 with 0 failures and 0 refusals
timojohlo
force-pushed
the
feat/logs-pstore-receiver
branch
from
July 13, 2026 08:41
6887a63 to
0d3e891
Compare
timojohlo
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Collect kernel crash dumps (panics, MCEs) from the systemd journal instead of reading files under
/var/lib/systemd/pstore/. This removes a hostPath volume mount and works regardless ofStorage=mode in/etc/systemd/pstore.conf.Gated by
kvmConfig.enabled.journald/pstorereceiver filtered on_SYSTEMD_UNIT=systemd-pstore.service(units:), withmerge: true(required — the collector container has/var/log/journalmounted but not/run/log/journal),start_at: beginning,all: true.transform/pstoreextractingMESSAGE,FILE,_BOOT_ID,_HOSTNAME,_SYSTEMD_UNITfromlog.bodyinto attributes / resource attributes, and taggingconfig.parsed: pstore.logs/kvm_pstorepipeline (nok8s_attributes— host-level source).0.4.12 -> 0.4.13/0.15.12 -> 0.15.13.Why journald and not the file directory
systemd-pstore.servicealways logs pstore contents to the journal viasd_journal_sendv(), independent ofStorage=— theFILE=field contains the raw dmesg payload,MESSAGE=is the summary. Seesrc/pstore/pstore.c(move_file()). Works with bothStorage=external(default; files + journal) andStorage=journal(journal only).Verification