Skip to content

Add chip label to DRM metrics - #3572

Open
Deezzir wants to merge 1 commit into
prometheus:masterfrom
Deezzir:add-drm-chip
Open

Add chip label to DRM metrics#3572
Deezzir wants to merge 1 commit into
prometheus:masterfrom
Deezzir:add-drm-chip

Conversation

@Deezzir

@Deezzir Deezzir commented Mar 4, 2026

Copy link
Copy Markdown

Issue

Add chip label to the node_drm_card_info metric.

Solution

Extend the DRM collector to have the chip label from the ClassDRMCardAMDGPUStats.
The change was also proposed in the procfs repo, and the PR was merged.

Context

The DRM metrics are impossible to relate to the hwmon metrics, which export other helpful information about AMD GPUs. The extension will allow us to relate metrics from both hwmon and drm and, in turn, create a better GPU Dashboard for AMD with proper filtering and labelling.

Example metrics:

# DRM
node_drm_card_info{card="card0",chip="0000:9d:00_0_0000:9e:00_0",memory_vendor="unknown",power_performance_level="auto",unique_id="",vendor="amd"} 1

# HWMON
node_hwmon_chip_names{chip="0000:9d:00_0_0000:9e:00_0",chip_name="amdgpu"} 1
node_hwmon_fan_enable{chip="0000:9d:00_0_0000:9e:00_0",sensor="fan1"} 0
node_hwmon_fan_max_rpm{chip="0000:9d:00_0_0000:9e:00_0",sensor="fan1"} 6900
node_hwmon_fan_min_rpm{chip="0000:9d:00_0_0000:9e:00_0",sensor="fan1"} 1800
node_hwmon_fan_rpm{chip="0000:9d:00_0_0000:9e:00_0",sensor="fan1"} 2035
node_hwmon_fan_target_rpm{chip="0000:9d:00_0_0000:9e:00_0",sensor="fan1"} 2035
node_hwmon_freq_freq_mhz{chip="0000:9d:00_0_0000:9e:00_0",sensor="mclk"} 1500
node_hwmon_freq_freq_mhz{chip="0000:9d:00_0_0000:9e:00_0",sensor="sclk"} 214
node_hwmon_in_volts{chip="0000:9d:00_0_0000:9e:00_0",sensor="in0"} 0.85
node_hwmon_power_average_watt{chip="0000:9d:00_0_0000:9e:00_0",sensor="power1"} 6.075
node_hwmon_power_cap_default_watt{chip="0000:9d:00_0_0000:9e:00_0",sensor="power1"} 35
node_hwmon_power_cap_max_watt{chip="0000:9d:00_0_0000:9e:00_0",sensor="power1"} 35
node_hwmon_power_cap_min_watt{chip="0000:9d:00_0_0000:9e:00_0",sensor="power1"} 0
node_hwmon_power_cap_watt{chip="0000:9d:00_0_0000:9e:00_0",sensor="power1"} 35
node_hwmon_pwm{chip="0000:9d:00_0_0000:9e:00_0",sensor="pwm1"} 49
node_hwmon_pwm_enable{chip="0000:9d:00_0_0000:9e:00_0",sensor="pwm1"} 2
node_hwmon_pwm_max{chip="0000:9d:00_0_0000:9e:00_0",sensor="pwm1"} 255
node_hwmon_pwm_min{chip="0000:9d:00_0_0000:9e:00_0",sensor="pwm1"} 0
node_hwmon_sensor_label{chip="0000:9d:00_0_0000:9e:00_0",label="edge",sensor="temp1"} 1
node_hwmon_sensor_label{chip="0000:9d:00_0_0000:9e:00_0",label="mclk",sensor="freq2"} 1
node_hwmon_sensor_label{chip="0000:9d:00_0_0000:9e:00_0",label="sclk",sensor="freq1"} 1
node_hwmon_sensor_label{chip="0000:9d:00_0_0000:9e:00_0",label="slowPPT",sensor="power1"} 1
node_hwmon_sensor_label{chip="0000:9d:00_0_0000:9e:00_0",label="vddgfx",sensor="in0"} 1
node_hwmon_temp_celsius{chip="0000:9d:00_0_0000:9e:00_0",sensor="temp1"} 42
node_hwmon_temp_crit_celsius{chip="0000:9d:00_0_0000:9e:00_0",sensor="temp1"} 97
node_hwmon_temp_crit_hyst_celsius{chip="0000:9d:00_0_0000:9e:00_0",sensor="temp1"} -273.15000000000003

@Deezzir Deezzir changed the title Add drm chip Add chip label to DRM metrics Mar 19, 2026
@Deezzir Deezzir changed the title Add chip label to DRM metrics Add chip label to DRM metrics Mar 19, 2026
@Deezzir

Deezzir commented Mar 19, 2026

Copy link
Copy Markdown
Author

@SuperQ and @discordianfish PTAL

@SuperQ

SuperQ commented Apr 3, 2026

Copy link
Copy Markdown
Member

This needs a rebase.

@Deezzir

Deezzir commented Apr 6, 2026

Copy link
Copy Markdown
Author

@SuperQ done

@Deezzir
Deezzir force-pushed the add-drm-chip branch 3 times, most recently from d3422a4 to ca0ae3e Compare April 13, 2026 01:00
@Deezzir

Deezzir commented Apr 28, 2026

Copy link
Copy Markdown
Author

@SuperQ PTAL

@Deezzir
Deezzir force-pushed the add-drm-chip branch 3 times, most recently from 53101c6 to 8e4666c Compare June 2, 2026 17:46
@Deezzir
Deezzir force-pushed the add-drm-chip branch 2 times, most recently from 209c02c to 9154335 Compare June 15, 2026 18:59

@rtreffer rtreffer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be simplified by using just one metric and exporting an empty string if missing (which is a common pattern).

Other than that: having common join keys would be helpful.

Comment thread collector/drm_linux.go
Comment thread collector/drm_linux.go Outdated
Comment thread collector/drm_linux.go

@SuperQ SuperQ left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @rtreffer, let's keep this simple and put chip on the existing info metric.

@Deezzir
Deezzir force-pushed the add-drm-chip branch 4 times, most recently from 3ab2175 to c3de2d0 Compare July 20, 2026 20:28
@Deezzir

Deezzir commented Jul 20, 2026

Copy link
Copy Markdown
Author

@SuperQ, @rtreffer I've simplified the logic, thanks

@SuperQ
SuperQ requested a review from rtreffer July 21, 2026 04:50
@Deezzir

Deezzir commented Jul 21, 2026

Copy link
Copy Markdown
Author

@SuperQ Can you reapprove plz, had to rebase

@rtreffer rtreffer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SuperQ I think there is one gap/risk in the cleanup/naming diverging between hwmon and drm. Aligning that would probably be significant work and I am not 100% convinced it will work easily or if it is relevant in the context of GPUs.

I think this is cool overall.

Comment thread collector/drm_linux.go
func chipName(s sysfs.ClassDRMCardAMDGPUStats) string {
// generate a chip name based on the deviceType and devName
cleanDevName := cleanMetricName(s.DevName)
cleanDevType := cleanMetricName(s.DevType)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should relate to https://github.com/prometheus/node_exporter/blob/master/collector/hwmon_linux.go#L517

However both parts come from different sources and will diverge.
This is annoying after #3646

I am not sure this is something that could actually harm us, but calling it out as an issue overall. Making both implementations align is probably not worth the effort.

Signed-off-by: Deezzir <deezzir@gmail.com>
@Deezzir

Deezzir commented Aug 6, 2026

Copy link
Copy Markdown
Author

@SuperQ Can you approve the workflows plz?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants