Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 46 additions & 29 deletions Documentation/ABI/testing/sysfs-driver-asahi-agx-stats
Original file line number Diff line number Diff line change
Expand Up @@ -86,14 +86,15 @@ Description:
older than the age limit below or its epoch ended.
unsupported is a fixed property of the firmware,
taken from the driver's per-firmware match data, and
never changes at runtime: the V25G83 and V26A428
firmwares accept tag 0x10 but never emit it, and no
other record carries the current pstate, so
pstate_status always reads unsupported on G16S and
G17P. utilisation_status is never unsupported on
these firmwares. Numeric metrics are printed only
while their capability is valid; otherwise their
value is the string unsupported.
never changes at runtime: the V25G83 firmware accepts
tag 0x10 but never emits it, and no other record of
it carries the current pstate, so pstate_status
always reads unsupported on G16S. V26A428 reports
its state in other records (see below).
utilisation_status is never unsupported on these
firmwares. Numeric metrics are printed only while
their capability is valid; otherwise their value is
the string unsupported.

util_tiler_percent, util_renderer_percent, util_device_percent
Unsigned integer percent (0..100), from tag 0x13
Expand All @@ -107,7 +108,9 @@ Description:

pstate Logical selector 0..10 from tag 0x10 word 6, on a
firmware whose match data says it emits tag 0x10
(none of the current G16S/G17P firmwares do). Words 1
(none of the current G16S/G17P firmwares do). A
firmware that reports its state in tags 0x19 and
0x1b (V26A428) never has tag 0x10 decoded. Words 1
and 2 form a little-endian u64 firmware timestamp.
The first timestamp establishes a baseline. Invalid
selectors and backwards timestamps remove pstate
Expand All @@ -116,39 +119,53 @@ Description:

requested_frequency_hz, achieved_frequency_hz, busy_ns,
power_mw, temperature_raw
Always unsupported on these firmwares. The statistics
packet does not identify performance table A or B;
a selector is neither a host request nor a measured
frequency. No qualified busy-time duration, power unit
or temperature field is available.
Always unsupported on V25G83 (G16S). No record of it
carries a state, a qualified busy-time duration, a
power unit or a temperature.

M5 firmware (the M5 command set of the shared G17 backend) uses
the same snapshot and keys, with these differences:
The V26A428 firmware, which the A18 Pro (T8140, G17P) and M5
run, uses the same snapshot and keys, with these differences:

pstate_status, pstate
Supported. The firmware reports the performance state
in effect in tag 0x1b, words 1 and 3, which must
agree: an index 1..13 into the board's performance
state table. The end of the interval record before it
(tag 0x19, words 1 and 2) is its firmware timestamp.
A report that names state 0 or a state beyond the
table removes pstate validity. The state reads stale
while the GPU cores are gated.
agree: an index into the performance state table,
1..10 on the A18 Pro, 1..13 on M5. The end of the
interval record before it (tag 0x19, words 1 and 2)
is its firmware timestamp. A report that names state
0 or a state beyond the table removes pstate
validity. While the GPU cores are gated, pstate reads
valid 0: the GPU is off. Power-up starts a new epoch,
stale until the next report. The driver asks for full
performance while work is in flight, which the
firmware takes as a floor, so an A18 Pro at work
normally reads its top state, 10; a lower state there
is a firmware limit, such as a thermal one.

achieved_frequency_hz
The frequency of pstate in the board's performance
state table, in Hz, while pstate_status is valid.
The frequency of pstate in Hz, while pstate_status is
valid, and 0 for state 0. M5: the board's performance
state table. A18 Pro: the device tree operating point
that the state's entry of performance table A names.
Its records do not name the table; at its top state a
sustained FP32 FMA load completes faster than table
B's frequency there allows, so it runs table A.

busy_ns Cumulative u64 since the driver was bound. The tag
0x13 records of this firmware carry a timestamp in
24 MHz ticks (words 1 and 2). Each record credits the
window since the previous record of its epoch at its
device utilisation. A record more than two 8 ms
24 MHz ticks (words 1 and 2), one every 8 ms on M5
and every 16 ms on the A18 Pro. Each record credits
the window since the previous record of its epoch at
its device utilisation. A record more than two
periods after the previous one, or with an earlier
timestamp, only starts a new window, so time the GPU
spent gated is never credited. MEASURED on T6050
spent gated is never credited, and a load sparser
than one period reads low. MEASURED on T6050
(J716c): 871 and 863 ms for three 300 ms compute jobs
at full clock (905 and 919 ms of work).
at full clock (905 and 919 ms of work). MEASURED on
T8140 (J700), 24 traces of GPU stress loads: within
about 20% of the union of the commands' own execution
intervals for loads busy more than half the time.

requested_frequency_hz, power_mw, temperature_raw
Unsupported.
Expand Down
71 changes: 43 additions & 28 deletions Documentation/gpu/asahi-usage-stats.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,15 +57,16 @@ cycles, or multiplying time by a sampled/maximum GPU frequency, would give
incorrect frequency-weighted utilization. Per-client execution-cycle counters
would require a separate hardware measurement source.

On the M5 each engine also has the frequency keys::
On the M5 and the A18 Pro each engine also has the frequency keys::

drm-curfreq-compute: 1620000000 Hz
drm-maxfreq-compute: 1620000000 Hz

``drm-curfreq-<engine>`` is ``achieved_frequency_hz`` of ``agx_stats`` (see
below): the frequency of the performance state the firmware reports. It is
absent while the firmware reports none, as when the GPU is gated.
``drm-maxfreq-<engine>`` is the highest frequency in the board's table. One clock
below): the frequency of the performance state the firmware reports, 0 Hz while
the GPU cores are gated. It is absent while no state is known, as right after
the GPU powers up. ``drm-maxfreq-<engine>`` is the highest frequency of the
performance states. One clock
drives the three engines, so both values are the device's, the same for every
engine and every client. They show the clock at the time of the read and are
not a weight for the engine times: without ``drm-cycles-*`` there is nothing to
Expand All @@ -80,9 +81,9 @@ per valid stage, with no new locks or allocations. The native backends reuse
existing timestamp reads. Classic completion callbacks read two internal cells
per stage; commands without application timestamps also execute two existing
timestamp microsequence operations per stage. Reading fdinfo loads three counters
and does not traverse queues, wait for fences or access hardware. On the M5 a
read also takes the device mutex once and drains the statistics rings, as a read
of ``agx_stats`` does, for the frequency keys.
and does not traverse queues, wait for fences or access hardware. On the M5 and
the A18 Pro a read also takes the device mutex once and drains the statistics
rings, as a read of ``agx_stats`` does, for the frequency keys.

G16S and G17P device utilisation
------------------------------
Expand Down Expand Up @@ -125,26 +126,39 @@ ring on powered cores becomes stale rather than fabricating a zero-utilisation
sample; only gated cores read as zero. Readers
must not convert these percentages into engine nanoseconds or cycles.

M5 performance state and busy time
----------------------------------
Performance state and busy time on V26A428 (A18 Pro and M5)
-----------------------------------------------------------

The M5 firmware runs on the same backend and decoder, and its statistics ring
carries more. ``agx_stats`` reports there:
The V26A428 firmware, which the A18 Pro (T8140, G17P) and the M5 run on the same
backend and decoder, carries more in its statistics ring. ``agx_stats`` reports
there:

* ``pstate``: the performance state in effect, from the firmware's state
reports (tag ``0x1b``, words 1 and 3, which must agree). It is an index into
the board's performance state table, 1 to 13. The interval record before a
report (tag ``0x19``) gives its firmware timestamp. A report that names state
0 or a state beyond the table is not used, and ``pstate_status`` reads
``stale`` while the cores are gated.
* ``achieved_frequency_hz``: the frequency of that state in the board's table,
while ``pstate_status`` is ``valid``.
* ``busy_ns``: the M5 utilisation records carry their own timestamps (words 1
and 2, 24 MHz ticks), so each record closes a window of known length, which
is credited at the record's device utilisation. A record that follows the
previous one by more than two 8 ms periods, or whose timestamp went back,
only starts a new window: time the GPU spent gated is never credited. The
total keeps counting across power epochs.
the performance state table: 1 to 10 on the A18 Pro, 1 to 13 on the M5. The
interval record before a report (tag ``0x19``) gives its firmware timestamp.
A report that names state 0 or a state beyond the table is not used. While
the cores are gated the GPU is off and ``pstate`` reads a valid 0; power-up
starts a new epoch, ``stale`` until the next report.
* ``achieved_frequency_hz``: the frequency of that state while
``pstate_status`` is ``valid``, 0 for state 0. The M5 takes it from its
board's table. The A18 Pro takes the device tree operating point that the
state's entry of its performance table A names: its records do not name the
table, but at its top state a sustained FP32 FMA load completes faster than
table B's frequency there allows.
* ``busy_ns``: the utilisation records carry their own timestamps (words 1 and
2, 24 MHz ticks), one every 8 ms on the M5 and every 16 ms on the A18 Pro, so
each record closes a window of known length, which is credited at the
record's device utilisation. A record that follows the previous one by more
than two periods, or whose timestamp went back, only starts a new window:
time the GPU spent gated is never credited, and a load sparser than one
period reads low. The total keeps counting across power epochs.

The driver asks for full performance while work is in flight, which the
firmware takes as a floor for its state. An A18 Pro at work therefore normally
reports its top state; a lower one is a firmware limit, such as a thermal one.
The tag ``0x19`` records are periodic samples, not busy intervals, and do not
contribute to ``busy_ns``.

``requested_frequency_hz``, ``power_mw`` and ``temperature_raw`` stay
unsupported. ``busy_ns`` is device-wide, as on G13 and G14; it does not change
Expand All @@ -153,10 +167,11 @@ fdinfo's per-client ``drm-engine-*`` counters.
Deferred shared power telemetry
------------------------------

Current pstate and requested/achieved frequency remain unsupported for these
firmware graphs (the M5 firmware reports its state in the statistics ring, see
above). The shared-power snapshot ABI is not sufficiently specified
to implement a read-only reader. For **each** of G16S and G17P, it still needs:
Current pstate and requested/achieved frequency remain unsupported on G16S
(V25G83); V26A428 reports its state in the statistics ring, see above. The
shared-power snapshot ABI is not sufficiently specified to implement a
read-only reader, which G16S would need, and which would also name the A18
Pro's table directly. For **each** of G16S and G17P, it still needs:

* The firmware-version-to-layout mapping, containing allocation and bounds,
offsets, widths and stride of the generation/seqlock word, live pstate,
Expand All @@ -176,7 +191,7 @@ A future implementation needs bounded seqlock retries and lifecycle invalidation
before publishing these fields. Tag ``0x10`` remains decoded for diagnostics but
is not emitted by the currently qualified firmware graphs. A dormant pstate
record, a host request or a command timer cannot provide achieved frequency.
``drm-curfreq-gpu`` must wait for a qualified shared snapshot.
On G16S, ``drm-curfreq-*`` must wait for a qualified shared snapshot.

Hardware acceptance
-------------------
Expand Down
8 changes: 2 additions & 6 deletions drivers/gpu/drm/asahi/g17/channel.rs
Original file line number Diff line number Diff line change
Expand Up @@ -92,16 +92,12 @@ pub(crate) struct Rings {
}

impl Rings {
/// `m5`: the firmware runs the M5 protocol, whose statistics records are decoded too.
/// `m5`: the firmware runs the M5 protocol ([`super::stats::protocol`]).
pub(crate) fn new(graph: super::fw::variant::InitGraph, m5: bool) -> Result<Self> {
Ok(Self {
quarantined: [[false; 2]; 2],
reports: super::reports::Reports::new()?,
telemetry: if m5 {
super::telemetry::Telemetry::m5()
} else {
super::telemetry::Telemetry::default()
},
telemetry: super::telemetry::Telemetry::new(super::stats::protocol(graph, m5)),
max_age_ns: super::stats::max_age_ns(graph),
suspended: false,
discard_statistics: [false; 2],
Expand Down
36 changes: 32 additions & 4 deletions drivers/gpu/drm/asahi/g17/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1816,12 +1816,10 @@ impl Gpu {
directory.read_only_file(c"hardware_data", reports::HardwareDataView(shared.clone())),
GFP_KERNEL,
)?;
let frequencies = Self::state_frequencies(dev, cfg, variant, id.num_clusters)?;
let stats = Arc::new(
crate::stats::StatsSnapshot {
g17_reader: Some(stats::Reader(
shared.clone(),
cfg.m5.map(|board| board.perf_states),
)),
g17_reader: Some(stats::Reader(shared.clone(), frequencies)),
..Default::default()
},
GFP_KERNEL,
Expand Down Expand Up @@ -1932,6 +1930,36 @@ impl Gpu {
cells.as_slice().try_into().ok()
}

/// The frequency of each performance state the firmware reports
/// ([`stats::reports_pstate`]): M5's from its board, the A18 Pro's from the operating
/// points its performance table A names. The A18 Pro's records do not name the table, but
/// at its top state, where it runs while work is in flight, a sustained FP32 FMA load
/// completes faster than table B's frequency there allows. `None` where the firmware
/// reports no state.
fn state_frequencies(
dev: &AsahiDrmDevice,
cfg: &'static Config,
variant: &fw::variant::Variant,
clusters: u32,
) -> Result<Option<stats::StateFrequencies>> {
if !stats::reports_pstate(variant.graph, variant.m5_commands) {
return Ok(None);
}
if let Some(board) = cfg.m5 {
return Ok(stats::StateFrequencies::new(
board.perf_states.iter().map(|state| state.0),
));
}
let perf = cfg.perf()?;
let opps = initdata::InitData::opps(dev, cfg.opps, clusters)?;
let mut mhz = [0u32; fw::initdata::PERF_STATES];
for (slot, &index) in mhz.iter_mut().zip(perf.states_a.iter()) {
let opp = opps.get(usize::from(index)).ok_or(EINVAL)?;
*slot = u32::try_from(opp.freq_hz / 1_000_000)?;
}
Ok(stats::StateFrequencies::new(mhz.into_iter()))
}

/// Returns the highest advertised operating-point frequency without changing table order.
fn max_frequency_khz(dev: &AsahiDrmDevice) -> Result<u32> {
let node = dev.as_ref().as_ref().of_node().ok_or(EIO)?;
Expand Down
Loading
Loading