Repository navigation
Deep sleep (suspend-to-RAM) for the M4 MacBook Air (T8132, J713) at EL2 - #1314
Conversation
RyanTheTide
left a comment
There was a problem hiding this comment.
What this branch needs before it can be merged, on head 715a022:
-
Signed-off-by. None of the 19 commits from
297d7a47d7c9(iommu: apple-dart: don't require DAPF ranges on a DART that has none) to715a0229e1b8(soc: apple: mcc: write T8132's cache back for a deep sleep), excluding the six signed radio commits6a720bcba1b3..b37219c61cbd, carries aSigned-off-by:line. Please add yours to each (git rebase -x 'git commit --amend --no-edit -s' feat/t8140-s2r) and push. -
Rebase onto the current
feat/t8140-s2r(31fd6d6). The branch forked at 5b1a671; the base has since merged #1311 and the merge now conflicts indrivers/pci/controller/pcie-apple.c(with b724b53 "PCI: apple: attach and retain all host power domains in s2idle" and f31c0a3 "PCI: apple: preserve probe without power-management support"). -
Build break.
bringup_va_markinarch/arm64/kernel/sleep.S(added by b6f4482) loadst8132_spmi_va, which is defined indrivers/soc/apple/t8140-sleep.cand only exists whenCONFIG_APPLE_T8140_SLEEP=y. Any config without that driver built in fails to link (ld.lld: error: undefined symbol: t8132_spmi_va, see the kunit job of https://github.com/aurora-silicon/linux/actions/runs/42695306042, which builds this branch's code). This goes away with item 4. -
Remove the bring-up parts listed in the description:
669e8db10c18(DO NOT MERGE workarounds), the progress marks and panic/die notifiers inb6f448244d44, thebringup_*module parameters, andcpu-reset.obuilt withoutKEXEC(enableCONFIG_KEXECinstead). -
arch/arm64/boot/dts/apple/t8132-j713-apcie-tunables.dtsiis 295 lines of calibration copied from the ADT. Please have the bootloader pass these on instead of carrying the table in the tree, as the description already plans. -
checkpatch on every commit.
-
The radio commits
6a720bcba1b3..b37219c61cbdare also open as #1385, #1378, #1381, #1382, #1383 and #1384. Keep them in one place so the two copies cannot diverge.
This PR is based on #1208, which has to land first.
|
Please merge PR 1378, 13{81-84}, 1414-20 into this branch before diverging it too much - should work smoothly on your machine, tested on what I believe is an identical machine! |
715a022 to
161c7d0
Compare
|
Thanks for the list. Pushed an updated branch; where each item stands:
Also fixed on the way: the 17 host tests the radio and dock series added weren't listed in Tested on the J713, as a full system with radios, GPU and cache on: 8 of 8 sleeps with a 256 MB check (no bad blocks), radios and GPU back after each, 2 of 2 last-moment refusals rolled back; then 8 of 8 again with the APCIe tables from the bootloader. Every commit builds; the host tests, the sleep, MCC and apple_sep KUnit suites, and CHECK_DTBS on the T8132 boards (no new warnings) pass. |
d9ec74b to
17161ff
Compare
|
A correction to my reply above. "The host tests ... pass" covered only the 17 tests the radio and dock series added and the suites that read the sleep driver; I hadn't run the whole
None of them is a behaviour change. Each mock now has the new state, inert as on other machines, in the commit that first needs it. The whole runner now passes on #1334 (228 of 228 runs) and at every commit of this branch: 228 of 228 up to the last commit, which registers the series' 17 new tests, and 245 of 245 at the head. Before the registration the runner refuses to start, so for those per-commit runs that check was relaxed and the unregistered tests were skipped. Without the registration (on #1420 alone, say) it stops before running anything, which is why none of this showed earlier. CI doesn't run it here either: |
1.821 is 1.820 plus twelve patches, with a new J604 device tree. 0265-0269 port the T8132 suspend-to-RAM sequence Aurora traced from macOS (aurora-silicon/linux#1314 by Bartosz Nitka and Diljit Singh; trace aurora-silicon/aura#109) behind apple_s2r.t8132_sequence, which is off by default. 0270-0276 carry Aurora's GPU and power-manager deep-sleep behaviour, which acts only in a deep sleep. Deep sleep is still not working: the sequence has not run on the J604, "deep" is not offered, and suspend-to-idle is the only sleep state. boot.bin with the 1.821 device tree is 4,704,706 bytes, confirmed by a trial boot. 276 patches. STATUS.md: the 1.821 quick-gate checks and the deep sleep status. UPSTREAM.md lists the Aurora work now carried. README.md, CREDITS.md and kernel/RENUMBER.md follow. Signed-off-by: Dylan Ironside <dylan@dylanironside.io>
Kernel 1.823 is 1.821 plus three patches, with a new J604 device tree: 0277 points the P-cluster power-gate word at +0x60 (node still disabled), 0278 adds sdhci_pci.gl9755_runtime_pm for the SD card reader (off by default), and 0279 (Bartosz Nitka, from aurora-silicon/linux#1314) powers the internal display controller down for a deep sleep. 279 patches. m1n1 1.611 adds aurora-silicon/aura#112 (Diljit Singh): the ten APCIe tables are published on the J604. omarchy-mac 2.14 keeps Chromium's hardware decode for H.264 and VP9 profile 0 only; AV1, 10-bit VP9 and HEVC play in software in Chromium, whose V4L2 path cannot lay out 10-bit P010 frames. boot.bin is 4,704,706 bytes, confirmed by trial boots. Deep sleep is still not working; suspend-to-idle is the only sleep state. kernel/patches/0263 now says in a bracket note that one co-author line of the upstream message is not reproduced; UPSTREAM.md states that convention once. Signed-off-by: Dylan Ironside <dylan@dylanironside.io>
17161ff to
19de60a
Compare
Shared radio PERST resets private BAR windows that generic PCI saved state does not cover. Restore the windows and subsystem control in resume_noirq, before interrupts or UNQUIESCE access the retained device. Without this, Wi-Fi returns after sleep but Bluetooth HCI commands time out. Restoring the windows permits Read Local Version to complete. Coordinate QUIESCE and recovery work with the managed host transaction, leaving other controllers on their existing path. Cover config failures and retained-path skips with a fault-injection fixture. Signed-off-by: Erik Bourget <erik@ebourget.net> [niteria: bcm4377-resume-tests' mocks now have the radio sleep hooks] Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
A consumer-switched power state can have a bus bridge in front of the block it powers. The bridge's two link words must have bit 0 cleared before the state powers off and set again once it is on, and they are only accessible while the state is on. Describe where they are: a phandle to the block's node, the index of its register range and the offset of the words in that range. The PMGR driver can then do that as part of the consumer's power-off and power-on. The T8132 external display system domains need it. Signed-off-by: Erik Bourget <erik@ebourget.net> Signed-off-by: Bartosz Nitka <niteria@gmail.com>
A T8132 external display system domain must not be powered off with the bus bridge of its block still linked. When a consumer-switched state has apple,bridge-link, clear bit 0 of the two link words just before the state powers off, and set it again once the state reads fully on, before the retention words are restored. The words are touched only while the state is on. Their address comes from the named block's register range and the offset in the property. This extends the consumer-switched power states. This builds on work by Bartosz Nitka. Signed-off-by: Erik Bourget <erik@ebourget.net> [niteria: credit moved from Based-on-work-by into the text] Signed-off-by: Bartosz Nitka <niteria@gmail.com>
A driver that takes its power domains down and up itself around a system sleep, through the genpd suspend and resume helpers, has to know that each one really reached the state it asked for before it goes on: the next step either removes power from, or touches, the block behind it. Add a helper that polls the state word of a virtual power-domain device for fully on or fully off. It refuses a device whose power domain is not a genpd, and firmware-owned and inherited-on states, which the driver does not control. Signed-off-by: Erik Bourget <erik@ebourget.net> Signed-off-by: Bartosz Nitka <niteria@gmail.com>
…er power-off apple_pmgr_ps_consumer_on() walks the consumer's chain parent first and stops at the first state it cannot switch. A state that is not off gives -EALREADY. After a consumer power-off that stopped partway, for example with the CPU state off and the frontend refusing because it was not fully on, a rollback power-on stops at the system state, which never went off, and leaves the CPU state off. Treat -EALREADY as done in the power-on walk, so that a rollback after a partial power-off switches back on every state that went off. A consumer rolling back also has to know whether its own block went through reset, which decides how its coprocessor is started again. Add apple_pmgr_ps_consumer_is_off() for that: whether the first state of its chain, its own domain, was powered off. Signed-off-by: Erik Bourget <erik@ebourget.net> Signed-off-by: Bartosz Nitka <niteria@gmail.com>
…ains A deep sleep powers the external display coprocessors off, and they have to be stopped first and started again afterwards. Mark the CPU, front-end and system domains of both as consumer-switched, with the CPU domains' power-gate retention words and the system domains' bridge links, so the coprocessor's driver can power the chain off after it has stopped the firmware and on again before it restarts it. The bridge links name the external DCP whose block they are in, so they live in the board file; the PMGR file is also built into the platform data, which has no DCP nodes. The domains stay firmware-owned otherwise: nothing changes at runtime. This builds on work by Bartosz Nitka. Signed-off-by: Erik Bourget <erik@ebourget.net> [niteria: credit moved from Based-on-work-by into the text; the domains' consumer-switched marks and retention words are in the J713's board file too, not t8132-j713-pmgr.dtsi, which the J715 shares] Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
A coprocessor that was powered off with apple_rtkit_poweroff() and then lost its power domains has no session left to wake: it has to boot again. Add a helper for that: reset the session state, warm and port-only restart state included, let the caller start the CPU, then start the mailbox, enable receive and run the normal boot handshake. The CPU is started before the mailbox is touched at all. On T8132 the ACIO mailbox registers are only clocked once the caller's start step has run, as at the first start, and accessing them earlier takes an SError. On failure the caller has to stop the CPU it started. Signed-off-by: Erik Bourget <erik@ebourget.net> [niteria: rtkit-oslog-tests now copies apple_rtkit_reset_session() as well] Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
…DARTs On T8132, the hosts of some DARTs power them off and restore them around a deep sleep: the ACIO NHIs, the tunneled PCIe ports and the external display coprocessors. They need the same clock protection slice as the internal display DARTs. Extend the description of apple,clock-protection to them. Signed-off-by: Erik Bourget <erik@ebourget.net> Signed-off-by: Bartosz Nitka <niteria@gmail.com>
A deep sleep with a Thunderbolt dock attached powers off the ACIO and tunneled PCIe DARTs and the external display coprocessors' DARTs, and their hosts have to bring them back before the first requester runs again. Runtime PM cannot do it: the host decides when its requesters are stopped and when its clocks are back. Add an owner interface for the T8110 DARTs whose device tree node names a clock protection slice, other than the internal display ones. The host binds once, then around a deep sleep calls prepare (before any suspend callback), off (requesters stopped, clocks still on), on (clocks back, before any DMA) and finish. off closes the DART to hardware commands, saves the stream table, tunables and partition words, disables the streams, flushes, and releases the clock-protection slice. on restores the slice and the saved state and reopens the DART. The generic suspend and resume callbacks leave an owned DART alone, and a DART without a bound owner takes no extra lock on its map paths. While a DART is closed, its page tables are still updated, in memory only: map, unmap, attach, release and free never touch its registers. Those writers are counted. Reopening, after a restore or after an off that failed before changing anything, waits for them, and if anything changed meanwhile flushes the whole TLB before the DART opens. finish always ends the transaction. A DART that was not restored stays closed, its registers unused, until reboot, even once its owner is gone; nothing panics on it. Things learned on the J713 that the restore accounts for: - The tunneled PCIe DART has 256 stream slots where Linux allocates 64. The whole bank, as the hardware reports it, is saved, the reserved bypass streams included. - An external display DART's clock-protection slice word reads zero after the sleep. It is saved at off and written back at on. - A locked DART can come back with other bits set in the root of a stream that is not enabled and whose root was, and still is, invalid. That is accepted. Every control register and every valid root must match, and a locked DART is never rewritten. The slice handling follows the internal display DART's power-down. A failed restore dumps the registers that differ. Add host-side models of the register sequence and of the ownership rules, each with negative controls. This builds on work by Bartosz Nitka. Signed-off-by: Erik Bourget <erik@ebourget.net> [niteria: credit moved from Based-on-work-by into the text; checkpatch fixes in the host-test harnesses; the DART fw and range host tests' mocks now have cio_active and the writer gate's return value] Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
…ction A deep sleep with a dock or an external display attached powers off the DARTs of the ACIO NHIs, the tunneled PCIe ports and the external display coprocessors, and their hosts restore them. Give each of them its clock protection slice, which also marks it as one its host may power-cycle. Signed-off-by: Erik Bourget <erik@ebourget.net> Signed-off-by: Bartosz Nitka <niteria@gmail.com>
On T8132 the PHY returns to its off mode when the DWC3 host on the port stops. With a USB4 or Thunderbolt link up that also drops the CIO lanes and with them every tunnel through the dock, although the cable is still connected and only the USB host went away, as it does for a system sleep. Leave a PHY that is in a CIO mode as it is when its USB host stops, and keep it busy for runtime PM. The Type-C mux still takes it down on a real disconnect, an ordinary USB mode still ends with its host, and a CIO mode whose last transition failed is still taken down. This applies on every T8132 board (J604, J623/J624, J713/J715, J773g), outside the J713 deep-sleep opt-in. Extend the USB3 host model with the T8132 cases and three mutations. Signed-off-by: Erik Bourget <erik@ebourget.net> Signed-off-by: Bartosz Nitka <niteria@gmail.com>
A deep sleep with a USB4 link up has to power the PHY down and bring it back in CIO mode with the same lane configuration. The system suspend callback refuses a deep sleep while the common block is in use, and only the ACIO host knows when the link's users are stopped. Add an interface for that host: prepare records the mode and orientation and lets the system suspend through, off stops the DP tunnel clock, saves the lane mode and crossbar and powers the PHY down, on powers it up and restores them, and finish ends the transaction. The tunnel clock is started again for the DP inputs that were using it. prepare takes part in the same sleeps as its host (a suspend to RAM). A tunnel clock that does not stop refuses the sleep before anything is powered down, and the DP inputs using it get their clocks back. After the power cycle the tunnel clock's command register reads with its acknowledge bit set and no request pending. The first clock start after it, in the restore or whenever a DP tunnel next asks for the clock, accepts that and issues the release command a stop ends with before it starts the clock, so the start's own acknowledge is real. Add a host-side model of the transaction with negative controls. Signed-off-by: Erik Bourget <erik@ebourget.net> [niteria: checkpatch fixes in the host-test harness; the t6030-clock and apple-dcp mocks now have the CIO sleep state, and t6030-clock two stale-ACK cases] Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
The tunneled PCIe port behind a Thunderbolt dock loses its clocks and its DART in a deep sleep, and nothing brings them back. With pcie_apple.cio_s2r=1 on the J713, the port's DART is owned by the port instead of linked for runtime PM, and the ACIO host prepares and finishes a native transaction around the sleep. In it the port's noirq suspend drains the port, closes config space, asserts the tunnel reset and PERST, powers the DART down and stops the port clock. The noirq resume runs the cold port setup again (tunables, port reset table, PERST and clock, root tunables, interrupt routing), restores the requester and MSI routes and the DART, releases the tunnel reset and waits for the link. If the suspend is abandoned after the local PERST, the same full setup runs. Config space is closed under a lock that config accesses hold from map_bus() to the access. A port whose link does not come back, for a dock unplugged or powered off during the sleep, ends like a surprise removal: the port stays stopped, its hierarchy is marked disconnected and removed with the tunnel after resume, and the DART transaction still ends. When the ACIO host already knows the dock is gone, the port does not wait for the link. A deep sleep that the ACIO host did not prepare for this port is refused in .prepare, before any other device suspends. T8132 acknowledges the tunnel PERST by setting bit 1, and the link wait is bounded to one second. The parameter defaults to off: the ACIO host, PHY and DART parts have to be present and enabled together. Signed-off-by: Erik Bourget <erik@ebourget.net> Signed-off-by: Bartosz Nitka <niteria@gmail.com>
An Apple host router hands its connection manager sixteen words of state before it is powered off and takes them back after it has been restarted. Both directions are vendor router operations, 0x8000 to save and 0x8001 to restore, with sixteen data words and no metadata. Add the two operations for the root switch. The words are opaque: they are kept as returned and written back unchanged. A failed restore leaves the saved copy in place so the caller can decide what to do. Add a host-side model of the operation with negative controls. Signed-off-by: Erik Bourget <erik@ebourget.net> Signed-off-by: Bartosz Nitka <niteria@gmail.com>
On Apple silicon a deep sleep powers the host router off while a dock stays attached, and the PCIe and USB devices behind the dock stay bound through it. The normal suspend path cannot express that: it either keeps the routers awake or tears the tunnels down. Add a connection-manager transaction for an NHI that owns such a power cycle, in four calls: - prepare, before any device suspends: check the topology, close hotplug and release the DisplayPort tunnels while their owners still run. A display does not survive the host router's power cycle and is paired again afterwards as after a plug. Every other tunnel is paused with its hop entries saved. - sleep, with the tunnels' users stopped: deactivate the paths, put the device routers to sleep, then the host router through the NHI's router_sleep hook, save its state and stop the control channel. - wake, after the NHI has restarted the router: start the control channel, enumerate the router again, restore its state, let the NHI start its links through router_wake and wait for them, resume the routers and replay the saved hops and adapter settings. - complete, after every device has resumed: reopen hotplug, retire what the wake found gone and pair the released DisplayPort resources again. A dock unplugged, swapped or powered off during the sleep is an unplug, not a failure: the routers that did not come back are marked unplugged, their tunnels are skipped by the replay and retired at complete, after their users resumed, and the NHI is told when the host's PCIe tunnel was among them. Only the host side has to come back. The host router's UID is read from its configuration space before the sleep and compared with what it reads after it. An Apple host router has no UID there (its identity comes from the device tree DROM), so both read zero; comparing against the DROM UID would always fail. A failed sleep unwinds through wake. Once the host router took its vendor sleep, only the NHI's restart wakes it; a state save that fails after that does not unwind, and the wake then brings the router back as after a cold start. If the unwind fails, the tunnels' users would resume onto dead paths, so it panics. The stock resume path is unchanged. Add a host-side model of the transaction with negative controls. Signed-off-by: Erik Bourget <erik@ebourget.net> [niteria: checkpatch fixes in the host-test harness; the CIO suspend fails only on a negative tb_switch_clx_disable(), which returns the CL states it disabled] Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
With thunderbolt_apple.cio_s2r=1 on the J713, the NHI owns the whole power cycle of its ACIO block around a deep sleep with a dock attached. In prepare it closes the connection manager, checks that only the control rings are in use, waits for the released displays' routes to go down and opens the PHY, DART and tunneled PCIe transactions. In its noirq suspend, once the PCIe port reports stopped, it puts the routers to sleep, powers the NHI DART down, powers the controller off, holds the LSTX pin, takes the four ACIO power domains down in order and powers the PHY down. The noirq resume does the reverse: PHY, domains, a cold restart of the controller, tunables, the DART, then the connection manager's wake. Complete closes the three transactions and reopens the connection manager. The host router is told to sleep the way its firmware expects: the router's own sleep bit (ROUTER_CS_5.SLP, with its PCIe and USB wake bits clear) first, then the vendor enter-sleep request, then a wait for the controller to report its sleep state. The vendor request on its own makes the firmware assert. After the restart the router is given its lane adapters unlocked and the cable word again, as at the first start, which is what starts link training. A dock lost during the sleep is retired as an unplug once the host side is back; if the host's PCIe tunnel went with it, the tunneled port is told not to wait for its link. A failure after the routers were changed recovers through a restart of the controller: the router only wakes from its vendor sleep that way. The ACIO reset cannot be asserted again, so the four domains are taken down first if they were not already, which resets the CPU, then the same power-on path runs. If a domain stays on, or the NHI DART could not be saved before the restart, or the recovery fails, the PCIe and USB devices behind the dock cannot be resumed safely, so it panics. A DART that is briefly busy with a mapping is waited for first. The power-on path replays the T8132 ACIO start sequence the driver already has for a cable connect. The parameter defaults to off. Without the PHY reference or the DART, the NHI logs it and runs without its deep-sleep transaction. Add host-side models of the ownership rules and of the dock's PM callback ordering. This builds on work by Diljit Singh. Signed-off-by: Erik Bourget <erik@ebourget.net> [niteria: credit moved from Based-on-work-by into the text; checkpatch fixes in the host-test harness. On every host, prepare pauses the DP-IN retries and defers a tunnel teardown's PCIe-C quiesce to complete, for every sleep. Only a sleep that may power-cycle the router, one with resume recovery as before or the native deep sleep, expects the PCIe tunnel back, and a failed CIO prepare restores that expectation; the host test covers both] Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
The built-in T8132 PMGR policy is shared by every T8132 machine on the firmware path. Some of its entries describe one board only, such as the J713's ACIO power states, and must not enable those states on the others. Let an entry name the machine it applies to with apple,policy-machine. On every other machine it is treated as absent, as before the entry existed. Signed-off-by: Erik Bourget <erik@ebourget.net> Signed-off-by: Bartosz Nitka <niteria@gmail.com>
… PHY On the J713 a deep sleep powers the USB4 host router off with its Type-C PHY, and the router's host takes the PHY through the sleep with it. Let the router name that PHY. Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
The ACIO host powers its Type-C PHY down and up itself around a deep sleep with a dock attached, so it needs a reference to it. Add the PHY of each port to its ACIO node as "cio". Move the ACIO and tunneled PCIe power-state overrides into their own file and include it from the built-in T8132 platform data as well, so the policy compiled into the kernel names the same providers as the board's device tree. Those entries name the J713, so the other T8132 boards keep these states as before. Add a check for both. Signed-off-by: Erik Bourget <erik@ebourget.net> [niteria: the PHY references are in the J713's board file, not t8132-j713-usb4.dtsi, which the J715 shares] Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
A deep sleep powers the J713's external display coprocessors off. Each one is adopted running from the boot loader and until now was left running across a sleep, which is only valid for suspend-to-idle: its DARTs refuse a deep sleep, and the sleep controller refuses one with a bound display coprocessor still running. Stop an external DCP that has a session in its deep-sleep suspend, as the internal one is powered down: release the AP link, let RTKit idle, stop the CPU, power its display DART and then its firmware DART down through their owner interface, and power its CPU, front-end and system domains off. In resume, power the domains on, restore both DARTs and start the CPU again through RTKit's retained wake. The firmware starts from its reset vector on the image it kept, and the display comes back with its next enable. The PMGR chain is asked first. A refusal before RTKit idles changes nothing and keeps the session. A failure after the CPU stopped is undone in reverse, and the sleep is refused: a CPU whose domain went off starts again through the retained wake, one that was only stopped runs on and gets RTKit's ordinary wake. A failed restart still ends both DART transactions and fails the session. The sequence is the internal DCP's deep-sleep power-down applied to the external ones, for the T8140 sleep controller's T8132 support. This covers a DCP with nothing attached too: it has a session from adoption and is stopped and restarted the same way. The DART owners are bound at probe. Binding adds a device link, which must not happen from a PM callback. A board whose DARTs have no owner (the J715 for now) keeps refusing a deep sleep with an external display. This builds on work by Bartosz Nitka and Diljit Singh. Signed-off-by: Erik Bourget <erik@ebourget.net> [niteria: credit moved from Based-on-work-by into the text; a failed stop ends the sleep's PM epoch at once, as resume would: the PM core doesn't resume a device whose suspend failed] Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
The J713 native radio and docked deep-sleep series added seventeen host tests without listing them in run-host-tests.py, and the runner fails a run on any test it doesn't list. List them. Three of them import sleep_test_utils from their own directory, which python3 -I, as the runner runs every test, leaves off sys.path: put it back. t8132-usb4-policy-tests.py needed the Python libfdt module and trees built beforehand. Build the T8132 platform data and the J713 and J715 trees with this tree's dtc instead, as the other device-tree tests do, and read them with a small flattened-tree reader. On the trees built here it finds the same ten USB4 PMGR providers per board as the libfdt version did, its reader agrees with libfdt on every node and property, and disabling any one of the ten in the policy fails it. Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
025305c to
04f43c7
Compare
|
Rebased onto
Tested at this head:
Good to merge from my side once CI is green on this head. |
|
Re-review of
The #1769 port on T8140. The new per-SoC fields reproduce the previous behaviour:
brcmfmac with #1729. Item 6 stays open for follow-ups, as agreed. On G17's matching fallback: thanks, we'll handle that on our side. Before the merge I'll post two more results here:
If |
|
Works with the TB4 patches. Kernel: #1314's head (04f43c7) plus our three patches (both |
3d015d3
into
aurora-silicon:integration/aurora
|
Merged as What the merge rests on
Hardware regression runs after the merge, on the landed tree
I'll post both results here. Anything attributable to this series gets a fix-forward or a revert. |
Where the sleep's commit couldn't take the GPU's stop and the sleep driver didn't require it, .prepare stopped the firmware itself. A later refusal of that sleep then left the GPU stopped, with publication closed, until the next real sleep: the hole aurora-silicon#1314 closes for G16. Refuse the sleep instead, as G16 does. On the A18 Pro, the only G17 chip that sleeps, the GPU's .prepare only runs for a sleep the sleep driver takes, and that leaves the stop to its commit, so the fallback wasn't reached. With it go the Rust stop and its tests. Signed-off-by: Diljit Singh <hey@djdev.me>
Deep sleep (suspend-to-RAM) for the M4 MacBook Air (T8132, J713) at EL2, on dj's generic sleep driver, with Erik Bourget's radio and docked-sleep series.
Base:
integration/aurora(aa68b3c81830; it merges cleanly with today's4277bddd6312), which has dj's T8140 suspend-to-RAM stack since #1445 merged. #1445 superseded #1334 and has the same genericapple-sleepdriver. T8132 is that driver's second SoC,t8132_sleep_socwitht8132_sleep_ops, in the per-SoC data and operations of Ryan's #1769, with the few fields and operations it needs added.What's in it, 52 commits:
iommu: apple-dart: don't require DAPF ranges on a DART that has nonetoarm64: dts: apple: t8132-j713: name the SEP's domain in the PMGR policy), including the system level cache write-back, the SMC's T8132 numbering of its wake reasons, and the PMP thermal samplers paused across a deep sleep.wifi: brcmfmac: don't cancel a bus reset that was never set up.pcie_apple.cio_s2randthunderbolt_apple.cio_s2r(both off by default). That's 18 commits, among them the four the earlier copy here lacked: rolling back a partial consumer power-off in pmgr-pwrstate, theapple,clock-protectionbinding and its J713 DT, andapple,policy-machinefor the platform-handoff entries. His kernel code is as on his branches in all 18, and 10 are identical outright. The rest differ in context, three of them (the DART, ATC CIO and Thunderboltapple.cones) after conflict resolution that keeps both sides, and in the host-test changes the notes below list.clkreq-gpiosand the USB4 router'scioPHY.tools/asahi: run the J713 sleep tests from the host-test runner: registers the 17 host tests the two series added, whichrun-host-tests.pyrequires, and makes them run under itspython3 -I.Erik's commits keep his authorship and sign-off. Thirteen of them have a bracketed note after it: his
Based-on-work-by:tags, which aren't kernel tags, moved into the text (five), checkpatch fixes in his host-test harnesses (four), older host tests' mocks updated for his changes (six, below), the review fixes in item 2 (three), and the fixes for dj's review in item 3 (four). If this PR is where they land, #1414 to #1420 can be closed too.Eight older host suites copy functions this series changes into mocks, and stop compiling against it. Each mock gets the new state, inert as on other machines, in the commit that first needs it:
apple-dart-fwandapple-dart-range(apple-dart.c): the radio sleep record, the sleep writer gate and, with the dock series,cio_activeand the gate's new return value;brcmf-suspendandbcm4377-resume, with their mutation runs: the radio sleep hooks;t6030-clockandapple-dcp(atc.c): the CIO sleep state;rtkit-oslog, with its mutation runs: it now also copiesapple_rtkit_reset_session(), which the cold-restart commit split out ofapple_rtkit_reinit();asahi-m4: G16'spublish()checks the sleep flag.asahi-m4also gets a case for a publish during a sleep, andt6030-clocktwo for the stale ACK a CIO sleep leaves. Erik's ownbrcmfmac-sleep-lifetimetest copiesbrcmf_pcie_pm_enter_D3(), which now has #1653's WoWLAN checks; its mocks get that state too, never armed.Two fixes the J713 needed in
integration/auroraitself are merged now: #1669 (the G16 probe overflowed its 32 KB stack) and #1668 (the scaler and JPEG DARTs had no power domain, so the DART probe panicked at boot).Review items
Signed-off-by: on every commit; Erik's carry his and then mine as the submitter.
Changes from a review of
19de60a3ee67, first pushed asc53fb99165a4. Each fix is in the commit that had the problem, or in a new commit where marked:arm64: dts: apple: t8132: add the sleep controller: the node is disabled int8132.dtsiand enabled int8132-j713.dts. It bound on every T8132 board and made deep sleep the default there, but the radio, DART and DCP recovery is the J713's alone, so every deep sleep was refused and s2idle was no longer the default.iommu: apple-dart: accept a display DART the sleep engine rebuilt: its mark that the sleep engine rebuilt a display DART is only set at resume now, and only the DCP's next power-up clears it. With the panel off across a deep sleep and then an s2idle or a sleep refused late, the mark was cleared while the DART still had the engine's streams on, and the next power-up failed:DCP SID23 retired: power-up: state lost (-5), internal panel dead until a reboot. Reproduced on the J713 before the fix, gone after it.thermal: apple-pmp: pause sampling across a deep sleep: sampling ran on through the stopped PMP. After every wake, "sensor error -19; CPU minimum caps retained" (about 100 lines per test run), the CPUs held at their minimum frequency for about a second, and the GPU's power level stepped back from 7 over about 15 s. Now no sensor or telemetry error after any wake.drm/asahi: G16: ride out dark sensors while the GPU is idle: an idle sample keeps the last cool reading current, so the 2 s allowance runs from when work starts rather than from the last valid reading. After a deep sleep the GPU takes no idle sample before its first work, so that case remains (see Known limitations).mfd: macsmc: number T8132's wake reasons as its SMC does: the wake reasons were named by T8140's numbering, where the lid is bit 10; the J713's SMC uses bit 8. A lid closed during a deep sleep woke the J713 fully, user space included. Now: "woken for the lid, which is closed", a dark wake with the outputs off, and straight back to sleep; opening the lid wakes it.dt-bindings: pci: apple,pcie: give T8132 its own register list: the T8132 branch rejected the third-port layout of the J604, J623, J624 and J773g (12 new CHECK_DTBS warnings). It accepts it now, and those boards' olderreg-nameswarnings are gone too.soc: apple: mcc: write T8132's cache back for a deep sleep: the cache is also written back when probe left it alone and when the driver is unbound, and an enable that isn't confirmed turns it off again: nothing else writes it back before a sleep.iommu: apple-dart: preserve native radio translations across system sleep:sleep-spin.hreadsCNTFRQ_EL0witharch_timer_get_cntfrq(), asarch_timer_get_rate()isn't exported: withCONFIG_APPLE_DART=m(arm64defconfig) modpost failed. The spin also reads the register once more at its deadline before it reports a timeout. The radio port commit's host-test mock follows.thunderbolt: keep PCIe and USB tunnels across a host router power cycle:tb_switch_suspend_cio()tooktb_switch_clx_disable()'s positive return, the CL states it disabled, for an error.wifi: brcmfmac: don't cancel a bus reset that was never set up: a suspend or a remove before the firmware had loaded rancancel_work_sync()on a work with no function, and the workqueue code warned.Changes from dj's review of
af8707ff5d61, each in the commit it concerns:thunderbolt: apple: power-cycle the J713 host router for a deep sleep: only a sleep that may power-cycle the router expects its PCIe tunnel back. That's one withresume_recovery, as before, or the J713's native deep sleep. On any other host, a tunnel nobody asks for again after a wake no longer sets off the post-wake-ENOLINKcycle of ACIO and the PHY. Every host still pauses the DP-IN retries and defers a teardown's PCIe-C quiesce to.completefor a sleep, astb-dock-pm-testsrequires. A failed CIO prepare also restores the expectation. The test has cases and mutations for each rule.t8132-j713.dts, each moved in the commit that added it:ps_sep(enable the SEP for system sleep,describe what a deep sleep needs);let the external DCPs switch their domains);cioPHY (Erik'sgive the ACIO hosts their PHY).name the SEP's domain in the PMGR policynow names it for the J713 only (apple,policy-machine), as the ACIO entries do. In the compiled J715 tree, those nodes match the base again.describe what a deep sleep needs: the clusters' power gates are disabled int8132.dtsiand enabled on the J713.drm/apple: m4: stop the external DCPs for a deep sleep and restart them: a failed stop ends the sleep's PM epoch at once, as.resumewould. The PM core doesn't resume a device whose suspend failed, and the odd epoch kept that display dark until a reboot.drm/asahi: G16: stop the firmware for a deep sleep and restart it warm: if the commit can't take the GPU stop,.preparerefuses the sleep, as the PMP and SIO drivers do. It used to stop the firmware itself, and a later refusal then left the GPU stopped until the next real deep sleep.Rebased from
f173f7c8537aonto028d76246b00, thenbf00add09e22,673f61842546andaa68b3c81830. Five conflicts: four in Erik's commits, one of which also needed a host-test change, and soc: apple: sleep: per-SoC data and operations for the sleep sequence #1769 in the two T8132 sleep commits:soc: apple: sleep: per-SoC data and operations for the sleep sequence #1769 (per-SoC data and operations for the sleep sequence) and
soc: apple: sleep: add T8132 (M4 Air, J713) supportandsoc: apple: sleep: get T8132 through a full sleep and wake: T8132 is ported onto it.t8132_sleep_opshas T8132's ownlast_mileandwake, and uses T8140's deep WFI and AIC operations.t8132_sleep_socfills soc: apple: sleep: per-SoC data and operations for the sleep sequence #1769's fields: the cluster and clock-mux words, a CPU start stride of 6, the PMU's boot-stage register, two MCC channels and two AMCC planes, and the wake block.lower_iooperation (T8140's isapple_sleep_lower_io()), theresumeentry the mailbox gives aura (T8140's iscpu_resume), an optionalpre_commitstep before the commit word, and a delay before the boot core's WFI.commit/rewakehooks are gone.wifi: brcmfmac: pcie: keep Apple firmware in D0 across s2idle #1722 (Apple firmware kept in D0 across s2idle) and
wifi: brcmfmac: coordinate retained firmware with J713 system sleep: onlybrcmf-suspend-tests' mock struct conflicted; it has both sides' fields. Inpcie.cthe new s2idle-D0 branch runs only for s2idle, so the J713's deep-sleep path (the lease, the D3 inform,apple_radio_sleep_endpoint_suspended()) is as before.brcmfmac-sleep-lifetime-testscopies the same function, and its mocks get the s2idle-D0 state, never taken in its deep-sleep cases.PCI: apple: t8140: bring the radios back after a sleep off resume_noirq #1717 (T8140 radio resume off
resume_noirq) andPCI: apple: stop and restart the J713 tunneled port for a deep sleep: both added anapple_pcie_config_read(). It's one function now: the Neo's D0 answer while its radio link comes back, then the J713 tunneled port'scio_cfg_lock. PCI: apple: t8140: bring the radios back after a sleep off resume_noirq #1717's T8140 resume is unchanged, and async resume stays the Neo host's: the J713 radio host still disables it.T8132: DisplayPort alt mode on the J604's USB-C ports #1678 (DisplayPort alt mode on T8132) and
phy: apple: atc: keep a T8132 CIO link when its USB host stopsandphy: apple: atc: take the T8132 CIO PHY through a deep sleep: T8132: DisplayPort alt mode on the J604's USB-C ports #1678'st8132_dp_altand the CIO sleep state are neighbouring fields, inatc.cand in the host tests' copies of its struct (atc-usb3-tests, and the DCP selftests'phy.pyanddwc3/mocks.c). Each keeps both. They concern different modes: the CIO sleep is only prepared in a USB4 or Thunderbolt mode, and only the J604 marks a port for DP alt mode, and deep sleep stays off there.wifi: brcmfmac: pcie: reset the chip when the firmware dies #1729 (reset the chip when the brcmfmac firmware dies) and
wifi: brcmfmac: coordinate retained firmware with J713 system sleep:brcmf_pcie_reset()first refuses while the J713 sleep owns the radio, before it touches anything. Then it marks the device down and drains, as wifi: brcmfmac: pcie: reset the chip when the firmware dies #1729 does. wifi: brcmfmac: pcie: reset the chip when the firmware dies #1729 also runs the reset work on a freezable workqueue, so a reset armed during a sleep now waits until the processes thaw, after.completehas given the radio back to the driver. That also covers the dropped reset in item 6 of dj's review ofaf8707ff5d61.Nothing else the base gained (163 commits, 257 with merges) conflicted. pmdomain: apple: pulse a latched-off domain's PS reset on T8140 only #1757 now pulses a latched-off domain's PS reset only on T8140. It never fired in our J713 runs, where it would have logged "off with BUSY latched".
Earlier: rebased from Apple SoC sleep: one driver for all Apple SoCs, matched only on tested ones #1334 onto
integration/auroraafter Apple SoC suspend-to-RAM on T8140 #1445 merged, with Erik's docked series replaced by his current one. Of the 31 other older patches,git range-diffagainst this PR's previous head shows 20 unchanged and three changed only in context. Five conflicted, and each keeps both sides:soc: apple: sleep: add T8132 ...: Apple SoC suspend-to-RAM on T8140 #1445 now panics when a commit returns, and that check follows the per-SoC commit hook; T8132's commit doesn't return either. Apple SoC suspend-to-RAM on T8140 #1445's report of coprocessor stop errors sits next to T8132's report of power words still on, and the latched power words dj's later change records sit next to T8132's quiesce word. Ryan's AVE complex change (soc: apple: sleep: power the AVE complex as the encoders' parent domain #1663) skips.prepare's complex power-off once genpd owns the complex; T8132 has no complex, so that power-off also stays T8140's (has_ave), as before. A comment left over from removed bring-up code is gone.iommu: apple-dart: accept a display DART the sleep engine rebuilt: a neighbouring bitfield.iommu: apple-dart: preserve native radio translations ...: the DART mock's struct also gets Apple SoC suspend-to-RAM on T8140 #1445'sfw_treefields, andapple_dart_suspend()returns early for the live J613 MTP first, then for a radio DART.soc: apple: resolve radio PERST through the sleep controller's PCIe host: it dropsradio_pciefrom the per-SoC data, next to soc: apple: sleep: power the AVE complex as the encoders' parent domain #1663's new AVE fields.wifi: brcmfmac: coordinate retained firmware with J713 system sleep, against Ryan's WoWLAN in s2idle (wifi: brcmfmac: Wake-on-WLAN in s2idle for the BCM4387/BCM4388 Apple laptops #1653). The suspend refuses WoWLAN without wakeup permission or outside s2idle before the J713 sleep lease is taken, so a deep sleep never reaches the WoWLAN path; the resume checks the lease first, then takes WoWLAN's hot path.brcmf-suspend's mocks have both sides' fields, and it passes with all its mutations.Three changed on purpose:
phy: apple: atc: bring the T8132 PHY back after a deep sleepno longer sets the PIPE handler's dummy PHY enable itself. Friedrich'sphy: apple: atc: enable the dummy PHY on T8132 SuperSpeed ports(phy: apple: atc: enable the dummy PHY on T8132 SuperSpeed ports #1492) now sets it at the same point for every T8132 port with its SuperSpeed banks, which both J713 ports have, and after a deep sleep this commit has the PIPE handler set up again.arm64: dts: apple: t8132: describe what a deep sleep needs: its message said the driver falls back to s2idle without the APCIe tables. Since Apple SoC suspend-to-RAM on T8140 #1445 it refuses a deep sleep that stops ANS instead.The rest of the base's sleep changes need nothing for T8132. The SEP's SLEEP request goes out at the platform's last returnable step. The USB host's DRAM vote stays T8140's alone (
apple,sleep-ddr). The latched power-word clearing and the T8140 domain changes take per-SoC tables T8132 doesn't fill. The refusal over an unbound running coprocessor and NVMe's shutdown notification at suspend apply to T8132 as well, and were in the tests below. So were the scaler and both JPEG engines thatintegration/auroranow enables on the J713: bound and runtime-suspended through every sleep. Its video decoder didn't bind in the tests (noapple/avd-fw-v5-t1.binon this Air), so no sleep has run with it bound.KUnit, built with GCC and run in QEMU:
apple-sleep-soc-data(T8132 in its tested list, and in soc: apple: sleep: per-SoC data and operations for the sleep sequence #1769's layout and operations checks) andapple-mccpass, 9 of 9, andsep.kunitconfig's 294apple_sep*tests pass withCONFIG_GPIOLIBadded, whichsensor_shim.cneeds under GCC, on the base too. The SMC'smacsmc-parsersuite passes 15 of 16 with the new T8132 wake-reason cases; the one failure,macsmc_sleep_order_test, fails the same way onintegration/aurora.Bring-up removed: the DO NOT MERGE commit, the progress marks in the sleep driver, PM core and arm64 resume path, the panic and die notifiers, the comparison with macOS's power words, every
bringup_*parameter (their defaults kept), andcpu-reset.owithoutKEXEC: the board's kernel config now hasCONFIG_KEXEC=y. The T8132 commit no longer adds the PMU commands macOS sends after the commit; sleep and wake work without them.t8132-j713-apcie-tunables.dtsiis gone: the bootloader passes ANS's APCIe tables. Aura 1.0.0b11 publishes them from the ADT on every chip: Publish APCIe ANS recovery tables with atomic packed-DT export aura#112's final revision opens a copy of the tree with the buffer's full capacity, so a treeprepare_dt()has already packed has room for them too. On the J713, 1.0.0b11 logsFDT: /arm-io/apcie tables published, 261 records, and the ten tables are byte-identical to the copy this PR used to carry. With an Aura that predates it (1.0.0b10 as tagged, say), nothing breaks, but every deep sleep is refused at the driver's last checks ("no tunables to bring ANS's PCIe block back") and resumes.checkpatch: every commit is clean apart from two prototypes in
.cfiles (the resume stub's, next to itsasm(), and one the Rust side of G16 calls, like its neighbours) and the MAINTAINERS reminders for new files.One copy: see above.
Known limitations
dwc3-applestops the USB host before the freezer runs and brings it back after the wake, as the base does; a mounted USB disk loses its block device, and no USB device can wake the Air.brcmfmacandhci_bcm4377are both bound: Erik's radio sleep needs both drivers to take the radio through its power loss.cio_s2roff, the default: a USB4 or Thunderbolt device in a port blocks every suspend, s2idle too. DWC3 stops its host for the default deep sleep, Erik'sphy: apple: atc: keep a T8132 CIO link when its USB host stopskeeps the PHY in a CIO mode, andatcphy_suspend()refuses. Onintegration/aurorathat check still let a docked s2idle through.af8707ff5d61,d0ac23c1cfe4,67c358b5b63d,025305c64266and this one). Twice the GPU's power budget inapple-pmp-thermalstarted at its lowest level and released one step a second, and twice G16's own sensor backstop found the die sensors dark at its first sample after the wake. Neither came back in 18 sleeps looked at closely (the budget polled every 50 ms), so their causes aren't settled yet.Speaker Volume Unlockdaemon the J713 runs today) come back almost silent after a deep sleep and take about 90 s to recover. At suspend, macaudio clips the speaker volumes ("Speaker volumes locked: System suspend"). A volume write from the daemon that lands in that window fails with EINVAL, and the daemon gives up ("leaving the speakers at the card's safe level"). After the wake it takes the lock back at about 97 dB of gain reduction and eases off from there. The kernel side behaves as intended: the pre-sleep volumes are restored at the unlock, and the amps keep their state through the sleep (their temperature registers survive it). speakerguardd v2 handles system sleep (packaging: orderly sleep, graph before logins, supervisor after sessions speakerguardd#84, Moved · 🧩 T6034 · M3 Max, 14-core — board index #85, PCI: apple: enable experimental J700 radio endpoint bootstrap #151) but doesn't support the J713 yet.Tested
On the J713 (24 GB, Linux at EL2) as a full system build with
CONFIG_KEXEC=y, radios, GPU driver and cache on, Aura 1.0.0b11 as Stage 2 (a one-bootaura-trytrial from the installed b9 Stage 1, with this branch's DTB), with this head's kernel, all on one boot:systemctl suspendsleeps from a lit desktop (40 s, and 3 min), each woken by its RTC alarm. In 7 of them a process rewrote and checked 256 MB up to the freeze: no bad blocks. After each, the radio recovered from the power loss, Wi-Fi carried 20 and 300 1400-byte pings to the gateway without loss, Bluetooth answered Read Local Version, and the GPU firmware was back and rendered. Then 2 sleeps refused at the last returnable step withapple_sep.sleep_refuse_before_send: each refused before SEP SLEEP was sent (-ECANCELED) and rolled back, and systemd's immediate second attempt then slept and woke normally.19de60a3ee67the same sequence left it dead until a reboot.19de60a3ee67, about 100 "sensor error -19" lines in a run of 13 sleeps).Mesa was built against this base's GPU UAPI, which has GEM userptr. The same run passed on the previous heads:
af8707ff5d61(on028d76246b00),d0ac23c1cfe4and67c358b5b63d(onbf00add09e22), and025305c64266(on673f61842546).Item 3's G16 refusal, on a scratch build of
67c358b5b63dwith a test switch that fails the GPU's commit-time stop registration (not part of this PR): with the switch on, both of systemd's attempts were refused in G16's.prepare("no GPU stop at the commit (EAGAIN): refusing the sleep"), and the GPU rendered afterwards. With it off, the same boot slept and woke normally, the GPU back in 715 µs.Before these rebases, on
c53fb99165a4(the same patches onf173f7c8537a), one boot also had:19de60a3ee67the same close woke it fully.The speaker amps keep their state through a deep sleep: their temperature registers, which a power-on reset clears, still held their last readings after it, and sound played about 4 s after the wake.
The docked series' DT changes the
apple,bridge-linkformat, so the bootloader has to pass this branch's DTB: with the previous one, two pmgr-pwrstate domains fail to probe ("bad bridge link").Before the rebase, on #1334: the same 8 sleeps and 2 refusals (on a boot without
apple_sep) with the tables in the device tree, and 8 more sleeps with them from the bootloader.Also, at this head: it builds (Image, modules, DTBs) with no warning beyond the base's, the whole
run-host-tests.pypasses (319 of 319 runs; the base passes its 302), KUnit is as in item 5, and CHECK_DTBS on all six T8132 boards (J604, J623, J624, J713, J715, J773g) shows no new warnings and ten fewer than the base: the T8132 PCIe host'sreg-namesnow match its binding (withti,cd2e22.yaml,via,vl122.yamlandapple,pmp.yamlset aside: theiruint8-arrayapple,tunablecrashes dtschema's decoder, on the base too). Before these rebases, onc53fb99165a4, every commit built and passed the whole host suite, and the head also built with the DART, PCIe, NVMe, SMC, DWC3, MCC and ATC PHY drivers as modules (19de60a3ee67failed modpost there:arch_timer_get_rate).Not tested here
external DCP not handed over; Aura does that only withchosen.asahi,j713-dcpext=1). Since Erik's own dock test, the base gained Ryan'sthunderbolt: apple: Restart the port when the peer link is lost(e847be1). It cycles a port whose peer stays gone for 3 s, from a freezable workqueue. The docked sleep path hasn't run with it.🤖 Generated with Claude Code