[erikwb]

The tunneled PCIe port behind a Thunderbolt dock loses its clocks and its DART in a deep sleep. With pcie_apple.cio_s2r=1 on the J713, the port owns its DART through the new owner interface and takes part in a transaction the ACIO host prepares and finishes.

The port's noirq suspend drains it, closes config space, asserts the tunnel reset and PERST, powers the DART down and stops the port clock. The noirq resume re-runs the cold port setup, restores the requester and MSI routes and the DART, releases the tunnel reset and waits up to one second for the link. An abandoned suspend recovers through the same path. Config space is closed under a lock that config accesses hold from map_bus() to the access.

A port whose link does not come back, for a dock unplugged or powered off during the sleep, ends like a surprise removal: it stays stopped, its hierarchy is marked disconnected and removed with the tunnel after resume, and the DART transaction still ends. When the ACIO host already knows the dock is gone, the port does not wait for the link.

A deep sleep the ACIO host did not prepare for this port (for example thunderbolt_apple.cio_s2r=0) is refused in .prepare, before any other device suspends.

The devices behind the dock stay bound across the sleep.

Part 5/7 of J713 docked S2R, stacked on the native radio series (#1384), which is stacked on #1314.

Series order: #1414 → #1415 → #1416 → #1417 → #1418 → #1419 → #1420.

Depends on #1417.

Validation of the series on this base:

• Full arm64 Image, modules and DTBs at every PR tip of the series.
• Ten focused host suites with negative controls (DART owner and register sequence, PHY sleep transaction, USB3 host model, router state transfer, tunnel replay including a dock lost during the sleep, NHI and tunneled-PCIe ownership including its failure paths, dock PM callback ordering, DP repair, DART writer gate and its callers) plus the device-tree policy check. checkpatch is clean for the driver code; it notes the Based-on-work-by: trailers and style in the tools/asahi fixtures.
• J713 with a CalDigit TS5 (USB4 link, PCIe tunnel with the dock's Ethernet, USB hub with two keyboards, two 4K DisplayPort monitors), on the first version of the series: one suspend-abort test at the platform stage, then one real deep sleep with an RTC wake on the same boot. After the wake the dock link was at 20 Gb/s, dock Ethernet passed traffic at 2.5G, the USB devices were back, both monitors were lit at their previous mode about 10 s after the wake, Wi-Fi and Bluetooth answered, all ten CPUs were online, and there were no kernel warnings or taint. Repeated with one monitor unplugged; a monitor plugged in after the wake came up.

The changes since that version (the DT-selected DART owners and memory-only updates while closed, the unplug handling, the failure unwinds, the stock tb.c resume restored, the PMGR and RTKit hardening) are built and covered by the host suites, but have not run on the J713 yet.

Not covered: repeated sleeps on one boot, wake sources other than the RTC, the desktop's own sleep path (lid, logind), a dock unplugged during the sleep on hardware, and docks other than the TS5. Both native parameters (pcie_apple.cio_s2r, thunderbolt_apple.cio_s2r) default to off.

This series was developed and tested by Claude (Anthropic), working with me on the J713.

[DjDeveloperr]

[5/7] PCI: apple: review at 43aec22

Gating: fine. The new paths only run with pcie_apple.cio_s2r=1 on t8132_pciec_hw with apple,j713. Everywhere else cio_dart is NULL and cio_prepared/cio_changed stay false. The only addition on a shared path, the cio_changed check in apple_pcie_map_bus(), reads false there. No T6050 change; pcie-apple.c already conflicts with the M5 branch in 5 places through the series base, and this adds none.

Stop/restart pairing: fine. A failed noirq suspend recovers itself, since PM won't resume this device. A later abort reaches resume_noirq, and the NHI finishes in .complete.

Findings

1. A link that doesn't come back panics. apple_pcie_cio_resume() waits up to 1 s for LINKSTS_UP, and resume_noirq panics if it never comes. A dock unplugged or power-cycled during sleep gets there, or gets to #1419's NHI panic first. A dead port should end like a surprise removal of the hierarchy, not a panic. That needs #1416 to stop panicking on release.
2. The two cio_s2r params can disagree. With pcie_apple.cio_s2r=1 and thunderbolt_apple.cio_s2r=0, the port binds its DART through the owner interface but nothing prepares it. suspend_noirq then refuses every deep sleep (and s2idle unless s2idle_keep_link), and only after every other device has suspended. Bind the owner from the NHI's prepare, use one parameter, or refuse in .prepare.
3. cio_changed is a plain store, read locklessly in map_bus(). A config access that passed map_bus() just before the flag flips can still hit the port after APPCLK is cleared. The risk is small in noirq, but pci_cfg_access_lock() or the bus pci_lock closes it.
4. Nit: the 1 s and 100 ms polls are read_poll_timeout_atomic() with 1 ms steps. Noirq callbacks may sleep, so read_poll_timeout() avoids a 1 s busy-wait.

Verdict: needs changes, mainly (1), together with #1416 and #1419.

---

Review only: read at the head commit, not built or run (no J713 here). Written with Claude Code.

[DjDeveloperr]

The commonest way here is a dock unplugged or powered off during sleep: the link can't train, and every wake becomes a panic. Please end this the way a surprise removal would: leave the port stopped, mark it dead, finish the transaction, and let the hierarchy be removed after resume. Today that removal would itself panic in apple_dart_release_device() (#1416), so the two need to change together.

[DjDeveloperr]

With pcie_apple.cio_s2r=1 but thunderbolt_apple.cio_s2r=0 (or the NHI not bound), cio_dart is bound and never prepared. Every deep sleep is then refused here, after every device has already gone through suspend and suspend_late, and s2idle is refused unless s2idle_keep_link. Since the NHI owns the transaction, it would be simpler to bind the owner when the NHI asks for it, or to share one parameter.

[DjDeveloperr]

apple_pcie_map_bus() reads this locklessly, and it's a plain store here. A config cycle that passed map_bus() just before can still be in flight when APPCLK goes off below. pci_cfg_access_lock() on the root port (or WRITE_ONCE plus taking the bus pci_lock around the flip) would order them.

[DjDeveloperr]

Nit: this is a busy-wait of up to 1 s in 1 ms steps. resume_noirq may sleep, so read_poll_timeout() works here, and the same goes for the other polls in this file.

[erikwb]

Closing: this commit set landed in integration/aurora through #1314, which carried the whole docked deep-sleep series (#1414 to #1420). The follow-up that turns the docked deep sleep on by default is #1855.