Skip to content

Run usbdevfs URBs asynchronously over IOKit #375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jserv merged 2 commits into from
Sep 14, 2026
Merged

Run usbdevfs URBs asynchronously over IOKit #375

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
200c5e3
Run usbdevfs URBs asynchronously over IOKit
jotpalch Sep 8, 2026
5e7bad4
Model a loopback USB device behind the IOKit seam
jotpalch Sep 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions .ci/check-usb-fixture-bin.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
#!/usr/bin/env bash

# Keep the USB loopback fixture out of the shipped binary by construction.
#
# USB_LOOPBACK_FIXTURE=1 swaps src/syscall/usbdev-fixture-stub.c for
# src/syscall/usbdev-fixture.c (the SRCS block in the top-level Makefile). That
# changes SRCS, not CFLAGS, and the stale-object guard in mk/common.mk is keyed
# on $(strip $(CFLAGS)) alone, so it has nothing to say about the switch: while
# both flavors linked to build/elfuse, building one and then asking for the
# other printed "Nothing to be done for 'elfuse'" and handed back whichever had
# been linked last. Measured before the split below, on the tree as it stood
# then: make clean; make elfuse; make USB_LOOPBACK_FIXTURE=1 elfuse; make elfuse
# left _usbdev_fixture_lock in build/elfuse until make clean. No byte counts
# here on purpose. That build needed both flavors to write one path, so it
# cannot be produced again to re-measure, and the pair this comment used to
# quote had gone stale against the tree twice over. The dated figures are in
# docs/internals.md; what reproduces today is what the check below asserts.
#
# What keeps them apart now is the path: mk/config.mk points ELFUSE_BIN at
# $(ELFUSE_LOOPBACK_BIN) when the fixture is asked for, so the two flavors never
# write the same file and the shipped one cannot be a stale copy of the other.
# That is one variable, in a file nothing stops a later change from
# re-simplifying, which is what this check is for. It asks make itself rather
# than reading the makefile, so a change that moves the decision elsewhere is
# still covered as long as the answer stays right.
#
# Cheap on purpose: two variable expansions, no compilation. The whole-tree
# reproduction above is what it stands in for.

set -e -u -o pipefail

MAKE_BIN="${MAKE:-make}"
ROOT="$(cd "$(dirname "$0")/.." && pwd)"

plain="$("$MAKE_BIN" -C "$ROOT" -s --no-print-directory print-elfuse-bin)"
fixture="$("$MAKE_BIN" -C "$ROOT" -s --no-print-directory \
USB_LOOPBACK_FIXTURE=1 print-elfuse-bin)"

if [ -z "$plain" ] || [ -z "$fixture" ]; then
echo "check-usb-fixture-bin: print-elfuse-bin produced nothing" >&2
exit 2
fi

if [ "$plain" = "$fixture" ]; then
echo "Error: USB_LOOPBACK_FIXTURE=1 links to $fixture, the same path a" >&2
echo " plain build writes, so a fixture build leaves the shipped" >&2
echo " binary carrying the loopback model until a make clean." >&2
echo " Point ELFUSE_BIN at \$(ELFUSE_LOOPBACK_BIN) for that flavor" >&2
echo " (mk/config.mk), or make the choice invalidate the binary." >&2
exit 1
fi
56 changes: 56 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,12 +80,40 @@ SRCS := \
debug/log.c \
debug/syscall-hist.c

# The USB fixture seam (src/syscall/usbdev-fixture.h). usbdev.c calls it with no
# conditional compilation of its own, so exactly one translation unit has to
# define the entry points and the choice is made here: the stub in every build,
# the loopback device model when USB_LOOPBACK_FIXTURE asks for it. Listing both
# would be a duplicate-symbol link error, which is the property that keeps a
# default build from quietly acquiring the model.
ifeq ($(USB_LOOPBACK_FIXTURE),1)

@jserv jserv Sep 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BUILD_FLAVOR is $(strip $(CFLAGS)) (mk/common.mk) and the fixture switch changes SRCS, not CFLAGS, so flipping it wipes nothing. Run the command mk/config.mk documents, make USB_LOOPBACK_FIXTURE=1, and the model is linked into build/elfuse; a later plain make finds build/elfuse newer than every object in the rebuilt OBJS and relinks nothing, so the default binary keeps the model until someone runs make clean. The make check path is safe because elfuse-loopback overrides ELFUSE_BIN, which is what the nm evidence measured. Either fold the fixture selection into the flavor stamp so the tree wipes on a flip, or make the documented manual command target $(ELFUSE_LOOPBACK_BIN) too.

@jotpalch jotpalch Sep 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed. BUILD_FLAVOR never moves when the flavor flips, so nothing invalidated the tree and a plain make after make USB_LOOPBACK_FIXTURE=1 handed back a build/elfuse still linked against the model.

I took the second option you offered rather than folding the switch into the flavor stamp: mk/config.mk now points ELFUSE_BIN at $(ELFUSE_LOOPBACK_BIN) when USB_LOOPBACK_FIXTURE=1, so the two builds write different paths and neither can be a stale copy of the other. .ci/check-usb-fixture-bin.sh asks make print-elfuse-bin under both flavors and fails if they ever collapse back to one path.

Validation: the four-command sequence you gave (make clean; make elfuse; make USB_LOOPBACK_FIXTURE=1 elfuse; make elfuse) now leaves build/elfuse at 815984 bytes with no _usbdev_fixture_lock beside build/elfuse-loopback at 834304 with it. Before the change the same sequence left the model in build/elfuse until make clean.

SRCS += syscall/usbdev-fixture.c
else
SRCS += syscall/usbdev-fixture-stub.c
endif

SRCS := $(addprefix src/,$(SRCS))
OBJS := $(patsubst src/%.c,$(BUILD_DIR)/%.o,$(SRCS))

# Every host source, whether or not this build links it. Only one of the two
# fixture-seam translation units is ever in SRCS, and a static analyzer wants
# both: make lint reads this rather than SRCS so that turning the fixture off
# does not also turn off the checking of it.
ALL_SRCS := $(sort $(SRCS) src/syscall/usbdev-fixture.c \
src/syscall/usbdev-fixture-stub.c)

DISPATCH_MANIFEST := src/syscall/dispatch.tbl
DISPATCH_GENERATOR := scripts/gen-syscall-dispatch.py
DISPATCH_HEADER := $(BUILD_DIR)/dispatch.h

# The usbdevfs departed-device vectors. Generated for the same reason
# dispatch.h is: the table is data, the join against usbdev_ioctl's dispatch is
# a gate, and neither is something to hand-edit. Under build/ so that the
# formatting gates, which cover tests/*.h, have no opinion about a file a script
# writes.
DEPARTED_MANIFEST := tests/usbdev-ioctl-departed.tbl
DEPARTED_GENERATOR := scripts/gen-usbdev-ioctl-departed.py
DEPARTED_HEADER := $(BUILD_DIR)/usbdev-ioctl-departed-vectors.h
HVF_LDFLAGS := -framework Hypervisor -framework IOKit -framework CoreFoundation -arch arm64

# Generated headers under build/ that must exist before compiling sources that
Expand Down Expand Up @@ -127,6 +155,11 @@ $(DISPATCH_HEADER): $(DISPATCH_MANIFEST) $(DISPATCH_GENERATOR) src/syscall/abi.h

$(BUILD_DIR)/syscall/syscall.o: $(DISPATCH_HEADER)

$(DEPARTED_HEADER): $(DEPARTED_MANIFEST) $(DEPARTED_GENERATOR) \
src/syscall/usbdev.c | $(BUILD_DIR)
@echo " GEN $@"
$(Q)python3 $(DEPARTED_GENERATOR) --output $@

## Build the elfuse executable
elfuse: $(ELFUSE_BIN)

Expand Down Expand Up @@ -289,6 +322,14 @@ $(BUILD_DIR)/test-usb-desc-host: $(BUILD_DIR)/test-usb-desc-host.o \
@echo " LD $@"
$(Q)$(CC) $(CFLAGS) -o $@ $^

## Build the usbdevfs URB bookkeeping host unit test (native macOS binary)
# usbdev-urb.h is header-only arithmetic with no IOKit and no I/O, so the test
# needs no object but its own.
$(BUILD_DIR)/test-usbdev-urb-host: \
$(BUILD_DIR)/test-usbdev-urb-host.o | $(BUILD_DIR)
@echo " LD $@"
$(Q)$(CC) $(CFLAGS) -o $@ $^

## Build the guest environment merge host test (native macOS binary)
# guest-env.o's only dependency is the log macro, which the test stubs.
$(BUILD_DIR)/test-guest-env-host: $(BUILD_DIR)/test-guest-env-host.o \
Expand Down Expand Up @@ -353,12 +394,27 @@ $(BUILD_DIR)/%: tests/%.c | $(BUILD_DIR)
@echo " CROSS $<"
$(Q)$(CROSS_COMPILE)gcc $(CROSS_TEST_CFLAGS) -o $@ $<

# test-usbdev-ioctl-departed reads the generated vectors out of build/, so it
# needs that directory on the include path where the other guest binaries do
# not.
$(BUILD_DIR)/test-usbdev-ioctl-departed: tests/test-usbdev-ioctl-departed.c \
$(DEPARTED_HEADER) | $(BUILD_DIR)
@echo " CROSS $<"
$(Q)$(CROSS_COMPILE)gcc $(CROSS_TEST_CFLAGS) -I$(BUILD_DIR) -o $@ $<

# test-usbdev-ioctl churns open/read/close on one usbdevfs node from four
# threads, so a close and a sibling's open contend for the same fd number.
$(BUILD_DIR)/test-usbdev-ioctl: tests/test-usbdev-ioctl.c | $(BUILD_DIR)
@echo " CROSS $< (with -lpthread)"
$(Q)$(CROSS_COMPILE)gcc $(CROSS_TEST_CFLAGS) -o $@ $< -lpthread

# test-usbdev-urb-loopback opens a second usbdevfs node from a thread while the
# first is closing, so the two contend for one guest fd number.
$(BUILD_DIR)/test-usbdev-urb-loopback: \
tests/test-usbdev-urb-loopback.c | $(BUILD_DIR)
@echo " CROSS $< (with -lpthread)"
$(Q)$(CROSS_COMPILE)gcc $(CROSS_TEST_CFLAGS) -o $@ $< -lpthread

# test-eventfd-semaphore-contended races two blocking readers on one eventfd.
$(BUILD_DIR)/test-eventfd-semaphore-contended: \
tests/test-eventfd-semaphore-contended.c | $(BUILD_DIR)
Expand Down
17 changes: 12 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,11 @@ linker resolved against an external sysroot via `--sysroot`.
- Synthetic `/proc` and selected `/dev` emulation for user-space probes
- USB device passthrough: `/dev/bus/usb` and `/sys/bus/usb/devices` are
built from the IOKit registry, and opening a device node yields a
usbdevfs fd whose synchronous ioctls (interface claim, control and bulk
transfers) drive the attached device through IOKit. Asynchronous URB
submission is not implemented; a udev-backed `lsusb` also needs
`name_to_handle_at`, and macOS publishes no root hubs, so there are no
`usbN` entries and `lsusb -t` lists devices without their bus rows
usbdevfs fd whose ioctls -- interface claim, control and bulk transfers,
and asynchronous URBs -- drive the attached device through IOKit. A
udev-backed `lsusb` also needs `name_to_handle_at`, and macOS publishes
no root hubs, so there are no `usbN` entries and `lsusb -t` lists devices
without their bus rows
- Guest-internal FUSE: `/dev/fuse` and `mount("fuse")` work without
macFUSE / FUSE-T / FSKit
- Built-in GDB Remote Serial Protocol stub usable from `gdb` or `lldb`
Expand Down Expand Up @@ -223,6 +223,13 @@ do.
mask); the host scheduler picks the actual CPU.
- `/proc`, `/dev`, and mount data are synthetic compatibility views,
not host pass-throughs.
- USB interfaces bound to an Apple class driver (CDC serial, HID, FTDI)
cannot be claimed; `CLAIMINTERFACE` reports `EBUSY`, and root-mode
device capture is not implemented. CDC serial devices are reachable by
opening the host's `/dev/cu.*` node instead.
- USB mass storage will never be claimable, even with capture; macOS
does not release it.
- Isochronous URBs are unimplemented; submitting one reports `EINVAL`.

@jserv jserv Sep 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it feasible to implement isochronous URB?

@jserv jserv Sep 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

@jotpalch jotpalch Sep 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feasible, and I did not find a structural blocker. It is bounded work rather than a redesign.

Today's -EINVAL is deliberate: the URB-type switch hits case LINUX_URB_TYPE_ISO: and returns it with a log_warn (usbdev.c:4089-4091), and the synchronous paths reject iso endpoints the same way at usbdev.c:3624 and :4164, citing devio.c:1715-1717.

The IOKit side is ReadIsochPipeAsync/WriteIsochPipeAsync: a buffer, a frame number from GetBusFrameNumber, a packet count, and an IOUSBIsocFrame[] (frReqCount in, frActCount/frStatus out per packet). No pre-registered buffers are needed; that is only the LowLatency variant, and libusb's own darwin backend skips it for isoch too. On the usbfs side the fields that have to map are number_of_packets (1-128), the usbdevfs_iso_packet_desc[] array, URB_ISO_ASAP, start_frame, and the per-packet actual_length/status that compute_isochronous_actual_length computes on completion.

Worth saying, since you asked before this existed: the async URB engine in this PR changes the calculus. ReadPipeAsync/WritePipeAsync completion plumbing, the per-endpoint FIFO with one URB in flight, and the orphan and late-callback handling under async_lock all run on the same completion-callback and run-loop model IOKit's isoch calls use, so that scaffolding carries over instead of having to be built first. What is left is the frame-list mapping both directions, ISO_ASAP/start_frame emulation via GetBusFrameNumber plus interval bookkeeping (libusb's submit_iso_transfer already does this, so the shape is proven), and per-packet status on reap.

I am not committing to it in this PR. If you want it scoped as a follow-up, say so and I will size it properly.

- `uname` and `/proc/version` report Linux 6.18 LTS, a floor for
version-gated userspace; `src/syscall/dispatch.tbl` states what is
implemented.
Expand Down
Loading
Loading