Skip to content

fix(loongarch64): make SMP IPI and Virtio IRQ delivery stateful - #380

Merged
enkerewpo merged 6 commits into
syswonder:devfrom
li041:smp-int-fix
Aug 19, 2026
Merged

enkerewpo merged 6 commits into
syswonder:devfrom
li041:smp-int-fix

Conversation

@li041

@li041 li041 commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR fixes LoongArch SMP guest startup and interrupt delivery issues by introducing explicit virtualization state for guest CPUs, IPIs, event notifications, and HWI interrupt lines.

The main changes are:

  • Add SMP guest CPU initialization based on zone CPU mappings instead of assuming CPU0.
  • Virtualize guest IPI status, enable, mailbox, send, and clear operations.
  • Improve event notification handling to avoid dropping pending events.
  • Track HWI0-HWI7 as independent level-triggered interrupt lines.
  • Fix Virtio interrupt injection by deasserting individual IRQ lines instead of using global interrupt state.

Motivation

The previous LoongArch implementation relied on UP guest assumptions, which caused incorrect behavior for SMP guests:

  • GCSR.CPUID was always initialized as zero, preventing correct secondary CPU startup.
  • Event notification could overwrite or lose pending events during concurrent delivery.
  • Physical IPI handling cleared unrelated IPI bits.
  • Guest IPI actions were not maintained after injection.
  • Virtio IRQ cleanup used global state and could clear unrelated pending interrupts.

These issues resulted in secondary CPU boot failures, lost reschedule/call-function IPIs, spurious interrupts, and Virtio console/block stalls.

Public API Changes

Hypercall ABI

This PR introduces a breaking change to HvClearInjectIrq:

  • Hypercall number changes from 20 to 11. The mandatory number change is caused by an ioctl command collision. HVISOR_SET_BOOT_MODE already uses ioctl command number 9:
 #define HVISOR_SET_BOOT_MODE \
     _IOW(1, 9, struct hv_zone_boot_mode *)

 #define HVISOR_DEASSERT_IRQ \
     _IOW(1, 9, struct hvisor_irq_line_args)

Linux ioctl values encode the direction, type, command number, and argument size. On a 64-bit platform, struct hv_zone_boot_mode * is 8 bytes and struct hvisor_irq_line_args is also 8 bytes. The two definitions therefore produce the same ioctl value, causing duplicate switch cases in the driver.

Virtio IRQ Deassert ABI

This PR replaces the previous broadcast interrupt-clear operation with a precise per-IRQ deassert interface.

The previous implementation ignored the IRQ identifier and broadcast a clear event to all running non-root CPUs. This behavior is incorrect for SMP guests because multiple interrupt sources may be pending simultaneously, and clearing global interrupt state can remove unrelated interrupts. The arguments for HvClearInjectIrq change to: arg0: target zone_id, arg1: target irq_id. The operation now deasserts only the specified guest interrupt line.

A matching hvisor-tool update(syswonder/hvisor-tool#111) is required : The driver must pass (zone_id, irq_id) to Hvisor.

Event Notification

This PR replaces the previous two-stage event notification interface:

arch_prepare_send_event(cpu_id, ipi_id, event_id);
arch_send_event(cpu_id, ipi_id);

with

arch_notify_event(cpu_id, ipi_id, event_id, queue_was_empty);

The new interface moves event queue state handling into the common event layer. The event is enqueued first, and the architecture layer is informed whether the target queue was previously empty.

This allows different architectures to apply architecture-specific notification strategies:

  • AArch64, RISC-V, and x86_64 continue to notify the target CPU for every event.
  • LoongArch sends the physical event doorbell only when the queue transitions from empty to non-empty, avoiding redundant IPIs without exposing LoongArch-specific state in the common event layer.

@enkerewpo enkerewpo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup of the LoongArch interrupt path — the SMP CPU-ID mapping, the level-triggered HWI lines, and the per-IRQ deassert are all clear improvements over the previous UP assumptions.

One thing worth calling out because it is easy to get wrong: the doorbell coalescing in arch_notify_event looks correct to me. The drain loop in handle_interrupt clears the doorbell bit before each fetch, never after the last one, so a doorbell that lands mid-drain still leaves the bit set and re-fires — the "sender skipped the doorbell because the queue was non-empty" case cannot lose an event. Might be worth a comment there, since the ordering is load-bearing and a later refactor that moves the clear after the fetch would silently break it.

A few smaller things inline. Note also that syswonder/hvisor-tool#111 as it stands still has HVISOR_HC_CLEAR_INJECT_IRQ 20 and an ioctl number that collides with HVISOR_SET_BOOT_MODE, so the two PRs are not yet consistent — I left details on that PR.

Comment thread src/hypercall/mod.rs
Comment thread src/device/irqchip/ls7a2000/mod.rs Outdated
Comment thread src/arch/loongarch64/ipi.rs Outdated
Comment thread src/arch/loongarch64/ipi.rs
@li041

li041 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

One thing worth calling out because it is easy to get wrong: the doorbell coalescing in arch_notify_event looks correct to me. The drain loop in handle_interrupt clears the doorbell bit before each fetch, never after the last one, so a doorbell that lands mid-drain still leaves the bit set and re-fires — the "sender skipped the doorbell because the queue was non-empty" case cannot lose an event. Might be worth a comment there, since the ordering is load-bearing and a later refactor that moves the clear after the fetch would silently break it.

Added a comment in handle_interrupt documenting why the doorbell must be cleared before each queue fetch. The comment explains that a producer racing with the final empty fetch must leave a new doorbell pending so the event is not lost.

li041 and others added 6 commits August 19, 2026 00:12
Merge guest IPI action bits, publish blocking sends before return, and keep the virtual IPI line asserted until all pending actions are cleared.
Maintain asserted HWI bits per physical CPU, update only GINTC.VIP, preserve PIP and HC, and remove timer-driven global injection cleanup.
Pass the target zone and IRQ through HvClearInjectIrq so backends can deassert one guest interrupt line without clearing unrelated pending interrupts.
enkerewpo

This comment was marked as off-topic.

@enkerewpo enkerewpo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@enkerewpo
enkerewpo merged commit 4c5e92b into syswonder:dev Aug 19, 2026
25 checks passed
agicy pushed a commit that referenced this pull request Aug 24, 2026
fix(loongarch64): make SMP IPI and Virtio IRQ delivery stateful
agicy pushed a commit that referenced this pull request Aug 24, 2026
fix(loongarch64): make SMP IPI and Virtio IRQ delivery stateful
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants