Skip to content

Inline asm lacks side-effect and clobber model, causing incorrect behavior in OS-level code #274

@LunaStev

Description

@LunaStev

Summary

Please build and test 67f368c428fa1de2014b14e490270a6f643efa6d or higher yourself.

Wave's inline asm currently has no side-effect or clobber model.
Because of this, asm blocks that perform memory-mapped I/O (e.g. VGA text buffer)
can be incorrectly optimized, reordered, or treated as having no observable effect.

This makes OS-level development unreliable and leads to incorrect or unstable behavior.

Wave Kernel Source: https://github.com/LunaStev/wave-kernel

Observed behavior

While experimenting with a simple kernel written in Wave:

  • Inline asm that writes to VGA memory (0xb8000) sometimes:
    • Produces random CP437 characters (☺ ♥ ♣, etc.)
    • Outputs incorrect characters instead of ASCII ('A', 'B', 'C')
    • Gets completely removed unless its return value is artificially used
  • Register values appear to be assumed preserved by the compiler,
    even though the asm block overwrites them (e.g. rdi, rsi)
  • The behavior changes depending on optimization and surrounding code

Expected behavior

  • Inline asm should be able to express:
    • That it has side effects (must not be removed even if outputs are unused)
    • Which registers or memory it clobbers
  • Memory-mapped I/O via asm should behave deterministically
  • OS-level code should not need artificial "sink" variables to keep asm alive

Root cause analysis

The current asm lowering does not provide:

  • A "has side effects" (volatile) flag
  • A clobber model for registers or memory

As a result:

  • The optimizer may remove asm blocks entirely
  • The compiler may assume registers are preserved when they are not
  • Inline asm becomes unsafe for low-level / OS development

Why this matters

Inline asm is essential for:

  • OS kernels
  • Hardware access
  • Memory-mapped I/O
  • Interrupt and CPU control instructions

Without side-effect and clobber semantics, inline asm cannot be used reliably
in these domains.

Suggested direction

Possible approaches (examples):

  • Add a side-effect / volatile flag to asm expressions
  • Allow explicit register clobbers (e.g. rdi, rsi, rax, memory)
  • Treat asm blocks as memory-affecting by default (conservative but safe)
  • Align asm semantics with LLVM InlineAsm expectations

Reproducibility

This issue is reproducible with a minimal VGA text-mode kernel
that writes characters via inline asm.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions