Skip to content

[AUTOMATED] Keep runtime stack reservations through stack-pointer merges - #1596

Open
thotypous wants to merge 5 commits into
Noelo-Lab:mainfrom
thotypous:fix/stackptr-dynamic-sp-merge
Open

thotypous wants to merge 5 commits into
Noelo-Lab:mainfrom
thotypous:fix/stackptr-dynamic-sp-merge

Conversation

@thotypous

Copy link
Copy Markdown
Contributor

Problem

Open Watcom's DOS alloca reserves stack space only on the branch where stackavail() says there is room, and returns NULL on the other branch:

call  stackavail_
cmp   dx,ax
jae   L$1          ; not enough room: p = NULL
sub   sp,ax        ; reserve (n + 1) & ~1 bytes
mov   ax,sp        ; p = sp

The x86-16 default model has unknown extrapop, so ActionStackPtrFlow::analyzeExtraPop (the StackSolver) runs. Its MULTIEQUAL rule equates every input of a stack-pointer merge. That hands sp - ax the other path's constant, and the solver then rewrites the INT_SUB(sp, ax) producer as INT_ADD(sp_in, -12). The runtime reservation disappears, and the allocated pointer prints as a fixed frame local. Before:

  if (stackavail() <= v1)
    v2 = NULL;
  else {
    v2 = &Stackfffe;          /* the reservation is gone */
  }

Fix

p6_variables/coreaction_stackptr.rs:

  • Runtime adjustments are marked dynamic. A stack-pointer value defined by arithmetic whose amount does not evaluate to a constant (kuna_bejoin::literal_value) is marked dynamic. A PTRADD element size does not count as an amount.
  • Only adjustments the merge erased stay dynamic. A trial solve keeps upstream behaviour for adjustments it gives a nonzero move: for example add sp,si with si == 2 on that path, which pops a pushed argument (FreeDOS ARJ, GZIP). A seed stays dynamic only if the trial solve gives it a zero move (the merge erased it) or leaves it unsolved.
  • Dynamic values spread forward and are never rewritten. The mark follows every forwarding op: MULTIEQUAL, COPY, CAST, INDIRECT, INT_ADD/SUB/AND and PTRADD/PTRSUB. Equations joining a dynamic value to a fixed one are dropped; relations among dynamic values stay but are never anchored to entry SP. Dynamic values are never rewritten.
  • Constant adjustments get exact equations. INT_SUB(sp, c) and adjustments whose amount evaluates to a constant (mov ax,8; sub sp,ax) now get their exact equation instead of being left to merge equalities.

After:

  else {
    v1 = -(n + 1 & 0xfffe);
    v8 = &v3[v1];            /* the runtime amount survives */
  }

Tests

New CLI test dynamic_sp_merge on two Open Watcom 2 DOS executables built from watcom_dos_alloca.c (wcl -bt=dos -ms -s -ecc, -od and -ox). It requires the reservation arm's pointer to use the (n + 1) & ~1 amount, and it fails on main. The 13 existing coreaction_stackptr unit tests pass.

Validation

  • Gate: every CI step passes except the WASI/dev-viz step, which needs git history.
  • DOS corpus (646 binaries / 91,189 functions: Turbo C samples plus FreeDOS 1.3): 93 functions change.
    • The C-compiled ones are genuine Watcom alloca sites (FORMAT, JWASMR, FDTUI): the reservation now survives.
    • The rest are hand-written assembler (LDEBUG, FASM, TINYASM) with irregular SP use, where the stack is no longer folded onto a guessed constant.
  • Not run: decbench and byte-match. The solver only runs under unknown default extrapop (x86-16, some Win32), which those corpora barely exercise.

Known limitation

The trial solve is evidence, not proof. Suppose a runtime reservation sits on one arm of a merge, and the sibling arm moves SP by a nonzero constant (for example alloca(c ? n : 8)). The trial then reads the sibling's constant and keeps upstream's rewrite, so the output there is the same as main's.

- The extrapop solver equated every input of a stack-pointer merge, so a path lowering SP by a runtime amount took the other path's constant and was rewritten away.
- Mark SP values moved by a non-constant amount, and merges they reach, as dynamic: no equation touches them and they are never rewritten.
- Regression: Open Watcom DOS alloca behind stackavail() at -od and -ox.
- Dynamic marks follow COPY, CAST, INDIRECT, INT_SUB and PTRADD/PTRSUB value inputs, not only merges.
- Only equations joining a dynamic value to a fixed one are dropped; relations among dynamic values stay, unanchored.
- PTRADD's element size is not an amount; INT_SUB(sp, c) gets the equation add sp,-c, so a constant reservation on one arm is solved instead of equated.
- A non-constant adjustment is often a constant not yet propagated (add sp,si with si == 2 after a push): a trial upstream solve implies a nonzero move there, so it stays solved as before.
- Only an adjustment whose solved move is zero (the merge equated it away) or that stays unsolved is treated as runtime. FreeDOS sweep: ARJ/GZIP return to main's output; FORMAT and JWASMR keep the Watcom alloca amount.
- mov ax,8; sub sp,ax is a fixed reservation: literal_value sees through the copy, so it is not marked runtime.
- is_const/const_off read literal_value, so mov ax,8; sub sp,ax is solved as an 8-byte adjustment instead of being equated through a merge.
- Document that confirm_dynamic is evidence, not proof: a runtime arm merged with a nonzero constant keeps upstream's answer.
- The regression test now requires the reservation arm's pointer to use the (n + 1) & ~1 amount.
@thotypous
thotypous force-pushed the fix/stackptr-dynamic-sp-merge branch from a2fcf2c to 1108ad5 Compare November 15, 2026 17:51

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant