Skip to content

portal: bulk process-tree walk (OSI_PROC_ALL) + syscall_event pid denormalization#88

Merged
lacraig2 merged 1 commit into
mainfrom
workspace/proctree-ux
Jul 16, 2026
Merged

portal: bulk process-tree walk (OSI_PROC_ALL) + syscall_event pid denormalization#88
lacraig2 merged 1 commit into
mainfrom
workspace/proctree-ux

Conversation

@lacraig2

Copy link
Copy Markdown
Contributor

What

Two additions to the portal OSI/syscall interface, in support of penguin epic B slice 1 (a queryable guest process-tree model — see rehosting/penguin).

1. HYPER_OP_OSI_PROC_ALL — kernel-side bulk process walk

Walks for_each_process under rcu_read_lock and returns a slim osi_proc_node per user process in one paginated transaction:

struct osi_proc_node {
    uint64_t pid, ppid, create_time;   // create_time = task->start_time (identity)
    uint32_t uid, gid, euid, egid;
    char comm[16];                     // inline task->comm, NUL-padded
};                                     // 56 bytes -> ~72 per 4 KB page

Why: to build a process tree the host was issuing one HYPER_OP_OSI_PROC per pid (get_proc_handles + N× get_proc) — 1 + N round-trips over a single-page portal region, and a snapshot that can tear as processes come and go between the per-pid reads. This is one RCU-consistent snapshot in ceil(N/72) transactions.

Safety: comm comes from task->comm (kernel memory only) and !mm tasks (kernel threads) are skipped, so it never calls access_remote_vm and is safe against exiting/stopped contexts. Kernel-thread enumeration is intentionally deferred.

Auto-wired through the portal_op_list.h X-macro (enum + dispatch + prototype); handler mirrors handle_op_osi_proc_handles/handle_op_read_fds.

2. pid / create_time denormalized into struct syscall_event

Set from current at fire time. Host-side syscall consumers (e.g. process-tree exit tracking) can identify the task directly off the event — zero extra round-trips, and no read of a dying task's memory.

Validation

  • Compiles clean for 4.10 + 6.13 across armel / arm64 / x86_64.
  • osi_proc_node layout (size 56, field offsets) and the new syscall_event fields verified identical in the emitted DWARF on both kernel versions.
  • Exercised live in an armel/6.13 guest: OSI_PROC_ALL returns correct pid/ppid/comm with proper genealogy; exit tracking via syscall_event.pid records exits correctly.

Consumer

rehosting/penguin will add osi.get_all_procs() + a processes plugin and bump IGLOO_DRIVER_VERSION to a release carrying this op.

… into syscall_event

New op HYPER_OP_OSI_PROC_ALL walks for_each_process under rcu_read_lock and
returns a slim osi_proc_node {pid, ppid, create_time, uid/gid/euid/egid,
comm[16]} per user process in one paginated, RCU-consistent transaction. The
host was otherwise issuing one HYPER_OP_OSI_PROC per pid to build a process
tree -- 1+N round-trips (the portal region is a single page) and a snapshot
that can tear between the per-pid reads. comm comes from task->comm (kernel
memory only) and !mm tasks (kthreads) are skipped, so it never touches
userspace and is safe against exiting/stopped contexts. Auto-wired via the
portal_op_list X-macro (enum + dispatch + prototype).

Also denormalize current->pid and current->start_time into struct
syscall_event, populated at fire time. Host-side syscall consumers (e.g.
process-tree exit tracking) can then identify the task straight off the event
with zero extra round-trips and without reading the dying task's memory.

Compiles for 4.10 + 6.13 across armel/arm64/x86_64. osi_proc_node layout and
the new syscall_event fields verified identical in the emitted DWARF on both
kernel versions, and exercised live in an armel/6.13 guest.
@lacraig2
lacraig2 force-pushed the workspace/proctree-ux branch from 6c142c4 to bbb6aa9 Compare July 13, 2026 21:45
@lacraig2
lacraig2 merged commit c4ddd8d into main Jul 16, 2026
1 check passed
@lacraig2
lacraig2 deleted the workspace/proctree-ux branch July 16, 2026 13:33
lacraig2 added a commit to rehosting/penguin that referenced this pull request Jul 18, 2026
…B slice 1)

Adds a `processes` analysis plugin giving users and the MCP agent a model of
the guest process tree, with two surfaces:

- Live query API (portal generators, MCP-shaped stable returns): get(pid),
  tree() -> {roots:[...]}, snapshot(). Built on a single kernel-side bulk walk
  (osi.get_all_procs -> HYPER_OP_OSI_PROC_ALL) rather than 1+N per-pid reads.
- DB-backed lifecycle -> derived artifact: exec_event -> ProcStart,
  exit/exit_group -> ProcExit (identity read off syscall.pid, denormalized by
  the driver -- zero round-trips, no read of the dying task). At teardown the
  plugin flushes + queries the DB and renders system_map.yaml
  (ProcStart LEFT JOIN ProcExit on (pid, create_time); structured records +
  an ASCII tree literal block).

Supporting changes:
- osi.get_all_procs(): paginated decode of the slim osi_proc_node array.
- pengutils.events: lean ProcStart / ProcExit event types.
- loggers/db.py: synchronous flush() + query() (plugins unload in reverse load
  order, so a consumer's uninit runs before the DB's own final flush).

Tests (host-side, penguin.testing harness): test_processes.py (event->DB,
DB->render, live join via doubles), test_osi_bulk.py (real-ISF ABI roundtrip,
skips unless a driver build with the op is provided), test_db.py flush/query.

Depends on rehosting/igloo_driver#88 (HYPER_OP_OSI_PROC_ALL + syscall_event
pid). The live query API needs that driver; the DB-backed system_map works
today. Bump IGLOO_DRIVER_VERSION once #88 is released.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant