Skip to content

Add native Tofu (uTofu) gather-scatter backend#2602

Open
njansson wants to merge 2 commits into
ExtremeFLOW:developfrom
njansson:feature/utofu
Open

Add native Tofu (uTofu) gather-scatter backend#2602
njansson wants to merge 2 commits into
ExtremeFLOW:developfrom
njansson:feature/utofu

Conversation

@njansson

Copy link
Copy Markdown
Collaborator

Adds a native Tofu (uTofu) one-sided RDMA gather-scatter backend for
Fugaku/Tofu-D, as a faster successor to the coarray (CAF) backend.

  • Selected with NEKO_GS_COMM=UTOFU (or comm_bcknd=GS_COMM_UTOFU);
    built with --with-utofu[=DIR] (links -ltofucom, gated on HAVE_UTOFU).
  • A send is a single utofu_put straight into the neighbour's receive
    buffer, with the slab tag fused into the put's edata — one network
    op per peer, no separate atomic/credit message.
  • Per-instance send/recv buffers and a private receive VCQ/MRQ per
    gs_t; double-buffered send/receive with an unpack-as-ready receive
    loop. Packing is parallel over the OpenMP team; puts are issued by the
    master thread (concurrent puts across VCQs are unsupported on TofuD).
  • NEKO_GS_UTOFU_NTNI selects the number of injection TNIs (default 1);
    NEKO_GS_UTOFU_CACHE_INJECT toggles Tofu cache injection.
  • Docs (appendices, performance, installation, man page) and build deps
    updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CPU CPU Fujitsu Fujitsu performance Performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants