Skip to content

Big refactor of the core + context - #12

Draft
bgailleton wants to merge 31 commits into
TopoToolbox:mainfrom
bgailleton:main
Draft

Big refactor of the core + context#12
bgailleton wants to merge 31 commits into
TopoToolbox:mainfrom
bgailleton:main

Conversation

@bgailleton

@bgailleton bgailleton commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Current framework limitations

In short, there are two main issues.

  1. Contexts are too convoluted and inflexible.
    The current design relies on many context types, each with its own parameters, kernels, helpers, and related logic. This makes the framework difficult to understand, hard to extend, and overly monolithic.

  2. The Taichi ecosystem has slowed down.
    While taichi is still maintained, its pace of development has decreased significantly. This is a good opportunity to redesign the framework around a single composition philosophy while remaining backend-agnostic. In particular, quadrants—a fork of taichi that has been actively developed since mid-2025—provides a promising alternative backend.

Refactor

The new architecture is significantly more modular. It is built around a small set of abstract classes, ordered from low-level building blocks to high-level abstractions.

  • Parameter: Represents a physical parameter that can be constant, scalar, or spatially varying (e.g. the erodability coefficient K).

  • Helper: Represents a device-side utility function. For example, a kernel may call a placeholder such as get_left_neighbour, which is bound at compile time to an implementation such as _get_left_neighbour_periodic, _get_left_neighbour_normal, or _get_left_neighbour_nodata_check. This allows boundary conditions and similar behaviors to be changed without modifying the kernel itself.

  • Bag: A structured collection of Parameters and Helpers that provides a clean namespace for kernel dependencies.

  • Kernel: Encapsulates the GPU computation itself. A kernel is bound to a Bag containing its parameters and helpers. Changing these dependencies requires recompilation, but the kernel implementation remains unchanged.

  • Routine: A collection of related Kernels. It automates resource management (e.g. acquiring and releasing temporary arrays from the memory pool), enforces a unique dependency bag, and enables backend-specific optimizations such as kernel fusion (taichi and quadrants) or CUDA Graph execution (cupy.RawKernel).

  • Program: The highest-level abstraction. A program represents a complete, ready-to-use simulation, including routines, owned data, accessors, setters, and conversion utilities (e.g. to numpy).

These abstract interfaces are implemented for multiple backends. Currently, three are supported:

  • taichi: the legacy, mature backend.
  • quadrants: a newer, actively developed backend with a taichi-like programming model.
  • cupy.RawKernel: a low-level CUDA backend. ⚠️ Kernels must be written in CUDA C, making this backend primarily intended for experienced users.

Once the building blocks have been assembled, a Kernel, Routine, or Program can be compiled and used in a simulation.

TODO

  • Define abstract base classes for Parameter, Helper, Kernel, and Routine.
  • Implement them for taichi, quadrants, and cupy.RawKernel.
  • Add an example using the new core architecture.
  • Replace GridContext with mergeable Bags of parameters and helpers.
  • Add basic tests for regression protection.
  • If the API proves stable, migrate the remaining contexts while cleaning up obsolete code.
  • Define the abstract Program interface.
  • Implement backend-specific Program classes.

bgailleton and others added 25 commits July 20, 2026 11:01
… - not possible because my param injections are ot visible by the latter
@bgailleton

Copy link
Copy Markdown
Collaborator Author

if the ref arrives anywhere, ignore the links to topotoolbox issues - this was not intended

@bgailleton bgailleton self-assigned this Jul 27, 2026
@bgailleton
bgailleton marked this pull request as draft July 27, 2026 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant