Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
900f01d
Adds an abstract pool and data class for pre-multi-backend refactor
bgailleton Jul 20, 2026
815c33a
Add the taichi field data handle
bgailleton Jul 20, 2026
6471865
Lay out the abstract classes for compiling the functions, helper/kern…
bgailleton Jul 20, 2026
58143a9
Implement taichi pool
bgailleton Jul 20, 2026
105f9ca
Implementing the taichi backend for the new context factory
bgailleton Jul 20, 2026
9b90f7c
prepare the new file structure for the module. Separate core form the…
bgailleton Jul 21, 2026
ed7a8f0
Massive refactore of the experimental core. Getting toward stable api
bgailleton Jul 22, 2026
d7785f4
Simplify taichi vs quadrants backend as they are very similar in syntax
bgailleton Jul 23, 2026
4cd7d25
clean now unused classes
bgailleton Jul 23, 2026
602c88f
Further clean and comments
bgailleton Jul 23, 2026
e85216d
Improve cache and a bit of cleaning
bgailleton Jul 23, 2026
479624e
Add a docstring warning about offline fastcache for quadrants backend…
bgailleton Jul 23, 2026
565bc77
Rework the comments (some were polluted by a LLM)
bgailleton Jul 23, 2026
734eb94
improve base.py docstring
bgailleton Jul 23, 2026
8201eca
Merge the different bags in a single entity
bgailleton Jul 23, 2026
686bdfb
Refactor to builder/composer state #1
bgailleton Jul 24, 2026
fb3c32e
Refactor to builder/composer state #2
bgailleton Jul 24, 2026
df9b78f
Refactor to builder/composer state #3
bgailleton Jul 24, 2026
fa5b500
Refactor to builder/composer state #4
bgailleton Jul 24, 2026
65cbd09
Refactor to builder/composer state #5
bgailleton Jul 24, 2026
5b085e7
adding core example to demonstrate kernel param and helepr builders
bgailleton Jul 24, 2026
adb318d
Add example with Rouine
bgailleton Jul 27, 2026
179ea53
Work on the cache. Taichi has some limitations
bgailleton Jul 27, 2026
1725e9d
Update README
bgailleton Jul 27, 2026
b0cb2ca
Merge branch 'TopoToolbox:main' into main
bgailleton Jul 27, 2026
20130e7
Fix a cupy key error and other minor bug
bgailleton Jul 28, 2026
d8fbac8
Add the grid bag helpers
bgailleton Jul 28, 2026
6bef1c1
gjj0
bgailleton Jul 28, 2026
95aaad6
Clean some routine inherited from first designs and remove unused bit…
bgailleton Jul 28, 2026
10eb0bb
Clean and restructure the core context base.py (1000+ LoC for multipl…
bgailleton Jul 29, 2026
bfb279c
Clean imports
bgailleton Jul 29, 2026
1f08dc1
More cleaning and add the grid manipulation example
bgailleton Jul 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 40 additions & 102 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,125 +1,63 @@
# PyFastFlow

**First full-GPU geomorphological and hydrodynamic toolbox powered by Taichi-lang**
> ⚠️ **Experimental.** This describes the in-progress v1 core. The API is
> settling and will change.

[![Python](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)
[![Taichi](https://img.shields.io/badge/taichi-≥1.6.0-orange.svg)](https://github.com/taichi-dev/taichi)
[![License](https://img.shields.io/badge/license-Custom-red.svg)](./LICENSE)

## Overview

**A lot of work will be put into finalising v1.0 in Q1 2026 + paper**
**GPU routines for Earth-surface processes — portable across backends.**

**02/03/2026: I am currently in a hackathlon to refactor the interface to fix it before submission, stay tuned**
Two things in one:

- **A portable GPU-routine engine.** Compose parameters, helpers and
multi-kernel *routines* once, and run them on **Taichi**, **Quadrants**,
or **CuPy** — same composition model, backend-native kernels.
- **A geomorphology toolbox built on it.** Flow routing, flooding, and
landscape evolution, engineered for grids of hundreds of millions of nodes.

<!-- PyFastFlow is a high-performance Python package for general geomorphological and hydrodynamic computations on GPU. Built on the Taichi programming language with bindings to `pytopotoolbox`, it provides efficient, portable parallel algorithms for flow accumulation, depression filling, (simple) shallow water flow modeling, and landscape evolution simulations.

The fast flow routines are implemented following **Jain et al., 2024** [📝](https://www-sop.inria.fr/reves/Basilic/2024/JKGFC24/FastFlowPG2024_Author_Version.pdf), delivering state-of-the-art performance for GPU-oriented flow computation (flow accumulation and local minima handling).
[![Python](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-CeCILL%20v2.1-red.svg)](./LICENSE)

The flooding/hydraulic modelling implements a GPU version of GraphFlood (Gailleton et al., 2024) for stationary solutions and an implementation of Bates et al., 2010 and inertial flow.
## Why

## 🚀 Key Features
Writing flexible GPU kernels for physics simulation can be time-consuming. Performance on GPU is highly affected by memory layout. Hence simple things such as a constant _vs_ a 2D varying parameter, or stencil operations (periodic boundaries, no-data, outlets, ...), each push you toward a different hand-tuned kernel — multiplied by every backend you want to run on. PyFastFlow lets you write a routine once and *bind* how each parameter and stencil is realised, then specialises it per backend and layout.

### **Flow Routing & Hydrology**
- **GPU-accelerated flow routing**: Steepest descent algorithms with multiple boundary conditions
- **Advanced depression filling**: Priority flood and carving algorithms for handling closed basins
- **Flow accumulation**: Efficient rake-and-compress algorithms for parallel tree traversal
- **Boundary conditions**: Normal, periodic (EW/NS), and custom per-node boundary handling
## Example

### **2D Shallow Water Flow (Flood Modeling)**
- **LisFlood implementation**: Bates et al. 2010 explicit finite difference scheme
- **GraphFlood**: Fast approximation of the 2D shallow water 2D stationary solution
- **Manning's friction**: Configurable roughness coefficients
- **Precipitation input**: Rainfall and boundary conditions
```python
import taichi as ti; ti.init(arch=ti.gpu)
from pyfastflow.experimental.core.context.taichi_backend import (
TaichiKernelBuilder, TaichiParameter,
)
from pyfastflow.experimental.core.pool.taichi_pool import TaichiPool

### **Landscape Evolution**
- **Refactor in progress, not usable right now**
- **Stream Power Law (SPL)**: Bedrock erosion with detachment and transport-limited models
- **Sediment transport**: Erosion-deposition coupling with transport capacity
- **Tectonic uplift**: Block and spatially-varying uplift patterns
- **Non linear hillslope**: Based on Carretier et al. 2016
pool = TaichiPool()
D = TaichiParameter("D", ti.f32, mode="scalar", value=1e-2, pool=pool)

### **Visualization & Analysis**
- **Hillshading**: GPU-accelerated terrain shading with multiple illumination models
- **Real-time 3D visualization**: Interactive terrain rendering with using mordernGL (Experimental)
def diffuse(z_out: ti.template(), z_in: ti.template()):
for i, j in z_in:
z_out[i, j] = z_in[i, j] + D.get(0) * (
z_in[i-1, j] + z_in[i+1, j] + z_in[i, j-1] + z_in[i, j+1] - 4.0 * z_in[i, j])

### **Performance & Memory**
- **Field pooling system**: Efficient GPU memory management for taichi with automatic field reuse/clean
- **General Parallel algorithms**: Utilities written in taichi (e.g. Blelloch parallel scan, ping-pong, swap, ...)
- **Scalable**: Handles large grids (up to hundreds of millions of nodes) efficiently
step = TaichiKernelBuilder().bind("D", D).ingest(diffuse).compile()
```

## 📦 Installation
The parameter / helper / routine API is identical on every backend; only the
kernel body changes — a Python `def` on Taichi and Quadrants, CUDA source on
CuPy. The same landscape-evolution model, built once per backend, lives in
[`examples/core/lem/`](./examples/core/lem/).

### From PyPI (will be available at launch)
```bash
pip install pyfastflow
```
## Install

### From Source
```bash
git clone https://github.com/bgailleton/pyfastflow.git
cd pyfastflow
pip install -e .
pip install -e . # from source
```

### Requirements
- **Python** ≥ 3.9
- **Taichi** ≥ 1.4.0
- **NumPy** ≥ 1.20.0
- **Matplotlib** ≥ 3.3.0 (for visualization)

## 🏃‍♂️ Quick Start

WIP


## 🔬 Scientific Background

### Flow Routing Algorithms

- **Flow routing through local minimas**: Improved depression handling with Cordonnier et al. (2019) adapted to GPU by Jain et al. (2024)
- **Rake-and-Compress**: Parallel flow accumulation following Jain et al. (2024)

### Shallow Water Flow
- **LisFlood**: Explicit finite difference scheme (Bates et al., 2010)
- **GraphFlood**: Graph-based implicit flow routing (Gaileton et al., 2024)

### Landscape Evolution
- **Stream Power Law**: E = K × A^m × S^n erosion model (Howard and Kerby 1983)
- **Transport-Limited**: Erosion-transport-deposition coupling (Davy and Lague, 2009 style - WIP)
- **Non linear hillslope**: adapted from CIDRE with a semi-implicit scheme (Carretier et al., 2016 - WIP)


## 📄 License

CeCILL v2.1

## 📖 Citation

If you use PyFastFlow in your research, please contact me.

### Related Publications
- Jain, A., et al. (2024). "Fast Flow Computation using GPU". *Proceedings of Graphics Interface 2024*.
- Bates, P. D., et al. (2010). "A simple inertial formulation of the shallow water equations". *Journal of Hydrology*.

## 👥 Authors

**Main Authors:**
- **Boris Gailleton** - Géosciences Rennes - boris.gailleton@univ-rennes.fr
- **Guillaume Cordonnier** - INRIA Sophia Antipolis
Requires Python ≥ 3.9, and the backend(s) you target (Taichi, Quadrants, CuPy).

## 🐛 Issues & Support
## Status

- **Bug Reports**: [GitHub Issues](https://github.com/bgailleton/pyfastflow/issues)
- **Feature Requests**: [GitHub Discussions](https://github.com/bgailleton/pyfastflow/discussions)
- **Documentation**: [Read the Docs](https://pyfastflow.readthedocs.io/) *(coming soon - hopefully)*
Core interface under active refactor toward v1.0 (+ paper). Flow/flood routines
are being ported onto the new core; landscape evolution is WIP.

## 🔗 Links
## License & authors

- **Repository**: https://github.com/bgailleton/pyfastflow
- **Documentation**: https://pyfastflow.readthedocs.io/ *(coming soon)*
- **PyPI Package**: https://pypi.org/project/pyfastflow/ *(coming soon)*
- **Jain et al. 2024 Paper**: [PDF](https://www-sop.inria.fr/reves/Basilic/2024/JKGFC24/FastFlowPG2024_Author_Version.pdf)
-->
---
CeCILL v2.1 — Boris Gailleton (Géosciences Rennes) · Guillaume Cordonnier (INRIA).
Loading