Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .all-contributorsrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"projectName": "module_powerplants",
"projectOwner": "modelblocks-org",
"repoType": "github",
"repoHost": "https://github.com",
"files": [
"README.md"
],
"imageSize": 100,
"commit": false,
"commitConvention": "none",
"contributors": [],
"contributorsPerLine": 7,
"linkToUsage": false,
"contributorsSortAlphabetically": true
}
7 changes: 4 additions & 3 deletions .copier-answers.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# Changes here will be overwritten by Copier
# !!!!! DO NOT MANUALLY MODIFY THIS FILE !!!!!
_commit: latest
_commit: v1.0.2
_src_path: https://github.com/modelblocks-org/data-module-template.git
author_email: i.ruizmanuel@tudelft.nl
author_family_name: Ruiz Manuel
author_given_name: Ivan
github_org: modelblocks-org
license: Apache-2.0
module_description: Aggregate global powerplant capacities into any resolution.
module_long_name: Modelblocks - Powerplants module
module_description: A data module to estimate global powerplant capacities for any
region in the world at any resolution.
module_long_name: Powerplants module
module_short_name: module_powerplants
12 changes: 6 additions & 6 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@ assignees: ''
---

**Describe the bug**
A clear and concise description of what the bug is.
A clear and concise description of the bug.

**To Reproduce**
**Steps to reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
2. Run '...'
3. Provide '...'
4. See error

**Expected behavior**
Expand All @@ -23,9 +23,9 @@ A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
**Environment (please complete the following information):**
- OS: [e.g. Linux Fedora 43, Windows 11...]
- Version [e.g. v0.1.1]
- Version: [e.g. v0.1.1]

**Additional context**
Add any other context about the problem here.
5 changes: 0 additions & 5 deletions .github/ISSUE_TEMPLATE/config.yaml

This file was deleted.

11 changes: 11 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
blank_issues_enabled: false
contact_links:
- name: Modelblocks project website
url: https://www.modelblocks.org/
about: Consult our website for general information on our initiative.
- name: Documentation
url: https://modelblocks.readthedocs.io/
about: Read our documentation for general questions and guidelines.
- name: Community chat
url: https://calliope-modelblocks.zulipchat.com
about: Reach out to our community on our official Zulip chat.
70 changes: 66 additions & 4 deletions .github/workflows/check-version.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,75 @@
# Check for changes in the upstream template. If changes are found, an issue is created
name: Template check.
# Check for changes in the upstream template. If changes are found, an issue is created.
name: Template check

on:
workflow_dispatch:
schedule:
- cron: '0 0 1 * *' # Runs at 00:00 UTC on the 1st day of every month

defaults:
run:
shell: bash --noprofile --norc -euo pipefail {0}

jobs:
copier-update:
copier-check-update:
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
uses: modelblocks-org/data-module-template/.github/workflows/template-check-version.yml@latest
env:
ISSUE_TITLE: Template update available
steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v5

- name: Check for template updates
id: copier_check_update
run: |
update_json="$(uvx --from 'copier>=9.15.2' copier check-update --output-format json)"
echo "$update_json"
{
echo "update_available=$(jq -r '.update_available // false' <<< "$update_json")"
echo "current_version=$(jq -r '.current_version // "unknown"' <<< "$update_json")"
echo "latest_version=$(jq -r '.latest_version // "unknown"' <<< "$update_json")"
} >> "$GITHUB_OUTPUT"

- name: Check for existing update issue
if: steps.copier_check_update.outputs.update_available == 'true'
id: existing_issue
run: |
issue_count="$(gh issue list \
--repo "${{ github.repository }}" \
--state open \
--search "$ISSUE_TITLE in:title" \
--json title \
--jq "map(select(.title == \"$ISSUE_TITLE\")) | length")"
if [[ "$issue_count" -gt 0 ]]; then
echo "exists=true" >> "$GITHUB_OUTPUT"
else
echo "exists=false" >> "$GITHUB_OUTPUT"
fi
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Open update issue
if: steps.copier_check_update.outputs.update_available == 'true' && steps.existing_issue.outputs.exists != 'true'
run: |
gh issue create \
--repo "${{ github.repository }}" \
--title "$ISSUE_TITLE" \
--body "A new version of the Modelblocks data module template is available.

Current template version: ${{ steps.copier_check_update.outputs.current_version }}
Latest template version: ${{ steps.copier_check_update.outputs.latest_version }}

Please update this project using:

\`\`\`shell
copier update --skip-answered --defaults
\`\`\`

Review the resulting changes before merging them."
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
7 changes: 3 additions & 4 deletions .github/workflows/pr-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,10 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.12"]
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Setup pixi
uses: prefix-dev/setup-pixi@v0.8.3
uses: prefix-dev/setup-pixi@v0.9.6
- name: Run integration tests
id: tests
run: pixi run test-integration
Expand All @@ -30,7 +29,7 @@ jobs:
continue-on-error: true
- name: Save integration logs
if: ${{ always() }}
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: integration-test-logs-${{ matrix.os }}
path: tests/integration/resources/module/logs
Expand Down
11 changes: 0 additions & 11 deletions .github/workflows/release.yml

This file was deleted.

3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ __pycache__
*.pyc

### Environments
.pixi/
.pixi/*
!.pixi/.gitignore

### Snakemake
.snakemake/
Expand Down
3 changes: 3 additions & 0 deletions .pixi/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
*
!.gitignore
!config.toml
1 change: 1 addition & 0 deletions .pixi/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pinning-strategy = "latest-up"
6 changes: 3 additions & 3 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,15 @@ repos:

# Python file formatting
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.9
rev: v0.15.18
hooks:
- id: ruff
- id: ruff-check
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format

# Snakemake file formatting
- repo: https://github.com/snakemake/snakefmt
rev: v1.0.0
rev: v2.0.2
hooks:
- id: snakefmt

Expand Down
2 changes: 1 addition & 1 deletion AUTHORS
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ This is the list of contributors for copyright purposes.

This does not necessarily list everyone who has contributed to this software's
code or documentation. For a full contributor list, see:
<https://github.com/calliope-project/module_powerplants/graphs/contributors>
<https://github.com/modelblocks-org/module_powerplants/graphs/contributors>

Ivan Ruiz Manuel, <i.ruizmanuel@tudelft.nl>
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
# https://citation-file-format.github.io/
cff-version: 1.2.0
message: If you use this software or data produced by it, please cite it using the metadata from this file.
title: "clio - module_powerplants: clio - Powerplants module"
repository: "https://github.com/calliope-project/module_powerplants"
title: "Modelblocks - module_powerplants: Powerplants module"
repository: "https://github.com/modelblocks-org/module_powerplants"
license: Apache-2.0
authors:
- given-names: Ivan
Expand Down
10 changes: 6 additions & 4 deletions INTERFACE.yaml
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
# Module Input-Output structure for automated doc. generation
# Module Input-Output structure for automated docs generation
convention_version: v1.0.0

pathvars:
snakemake_defaults:
logs:
default: "<logs>"
description: location of rule log files.
description: location of snakemake log files.
resources:
default: "<resources>"
description: "location of module resource files."
description: "location of module resource (input) files."
results:
default: "<results>"
description: "location of module results."
description: "location of module result (output) files."
user_resources:
shapes:
default: "<resources>/user/{shapes}/shapes.parquet"
Expand Down
62 changes: 57 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Modelblocks - Powerplants module
# Powerplants module

Aggregate global powerplant capacities into any resolution.
A data module to estimate global powerplant capacities for any region in the world at any resolution.

<!-- Place an attractive image of module outputs here -->
<p align="center">
Expand Down Expand Up @@ -29,7 +29,7 @@ Data processing steps:
1. Stable version-controlled global datasets are downloaded, including:
- Disaggregated powerplant statistics from [GEM](https://globalenergymonitor.org/), [Transition-Zero](https://www.transitionzero.org/products/solar-asset-mapper), and [GloHydroRES](https://zenodo.org/records/14526360).
- National-level statistics from the [EIA](https://www.eia.gov/).
2. Individual powerplants are prepared into seven different categories (bioenergy, fossil, geothermal, hydropower, nuclear, solar, wind).
2. Individual powerplants are prepared into point-source categories: bioenergy, fossil, geothermal, hydropower, nuclear, large_solar, and wind.
- Fuel-burning powerplants (fossil, bioenergy) are assigned unique fuel-classes depending on the combination of fuels they utilise.
- For utility-scale solar projects, satellite detected [TZ-Solar Asset Mapper](https://www.transitionzero.org/products/solar-asset-mapper) facilities are matched to [GEM-Global Solar Power Tracker](https://globalenergymonitor.org/) data to obtain a highly complete dataset of large-scale solar facilities.
3. Powerplants are selected according to the shapes file provided by the user. Depending on the configuration, their placement may be adjusted per technology and country.
Expand Down Expand Up @@ -70,6 +70,7 @@ Data processing steps:
1. Per country: $solar_{rooftopPV} = solar_{nationalStatistics} - solar_{largeScale}$.
2. A user-provided proxy raster is used to determine how to disaggregate $solar_{rooftopPV}$.
3. This proxy is used to determine the aggregated rooftop PV capacity per-shape.
4. The final aggregated `solar` output combines `large_solar` facilities with proxied rooftop PV capacity.

<p align="center">
<img src="./figures/rooftop_pv_proxy_MEX.png" width="60%">
Expand All @@ -78,6 +79,13 @@ Data processing steps:
> [!NOTE]
> Due to this assumption, the lifetime of rooftop PV capacity is left undetermined.

### Important assumptions

- The current reference year for operating capacity, national statistics adjustment, and status imputation is `2024`.
- User-provided shapes should add up to whole countries. This is required for national statistics and rooftop PV proxying to remain meaningful.
- Adjusted outputs rescale operating powerplants to match EIA national category totals. Future and retired plants are kept unchanged.
- `large_solar` is available as a point-source category for utility PV and CSP. `solar` is only available for aggregated outputs because rooftop PV is represented through a proxy raster rather than individual plant points.


## Configuration
<!-- Please describe how to configure this module below -->
Expand All @@ -87,7 +95,22 @@ Please consult the configuration [README](./config/README.md) and the [configura
## Input / output structure
<!-- Please describe input / output file placement below -->

Please consult the [interface file](./INTERFACE.yaml) for more information.
Required user inputs:

- `<shapes>`: GeoParquet file with the target regional disaggregation. It must contain `shape_id`, `country_id` (ISO-3), `shape_class` (`land` or `maritime`), and valid polygon geometry.
- `<proxy_rooftop_pv>`: GeoTIFF proxy raster to use for adjusted aggregated `solar` outputs.

Optional user inputs:

- `<imputed_powerplants>`: category-specific GeoParquet files with additional point-source powerplants. These can add missing facilities or replace source records when combined with the `excluded_ids` configuration value.
- `<wemi>`: Wind Energy Market Intelligence `.xls` file, required only when `category.wind.source` is set to `wemi`.

Main outputs:

- `<powerplants>`: disaggregated point-source powerplants. `unadjusted` outputs are available for `bioenergy`, `fossil`, `geothermal`, `hydropower`, `nuclear`, `large_solar`, and `wind`; `adjusted` outputs are available for the same categories except `large_solar`.
- `<aggregated_capacity>`: capacity aggregated to the user-provided shapes. `bioenergy`, `fossil`, `geothermal`, `hydropower`, `nuclear`, and `wind` are available as `adjusted` or `unadjusted`; `large_solar` is available as `unadjusted`; `solar` is available as `adjusted`.

Please consult the [interface file](./INTERFACE.yaml) for exact path variables and wildcards.

## Development
<!-- Please do not modify this templated section -->
Expand All @@ -101,6 +124,21 @@ cd module_powerplants
pixi install --all
```

Please be aware that this is a multi-environment project (see [pixi.toml](./pixi.toml) for details).
- `default`: used for development and integration testing.
Because it contains `Snakemake`, `conda` and `pytest` as dependencies it **should not be used** in `Snakemake` rules.
- `module`: contains minimal dependencies used in `Snakemake` rules.
If modified, be sure to export it to `Snakemake` so it can be recreated by module users:

```shell
# create module.yaml and conda-spec pin files in workflow/envs/
pixi run export-snakemake-env module
```


## Testing
<!-- Please do not modify this templated section -->

For testing, simply run:

```shell
Expand All @@ -118,7 +156,7 @@ snakemake --use-conda --cores 2 # run the workflow!
## References
<!-- Please provide thorough referencing below -->

This module is based on the following research and datasets.
This module is based on the following research and datasets:
For specific versions please consult our [stable dataset repository](https://doi.org/10.5281/zenodo.16037139).

* **Global Energy Monitor datasets.** <https://globalenergymonitor.org/>. License: CC BY 4.0.
Expand All @@ -136,3 +174,17 @@ U.S. Energy Information Administration (Oct 2008). <https://www.eia.gov/internat
* **Satellite Utility-scale PV dataset.**
TransitionZero Solar Asset Mapper, TransitionZero. <https://www.transitionzero.org/products/solar-asset-mapper>.
License: CC BY-NC 4.0.


## Contributors ✨

Thanks goes to these wonderful people, sorted alphabetically ([emoji key](https://allcontributors.org/en/reference/emoji-key/)):

<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->
<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->
<!-- ALL-CONTRIBUTORS-LIST:END -->

This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
Loading
Loading