From 495bd29dbc1c8f699f37c514769798f75c617e39 Mon Sep 17 00:00:00 2001 From: nsheff Date: Fri, 13 Feb 2026 11:29:57 -0500 Subject: [PATCH 1/5] Add pypiper v0.15.0 changelog entry Co-Authored-By: Claude Opus 4.5 --- docs/pypiper/changelog.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/docs/pypiper/changelog.md b/docs/pypiper/changelog.md index 96d4ede3..9817f135 100644 --- a/docs/pypiper/changelog.md +++ b/docs/pypiper/changelog.md @@ -1,5 +1,29 @@ # Changelog +## [0.15.0] -- 2026-02-13 + +### Changed +- Migrated from setup.py to pyproject.toml with hatchling backend +- Replaced Black/isort with Ruff for linting and formatting +- Updated GitHub Actions to v4/v5 with Python 3.10-3.14 CI matrix +- Removed Python 2 compatibility code and legacy packaging files +- Version now sourced from importlib.metadata (removed _version.py) +- Converted all docstrings to Google-style formatting +- Consolidated documentation into pepspec; removed local docs/ and mkdocs.yml +- Updated README badges and documentation links to pep.databio.org/pypiper + +### Added +- `clean_add()` now accepts None values, enabling safe use with conditionally-assigned variables +- Pipestat version displayed in pipeline version log output +- `pipestat_lenient` parameter on PipelineManager (defaults to True) + +### Removed +- Removed magic schema discovery (`default_pipestat_output_schema()`) +- Removed `pipestat_lenient` parameter from lower-level API (now on PipelineManager) + +### Improved +- Test quality: added missing assertions, removed permanently-skipped tests, added shared fixtures + ## [0.14.5] -- 2025-09-22 ### Changed - Remove veracitools dependency [#233](https://github.com/databio/pypiper/issues/233) From 6533eab8db23db942df10efe5f8b0b3fcd1062cf Mon Sep 17 00:00:00 2001 From: nsheff Date: Tue, 17 Feb 2026 09:04:37 -0500 Subject: [PATCH 2/5] update pipestat docs --- docs/pipestat/code/api-quickstart.md | 2 +- docs/pipestat/code/python-tutorial.md | 2 +- docs/pipestat/notebooks/api-quickstart.ipynb | 2 +- docs/pipestat/notebooks/python-tutorial.ipynb | 2 +- docs/pipestat/pipestat-schema.md | 3 +-- docs/pipestat/pipestat-specification.md | 3 +-- docs/pipestat/tests/data/sample_output_schema.yaml | 1 - docs/pypiper/changelog.md | 4 +--- docs/pypiper/features.md | 2 +- 9 files changed, 8 insertions(+), 13 deletions(-) diff --git a/docs/pipestat/code/api-quickstart.md b/docs/pipestat/code/api-quickstart.md index 613a64e1..71f1956e 100644 --- a/docs/pipestat/code/api-quickstart.md +++ b/docs/pipestat/code/api-quickstart.md @@ -38,7 +38,7 @@ print(psm.schema) - name_of_something : {'type': 'string', 'description': 'Name of something'} - switch_value : {'type': 'boolean', 'description': 'Is the switch on or off'} - output_file : {'description': 'This a path to the output file', 'type': 'object', 'object_type': 'file', 'properties': {'path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'title']} - - output_image : {'description': 'This a path to the output image', 'type': 'object', 'object_type': 'image', 'properties': {'path': {'type': 'string'}, 'thumbnail_path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'thumbnail_path', 'title']} + - output_image : {'description': 'This a path to the output image', 'type': 'object', 'object_type': 'image', 'properties': {'path': {'type': 'string'}, 'thumbnail_path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'title']} - md5sum : {'type': 'string', 'description': 'MD5SUM of an object', 'highlight': True} Status properties: - None diff --git a/docs/pipestat/code/python-tutorial.md b/docs/pipestat/code/python-tutorial.md index edf7ba0c..08babb32 100644 --- a/docs/pipestat/code/python-tutorial.md +++ b/docs/pipestat/code/python-tutorial.md @@ -154,7 +154,7 @@ psm.result_schemas 'properties': {'path': {'type': 'string'}, 'thumbnail_path': {'type': 'string'}, 'title': {'type': 'string'}}, - 'required': ['path', 'thumbnail_path', 'title']}, + 'required': ['path', 'title']}, 'md5sum': {'type': 'string', 'description': 'MD5SUM of an object', 'highlight': True}} diff --git a/docs/pipestat/notebooks/api-quickstart.ipynb b/docs/pipestat/notebooks/api-quickstart.ipynb index 2e3bba39..a440c0c9 100644 --- a/docs/pipestat/notebooks/api-quickstart.ipynb +++ b/docs/pipestat/notebooks/api-quickstart.ipynb @@ -65,7 +65,7 @@ " - name_of_something : {'type': 'string', 'description': 'Name of something'}\n", " - switch_value : {'type': 'boolean', 'description': 'Is the switch on or off'}\n", " - output_file : {'description': 'This a path to the output file', 'type': 'object', 'object_type': 'file', 'properties': {'path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'title']}\n", - " - output_image : {'description': 'This a path to the output image', 'type': 'object', 'object_type': 'image', 'properties': {'path': {'type': 'string'}, 'thumbnail_path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'thumbnail_path', 'title']}\n", + " - output_image : {'description': 'This a path to the output image', 'type': 'object', 'object_type': 'image', 'properties': {'path': {'type': 'string'}, 'thumbnail_path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'title']}\n", " - md5sum : {'type': 'string', 'description': 'MD5SUM of an object', 'highlight': True}\n", " Status properties:\n", " - None\n" diff --git a/docs/pipestat/notebooks/python-tutorial.ipynb b/docs/pipestat/notebooks/python-tutorial.ipynb index 8ed639f4..4609dd9a 100644 --- a/docs/pipestat/notebooks/python-tutorial.ipynb +++ b/docs/pipestat/notebooks/python-tutorial.ipynb @@ -249,7 +249,7 @@ " 'properties': {'path': {'type': 'string'},\n", " 'thumbnail_path': {'type': 'string'},\n", " 'title': {'type': 'string'}},\n", - " 'required': ['path', 'thumbnail_path', 'title']},\n", + " 'required': ['path', 'title']},\n", " 'md5sum': {'type': 'string',\n", " 'description': 'MD5SUM of an object',\n", " 'highlight': True}}" diff --git a/docs/pipestat/pipestat-schema.md b/docs/pipestat/pipestat-schema.md index 95e4d00e..de163aa1 100644 --- a/docs/pipestat/pipestat-schema.md +++ b/docs/pipestat/pipestat-schema.md @@ -22,7 +22,7 @@ Pipestat also extends the json schema vocabulary by adding two _additional_ type - `title`: human readable description of the file - `image`: - `path`: path to the reported image, usually PDF - - `thumbnail`: path to the reported thumbnail, usually PNG or JPEG + - `thumbnail_path` (optional): path to the reported thumbnail, usually PNG or JPEG. If not provided, falls back to `path`. - `title`: human readable description of the image @@ -147,7 +147,6 @@ $defs: type: string required: - path - - thumbnail_path - title file: type: object diff --git a/docs/pipestat/pipestat-specification.md b/docs/pipestat/pipestat-specification.md index 1641c903..7c5d8b47 100644 --- a/docs/pipestat/pipestat-specification.md +++ b/docs/pipestat/pipestat-specification.md @@ -37,7 +37,7 @@ Importantly, pipestat extends the jsonschema vocabulary by adding two additional - `title`: human readable description of the file - `image`: - `path`: path to the reported image, usually PDF - - `thumbnail`: path to the reported thumbnail, usually PNG or JPEG + - `thumbnail_path` (optional): path to the reported thumbnail, usually PNG or JPEG. If not provided, falls back to `path`. - `title`: human readable description of the image # Pipestat output schema @@ -165,7 +165,6 @@ $defs: type: string required: - path - - thumbnail_path - title file: type: object diff --git a/docs/pipestat/tests/data/sample_output_schema.yaml b/docs/pipestat/tests/data/sample_output_schema.yaml index eb7bbd2d..c3a4ef79 100644 --- a/docs/pipestat/tests/data/sample_output_schema.yaml +++ b/docs/pipestat/tests/data/sample_output_schema.yaml @@ -43,7 +43,6 @@ $defs: type: string required: - path - - thumbnail_path - title file: type: object diff --git a/docs/pypiper/changelog.md b/docs/pypiper/changelog.md index 9817f135..dab30811 100644 --- a/docs/pypiper/changelog.md +++ b/docs/pypiper/changelog.md @@ -10,16 +10,14 @@ - Version now sourced from importlib.metadata (removed _version.py) - Converted all docstrings to Google-style formatting - Consolidated documentation into pepspec; removed local docs/ and mkdocs.yml -- Updated README badges and documentation links to pep.databio.org/pypiper +- Updated README badges and documentation links ### Added - `clean_add()` now accepts None values, enabling safe use with conditionally-assigned variables - Pipestat version displayed in pipeline version log output -- `pipestat_lenient` parameter on PipelineManager (defaults to True) ### Removed - Removed magic schema discovery (`default_pipestat_output_schema()`) -- Removed `pipestat_lenient` parameter from lower-level API (now on PipelineManager) ### Improved - Test quality: added missing assertions, removed permanently-skipped tests, added shared fixtures diff --git a/docs/pypiper/features.md b/docs/pypiper/features.md index 93b62635..7595bb91 100644 --- a/docs/pypiper/features.md +++ b/docs/pypiper/features.md @@ -34,4 +34,4 @@ Pypiper closes pipelines gracefully on interrupt or termination signals, convert ![](img/recovery.svg) **Dynamic recovery** -If a job is interrupted (with SIGINT or SIGTERM), either from a user or by a cluster resource manager, pypiper will set a `dynamic recovery` flag. The next time the run is started, it will automatically pick up where it left off. This makes pypiper pipelines `automatically pre-emption ready`, so they can be immediately deployed on servers where jobs may be pre-emptied. +If a job is interrupted (with SIGINT or SIGTERM), either from a user or by a cluster resource manager, pypiper will set a `dynamic recovery` flag. The next time the run is started, it will automatically pick up where it left off. This makes pypiper pipelines `automatically pre-emption ready`, so they can be immediately deployed on servers where jobs may be pre-empted. From 3b8969800e97e92ea97367a4d50f7e32e479e1d7 Mon Sep 17 00:00:00 2001 From: nsheff Date: Fri, 13 Feb 2026 11:29:57 -0500 Subject: [PATCH 3/5] Add pypiper v0.15.0 changelog entry Co-Authored-By: Claude Opus 4.5 --- docs/pypiper/changelog.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/docs/pypiper/changelog.md b/docs/pypiper/changelog.md index 96d4ede3..9817f135 100644 --- a/docs/pypiper/changelog.md +++ b/docs/pypiper/changelog.md @@ -1,5 +1,29 @@ # Changelog +## [0.15.0] -- 2026-02-13 + +### Changed +- Migrated from setup.py to pyproject.toml with hatchling backend +- Replaced Black/isort with Ruff for linting and formatting +- Updated GitHub Actions to v4/v5 with Python 3.10-3.14 CI matrix +- Removed Python 2 compatibility code and legacy packaging files +- Version now sourced from importlib.metadata (removed _version.py) +- Converted all docstrings to Google-style formatting +- Consolidated documentation into pepspec; removed local docs/ and mkdocs.yml +- Updated README badges and documentation links to pep.databio.org/pypiper + +### Added +- `clean_add()` now accepts None values, enabling safe use with conditionally-assigned variables +- Pipestat version displayed in pipeline version log output +- `pipestat_lenient` parameter on PipelineManager (defaults to True) + +### Removed +- Removed magic schema discovery (`default_pipestat_output_schema()`) +- Removed `pipestat_lenient` parameter from lower-level API (now on PipelineManager) + +### Improved +- Test quality: added missing assertions, removed permanently-skipped tests, added shared fixtures + ## [0.14.5] -- 2025-09-22 ### Changed - Remove veracitools dependency [#233](https://github.com/databio/pypiper/issues/233) From aff5ae20efe66008804e9f2d84b83adf052ebfe7 Mon Sep 17 00:00:00 2001 From: nsheff Date: Tue, 17 Feb 2026 09:04:37 -0500 Subject: [PATCH 4/5] update pipestat docs --- docs/pipestat/code/api-quickstart.md | 2 +- docs/pipestat/code/python-tutorial.md | 2 +- docs/pipestat/notebooks/api-quickstart.ipynb | 2 +- docs/pipestat/notebooks/python-tutorial.ipynb | 2 +- docs/pipestat/pipestat-schema.md | 3 +-- docs/pipestat/pipestat-specification.md | 3 +-- docs/pipestat/tests/data/sample_output_schema.yaml | 1 - docs/pypiper/changelog.md | 4 +--- docs/pypiper/features.md | 2 +- 9 files changed, 8 insertions(+), 13 deletions(-) diff --git a/docs/pipestat/code/api-quickstart.md b/docs/pipestat/code/api-quickstart.md index 613a64e1..71f1956e 100644 --- a/docs/pipestat/code/api-quickstart.md +++ b/docs/pipestat/code/api-quickstart.md @@ -38,7 +38,7 @@ print(psm.schema) - name_of_something : {'type': 'string', 'description': 'Name of something'} - switch_value : {'type': 'boolean', 'description': 'Is the switch on or off'} - output_file : {'description': 'This a path to the output file', 'type': 'object', 'object_type': 'file', 'properties': {'path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'title']} - - output_image : {'description': 'This a path to the output image', 'type': 'object', 'object_type': 'image', 'properties': {'path': {'type': 'string'}, 'thumbnail_path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'thumbnail_path', 'title']} + - output_image : {'description': 'This a path to the output image', 'type': 'object', 'object_type': 'image', 'properties': {'path': {'type': 'string'}, 'thumbnail_path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'title']} - md5sum : {'type': 'string', 'description': 'MD5SUM of an object', 'highlight': True} Status properties: - None diff --git a/docs/pipestat/code/python-tutorial.md b/docs/pipestat/code/python-tutorial.md index edf7ba0c..08babb32 100644 --- a/docs/pipestat/code/python-tutorial.md +++ b/docs/pipestat/code/python-tutorial.md @@ -154,7 +154,7 @@ psm.result_schemas 'properties': {'path': {'type': 'string'}, 'thumbnail_path': {'type': 'string'}, 'title': {'type': 'string'}}, - 'required': ['path', 'thumbnail_path', 'title']}, + 'required': ['path', 'title']}, 'md5sum': {'type': 'string', 'description': 'MD5SUM of an object', 'highlight': True}} diff --git a/docs/pipestat/notebooks/api-quickstart.ipynb b/docs/pipestat/notebooks/api-quickstart.ipynb index 2e3bba39..a440c0c9 100644 --- a/docs/pipestat/notebooks/api-quickstart.ipynb +++ b/docs/pipestat/notebooks/api-quickstart.ipynb @@ -65,7 +65,7 @@ " - name_of_something : {'type': 'string', 'description': 'Name of something'}\n", " - switch_value : {'type': 'boolean', 'description': 'Is the switch on or off'}\n", " - output_file : {'description': 'This a path to the output file', 'type': 'object', 'object_type': 'file', 'properties': {'path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'title']}\n", - " - output_image : {'description': 'This a path to the output image', 'type': 'object', 'object_type': 'image', 'properties': {'path': {'type': 'string'}, 'thumbnail_path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'thumbnail_path', 'title']}\n", + " - output_image : {'description': 'This a path to the output image', 'type': 'object', 'object_type': 'image', 'properties': {'path': {'type': 'string'}, 'thumbnail_path': {'type': 'string'}, 'title': {'type': 'string'}}, 'required': ['path', 'title']}\n", " - md5sum : {'type': 'string', 'description': 'MD5SUM of an object', 'highlight': True}\n", " Status properties:\n", " - None\n" diff --git a/docs/pipestat/notebooks/python-tutorial.ipynb b/docs/pipestat/notebooks/python-tutorial.ipynb index 8ed639f4..4609dd9a 100644 --- a/docs/pipestat/notebooks/python-tutorial.ipynb +++ b/docs/pipestat/notebooks/python-tutorial.ipynb @@ -249,7 +249,7 @@ " 'properties': {'path': {'type': 'string'},\n", " 'thumbnail_path': {'type': 'string'},\n", " 'title': {'type': 'string'}},\n", - " 'required': ['path', 'thumbnail_path', 'title']},\n", + " 'required': ['path', 'title']},\n", " 'md5sum': {'type': 'string',\n", " 'description': 'MD5SUM of an object',\n", " 'highlight': True}}" diff --git a/docs/pipestat/pipestat-schema.md b/docs/pipestat/pipestat-schema.md index 95e4d00e..de163aa1 100644 --- a/docs/pipestat/pipestat-schema.md +++ b/docs/pipestat/pipestat-schema.md @@ -22,7 +22,7 @@ Pipestat also extends the json schema vocabulary by adding two _additional_ type - `title`: human readable description of the file - `image`: - `path`: path to the reported image, usually PDF - - `thumbnail`: path to the reported thumbnail, usually PNG or JPEG + - `thumbnail_path` (optional): path to the reported thumbnail, usually PNG or JPEG. If not provided, falls back to `path`. - `title`: human readable description of the image @@ -147,7 +147,6 @@ $defs: type: string required: - path - - thumbnail_path - title file: type: object diff --git a/docs/pipestat/pipestat-specification.md b/docs/pipestat/pipestat-specification.md index 1641c903..7c5d8b47 100644 --- a/docs/pipestat/pipestat-specification.md +++ b/docs/pipestat/pipestat-specification.md @@ -37,7 +37,7 @@ Importantly, pipestat extends the jsonschema vocabulary by adding two additional - `title`: human readable description of the file - `image`: - `path`: path to the reported image, usually PDF - - `thumbnail`: path to the reported thumbnail, usually PNG or JPEG + - `thumbnail_path` (optional): path to the reported thumbnail, usually PNG or JPEG. If not provided, falls back to `path`. - `title`: human readable description of the image # Pipestat output schema @@ -165,7 +165,6 @@ $defs: type: string required: - path - - thumbnail_path - title file: type: object diff --git a/docs/pipestat/tests/data/sample_output_schema.yaml b/docs/pipestat/tests/data/sample_output_schema.yaml index eb7bbd2d..c3a4ef79 100644 --- a/docs/pipestat/tests/data/sample_output_schema.yaml +++ b/docs/pipestat/tests/data/sample_output_schema.yaml @@ -43,7 +43,6 @@ $defs: type: string required: - path - - thumbnail_path - title file: type: object diff --git a/docs/pypiper/changelog.md b/docs/pypiper/changelog.md index 9817f135..dab30811 100644 --- a/docs/pypiper/changelog.md +++ b/docs/pypiper/changelog.md @@ -10,16 +10,14 @@ - Version now sourced from importlib.metadata (removed _version.py) - Converted all docstrings to Google-style formatting - Consolidated documentation into pepspec; removed local docs/ and mkdocs.yml -- Updated README badges and documentation links to pep.databio.org/pypiper +- Updated README badges and documentation links ### Added - `clean_add()` now accepts None values, enabling safe use with conditionally-assigned variables - Pipestat version displayed in pipeline version log output -- `pipestat_lenient` parameter on PipelineManager (defaults to True) ### Removed - Removed magic schema discovery (`default_pipestat_output_schema()`) -- Removed `pipestat_lenient` parameter from lower-level API (now on PipelineManager) ### Improved - Test quality: added missing assertions, removed permanently-skipped tests, added shared fixtures diff --git a/docs/pypiper/features.md b/docs/pypiper/features.md index 93b62635..7595bb91 100644 --- a/docs/pypiper/features.md +++ b/docs/pypiper/features.md @@ -34,4 +34,4 @@ Pypiper closes pipelines gracefully on interrupt or termination signals, convert ![](img/recovery.svg) **Dynamic recovery** -If a job is interrupted (with SIGINT or SIGTERM), either from a user or by a cluster resource manager, pypiper will set a `dynamic recovery` flag. The next time the run is started, it will automatically pick up where it left off. This makes pypiper pipelines `automatically pre-emption ready`, so they can be immediately deployed on servers where jobs may be pre-emptied. +If a job is interrupted (with SIGINT or SIGTERM), either from a user or by a cluster resource manager, pypiper will set a `dynamic recovery` flag. The next time the run is started, it will automatically pick up where it left off. This makes pypiper pipelines `automatically pre-emption ready`, so they can be immediately deployed on servers where jobs may be pre-empted. From e2ccc82762517e03615407de9931ef8c6ea311f8 Mon Sep 17 00:00:00 2001 From: nsheff Date: Thu, 5 Mar 2026 22:22:15 -0500 Subject: [PATCH 5/5] changelogs updates --- docs/looper/changelog.md | 10 ++++++---- docs/pipestat/changelog.md | 12 ++++++++++++ docs/pypiper/changelog.md | 4 ++++ 3 files changed, 22 insertions(+), 4 deletions(-) diff --git a/docs/looper/changelog.md b/docs/looper/changelog.md index 3e7cf438..b7fc44dc 100644 --- a/docs/looper/changelog.md +++ b/docs/looper/changelog.md @@ -2,12 +2,14 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) and [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) format. -## [Unreleased] +## [2.1.0] -- 2026-03-05 ### Added -- Added `inject_env_vars` pipeline interface property for injecting environment variables into submission scripts -- Added `pipestat_config_required` pipeline interface property to control pipestat handoff validation -- Added validation that pipestat-enabled interfaces (with `output_schema`) pass config to the pipeline via CLI (`{pipestat.*}`) or environment variable (`PIPESTAT_CONFIG` in `inject_env_vars`) +- `inject_env_vars` pipeline interface property for environment variables in submission scripts +- `pipestat_config_required` pipeline interface property for pipestat handoff validation + +### Changed +- Fixed import-time logging that overrode root logger configuration ## [2.0.3] -- 2025-09-23 ### Fixed diff --git a/docs/pipestat/changelog.md b/docs/pipestat/changelog.md index 077ba395..79417dd0 100644 --- a/docs/pipestat/changelog.md +++ b/docs/pipestat/changelog.md @@ -2,6 +2,18 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) and [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) format. +## [0.13.1] - 2026-03-05 +### Changed +- Fixed import-time logging that overrode root logger configuration + +## [0.13.0] - 2026-02-25 +### Changed +- Renamed `PipestatBoss` to `PipestatDualManager` +- Added classmethod constructors for each backend type +- Added `validate_results` and `additional_properties` options +- Migrated to yacman v1 API +- Modernized codebase: Google-style docstrings, type hints + ## [0.12.2] - 2025-09-25 ### Fixed - Fix setting with copy warning for pephub backend [#206](https://github.com/pepkit/pipestat/issues/206) diff --git a/docs/pypiper/changelog.md b/docs/pypiper/changelog.md index dab30811..81f5c88c 100644 --- a/docs/pypiper/changelog.md +++ b/docs/pypiper/changelog.md @@ -1,5 +1,9 @@ # Changelog +## [0.15.1] -- 2026-03-05 +### Changed +- Updated for logmuse 0.3.0 compatibility + ## [0.15.0] -- 2026-02-13 ### Changed