Skip to content

Add Cosmos segmented checkpointing support - #2937

Merged
bghira merged 2 commits into
mainfrom
agent/segmented-checkpointing-cosmos
Aug 3, 2026
Merged

Add Cosmos segmented checkpointing support#2937
bghira merged 2 commits into
mainfrom
agent/segmented-checkpointing-cosmos

Conversation

@bghira

@bghira bghira commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Summary

Splits the Cosmos segmented checkpointing support model integration out of #2925.

  • applies the model-family checkpointing hooks and support flags
  • adds this family to the relevant safety-check allow-lists
  • keeps the shared runtime/docs in the base PR so this diff stays model-specific

Stack

Base branch: agent/segmented-checkpointing-chroma

Validation

  • .venv/bin/python -m unittest tests.test_segmented_checkpointing_model_support -v
  • commit hooks: Black, isort, flake8, whitespace checks

@bghira
bghira force-pushed the agent/segmented-checkpointing-cosmos branch from ecdfa21 to 7507487 Compare August 3, 2026 08:50
@bghira
bghira requested a review from Copilot August 3, 2026 08:52
@bghira
bghira marked this pull request as ready for review August 3, 2026 08:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Cosmos (T2I) model-family integration for the segmented gradient checkpointing controls introduced in the stacked base PR, ensuring Cosmos is recognized by safety checks and exposes the expected model hooks.

Changes:

  • Adds a segmented-checkpointing support test for CosmosTransformer3DModel.
  • Allows cosmos in safety-check allow-lists for --gradient_checkpointing_interval and --gradient_checkpointing_segment_stride.
  • Wires Cosmos’ transformer forward pass to use should_checkpoint_block(...) gated by interval/segment-stride settings.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
tests/test_segmented_checkpointing_model_support.py Adds a capability/controls presence test for Cosmos segmented checkpointing support.
simpletuner/helpers/training/default_settings/safety_check.py Extends safety-check allow-lists to permit interval + segment stride for the cosmos model family.
simpletuner/helpers/models/cosmos/transformer.py Adds interval/stride setters and uses the shared should_checkpoint_block helper to gate checkpointing per block.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +901 to +906
if torch.is_grad_enabled() and should_checkpoint_block(
bid,
self.gradient_checkpointing,
self.gradient_checkpointing_interval,
self.gradient_checkpointing_segment_stride,
):
@bghira
bghira force-pushed the agent/segmented-checkpointing-cosmos branch from 7507487 to 3067731 Compare August 3, 2026 09:02
@bghira
bghira force-pushed the agent/segmented-checkpointing-cosmos branch from d32e7c4 to bcee787 Compare August 3, 2026 10:45
@bghira
bghira force-pushed the agent/segmented-checkpointing-cosmos branch from bcee787 to 9e0f5f3 Compare August 3, 2026 15:30
@bghira
bghira force-pushed the agent/segmented-checkpointing-cosmos branch from 9e0f5f3 to 621fca5 Compare August 3, 2026 15:32
Base automatically changed from agent/segmented-checkpointing-chroma to main August 3, 2026 15:33
@bghira
bghira force-pushed the agent/segmented-checkpointing-cosmos branch from 621fca5 to c888de5 Compare August 3, 2026 15:33
@bghira
bghira force-pushed the agent/segmented-checkpointing-cosmos branch from c888de5 to 8e9047f Compare August 3, 2026 15:35
@bghira
bghira merged commit 3504e55 into main Aug 3, 2026
@bghira
bghira deleted the agent/segmented-checkpointing-cosmos branch August 3, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants