Skip to content

Fix degenerate quantile normalization - #30

Draft
milkieTigera wants to merge 1 commit into
PKU-Alignment:mainfrom
milkieTigera:codex/qzf-normalization-strategy
Draft

Fix degenerate quantile normalization#30
milkieTigera wants to merge 1 commit into
PKU-Alignment:mainfrom
milkieTigera:codex/qzf-normalization-strategy

Conversation

@milkieTigera

Copy link
Copy Markdown

Summary

  • Fall back to z-score normalization per dimension when q99 - q01 <= 1e-6.
  • Apply the same fallback during unnormalization so the transform remains symmetric.
  • Preserve action dimensions that do not have normalization statistics.

Motivation

Constant or nearly constant dimensions can have a degenerate quantile range. Applying quantile scaling to those dimensions amplifies tiny numerical differences into very large normalized values. The existing mean and standard deviation statistics already provide a stable fallback for those dimensions.

Compatibility

The change is limited to degenerate dimensions when quantile normalization is enabled. Valid quantile ranges continue to use the existing formula, and the default z-score path (use_quantiles=False) is unchanged. No model, data loader, or policy configuration is modified.

Validation

  • Targeted local regression tests: 5 passed.
  • Red/green check: the original implementation failed the two fallback-specific cases; the patched implementation passes all five cases.
  • Python compilation check passed for the modified module.
  • git diff --check passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant