Skip to content

Preserve histogram counts during entropy calibration - #2655

Open
sylvesterkaczmarek wants to merge 1 commit into
NVIDIA:mainfrom
sylvesterkaczmarek:fix/entropy-preserve-calibration-histogram
Open

sylvesterkaczmarek wants to merge 1 commit into
NVIDIA:mainfrom
sylvesterkaczmarek:fix/entropy-preserve-calibration-histogram

Conversation

@sylvesterkaczmarek

@sylvesterkaczmarek sylvesterkaczmarek commented Oct 4, 2026 •

Copy link
Copy Markdown

Fixes #2654.

What changed

Make the entropy calibration helper copy its input histogram before applying its existing first-bin adjustment. A NumPy slice is a view, so the previous implementation modified the collector's persistent counts. Subsequent percentile/MSE queries and continued collection could then use the altered distribution.

The regression uses the public HistogramCalibrator with both histogram backends. It checks unchanged counts and edges, repeated entropy queries, unchanged percentile thresholds, and equivalence with an untouched reference collector after another batch.

Validation

python -m pytest -q tests/unit/torch/quantization/test_calibrator.py   tests/unit/torch/quantization/test_tensor_quantizer_cpu.py   tests/unit/torch/quantization/test_print.py

79 passed, 1 existing skip on CPU. The focused regression on unchanged upstream gives 1 failure for NumPy histograms and 1 pass for PyTorch histograms. Repository pre-commit checks pass for the changed files.

No new dependencies or public API changes. CUDA execution was not run. This is independent of #2369: its zero-prefix changes affect collection, while this patch changes only the entropy calculation. The changelog is updated.

Summary by CodeRabbit

  • Bug Fixes
    • Fixed entropy-based calibration so repeated calculations preserve histogram counts and bin edges, keeping subsequent calibration results consistent.

Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 4, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

🧰 Additional context used
📚 Code guidelines (1)
.github/PULL_REQUEST_TEMPLATE.md — auto-discovered

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA/Model-Optimizer/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: e57558a9-6212-4a65-bb95-da04d1cec0be
📥 Commits

Reviewing files that changed from the base of the PR and between 6a10687 and 74c4858.

📒 Files selected for processing (3)
  • CHANGELOG.rst
  • modelopt/torch/quantization/calib/histogram.py
  • tests/unit/torch/quantization/test_calibrator.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.


📝 Walkthrough

Walkthrough

The entropy calibration helper now copies histogram data before modifying its first bin. Regression tests check that repeated calculations preserve collected histogram data and edges, percentile results, and later collection behavior.

Changes

Entropy histogram preservation

Layer / File(s) Summary
Copy histogram data and verify repeated calculations
modelopt/torch/quantization/calib/histogram.py, tests/unit/torch/quantization/test_calibrator.py, CHANGELOG.rst
_compute_amax_entropy now copies the collected histogram before changing its first bin. Tests cover both histogram backends and verify repeated calculations, percentile results, and subsequent collection. The changelog records the fix.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: kevalmorabia97

Merge Risk: ⚪ Minimal · up to 74c48

The histogram copy works for both backends; no actionable merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 2 files. (1 skipped: 1… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: preserving histogram counts during entropy calibration.
Linked Issues check ✅ Passed Issue #2654 requires a private histogram copy during entropy calibration and regression coverage for persistent counts, repeated queries, and continued collection. The PR changes `_compute_amax_entrop…
Out of Scope Changes check ✅ Passed The helper change and regression tests implement #2654. The changelog entry documents the same fix. The supplied change summary shows no unrelated changes.
Security Anti-Patterns ✅ Passed The reviewed diff changes only the histogram helper, its regression test, and the changelog. The production change copies the histogram with calib_hist.copy() before adjusting the first bin. The add…
Full details: Docstring Coverage

Explanation

Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 2 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Entropy calibration mutates the stored NumPy histogram

1 participant