Skip to content

Add OpenEntropy entropy source and ECDF amplification#4

Open
ereid7 wants to merge 2 commits intoalchemystack:mainfrom
ereid7:feat/openentropy-source
Open

Add OpenEntropy entropy source and ECDF amplification#4
ereid7 wants to merge 2 commits intoalchemystack:mainfrom
ereid7:feat/openentropy-source

Conversation

@ereid7
Copy link
Contributor

@ereid7 ereid7 commented Mar 2, 2026

Summary

  • add an OpenEntropy entropy source implementation and register it via qr_sampler.entropy_sources
  • add an ECDF amplifier with calibration, config support, and processor wiring for calibrated amplification
  • add deployment/docs updates for OpenEntropy usage, configuration, and source catalog references
  • add comprehensive tests for ECDF behavior, OpenEntropy source behavior, and OpenEntropy config fields

Why ECDF Amplifier

  • z-score amplification assumes byte means follow the expected distribution for ideal uniform bytes; real hardware/QRNG sources can violate that assumption and introduce bias in u
  • ECDF is non-parametric: it calibrates on observed source behavior and maps ranks to uniform values, which makes token selection robust to source-specific skew
  • this gives a better research baseline: if source distributions drift, the sampler still targets uniform u without relying on fragile distributional assumptions

Configurability

  • yes, it is configurable
  • select amplifier via QR_SIGNAL_AMPLIFIER_TYPE (ecdf or zscore_mean)
  • tune calibration size via QR_ECDF_CALIBRATION_SAMPLES (default 2000, min 100)
  • per-request override supports qr_signal_amplifier_type; calibration sample count is configured at process/env config level

OpenEntropy Configuration Note

  • set QR_ENTROPY_SOURCE_TYPE=openentropy to enable the source
  • choose conditioning with QR_OE_CONDITIONING (raw, vonneumann, sha256)
  • optionally pin specific sources with QR_OE_SOURCES (for example: counter_beat)
  • for full setup and source catalog details, see:

Why Not gRPC for OpenEntropy

  • OpenEntropy is already a local in-process API; adding gRPC would add transport overhead and another failure surface without improving local functionality
  • source and conditioning controls are first-class in config (QR_OE_SOURCES, QR_OE_CONDITIONING); routing through a gRPC adapter would add extra config/plumbing complexity
  • gRPC remains appropriate for remote entropy providers (quantum_grpc), while native integration is simpler and lower-friction for local OpenEntropy research workflows

Validation

  • python3 -m ruff check src/ tests/
  • python3 -m pytest tests/ -q
  • python3 -m build --sdist --wheel

@alchemystack
Copy link
Owner

Hi, the PR failed some of the CI checks. I've put the error logs in a txt file for you to troubleshoot, CC should be able to handle it.
openentropy PR error logs.txt
Also plz let me know if you're able to run the checks, or if I need to se up a "develop" branch or something so people can do these checks themselves, before opening PRs for main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants