Dynamic DEM Initial Infrastructure

[tlshannon]

Description

Dynamic DEM construction for streaming decoders

• css_code_matrices / css_noise_params (code_matrices.h) — new types that encode a CSS code's parity-check and logical-operator matrices plus a depolarizing (optionally phenomenological) noise model as plain sparse-binary structs, decoupled from the code base class.

• dem_from_css_matrices() (dem_construction.h) — builds a T-round detector_error_model directly from CSS matrices and noise, no stabilizer circuit or Stim round-trip required. A code-object overload (dem_construction_code.h) wraps the matrix version for callers that already have a code instance.

• extended_dem (extended_dem.h) — a sparse, partitioned DEM chunk carrying four row groups: interior, observables, in_syndrome, and out_syndrome. Supports an explicit stitching algebra:

  • extended_dem_from_css_matrices() — build a one-round chunk.
  • dem_stitch(a, b) / dem_stitch_all(chunks) — left-fold chunk composition, contracting outgoing/incoming seam rows.
  • dem_merge_duplicate_columns() — collapse fault columns with identical support using either exact XOR-probability or linear-sum combination.
  • dem_close() / dem_close_all() — convert a stitched chunk to a flat detector_error_model in O(T) time.
  • dem_chunks_spec / dem_chunk_spec — declarative init/bulk/final phase specs (YAML-serialisable).
  • Streaming helpers: dem_chunks_to_pcm(), dem_chunks_to_o_sparse(), dem_chunks_to_d_sparse(), dem_chunks_to_detector_round() — produce the exact inputs decoder::set_O_sparse() / set_D_sparse() expect, so no decoder plugin needs to know about extended_dem internals.

• decoder_config additions (realtime/decoding_config.h) — the flat-form fields (H_sparse, O_sparse, D_sparse, block_size, syndrome_size) are now complemented by an optional dem_chunks + num_rounds pair. expand_dem_chunks() rewrites a chunk-form config into the equivalent flat form at decoder construction time, so the rest of the realtime pipeline is unchanged.

---

New YAML format

Decoders may now be configured with a compact phase description instead of pre-expanded flat matrices. The round count becomes a runtime field, so the same config serves experiments of any length.

# decoder_config (chunk form)
id: 0
name: single_error_lut
num_rounds: 5          # total rounds: 1 init + 3 bulk + 1 final

dem_chunks:
  init:
    num_faults: 9
    # no H_in_sparse — init has no incoming seam
    H_mid_sparse: [ 0, 1, 5, -1, 1, 2, 6, -1, 2, 3, 7, -1, 3, 4, 8, -1 ]
    H_out_sparse: [ 5, -1, 6, -1, 7, -1, 8, -1 ]
    O_sparse:     [ 0, -1 ]
    error_rates:  [ 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02 ]

  bulk:
    num_faults: 9
    H_in_sparse:  [ 0, 1, 5, -1, 1, 2, 6, -1, 2, 3, 7, -1, 3, 4, 8, -1 ]
    H_out_sparse: [ 5, -1, 6, -1, 7, -1, 8, -1 ]
    O_sparse:     [ 0, -1 ]
    error_rates:  [ 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02 ]

  final:
    num_faults: 5
    H_in_sparse: [ 0, 1, -1, 1, 2, -1, 2, 3, -1, 3, 4, -1 ]
    # no H_out_sparse — final has no outgoing seam
    O_sparse:    [ 0, -1 ]
    error_rates: [ 0.02, 0.02, 0.02, 0.02, 0.02 ]

expand_dem_chunks() is called during finalize_decoders() and rewrites the chunk form into the flat fields in-place; all downstream decoder construction sees the familiar flat representation. A config that already carries a nonempty H_sparse is left untouched, so an expanded config round-trips through YAML without re-expansion.

---

Future PR: decoder plugin updates

This PR does not modify any decoder plugin. A follow-up PR will wire dem_chunks awareness directly into individual decoders where it adds value.

Lines of code

This PR is >1000 lines due to the creation of new functionality. Tests are almost half the line count.

Runtime / performance impact

N/A

Self-review checklist

Please confirm each item before requesting review. Check [x] or strike
through and explain.

Before requesting review

• I reviewed my own full diff in GitHub or my editor.
• PR is in Draft if it is not yet ready for review.
• Temporary / debugging changes have been removed.
• Local test logs reviewed; no unexplained warnings or errors.
• CI logs reviewed; no unexplained warnings or errors.
• Full CI has been run.

Scope and size

• PR is under ~1000 lines, or an exception is justified in the description. (see description)
• Refactoring-only changes are isolated in their own PR(s).
• No existing tests were disabled or modified just to make this PR pass
  (if so, an issue has been raised).

Tests

• New functionality has new tests.
• Tests fail if the new functionality is broken (including crashes), not
  just when it is missing.
• Negative tests added where exceptions are expected.
• Truth data added where simple EXPECT_* / assert checks are
  insufficient for algorithmic correctness.
• CI runtime impact considered; team notified if significant.

Documentation

• Public-facing APIs have Doxygen docs.
• User-visible behavior changes have public docs, or a follow-up is
  tracked.

Code style

• Naming follows the existing convention (snake_case vs camelCase) for
  the area being modified.

Dependencies

• No new third-party dependencies, or the team has been notified and
  OSRB tickets filed.