Skip to content

Add calibrated harder needle probe #5

Description

@Danny1218

Design a harder long-context or retrieval-style needle probe that still has a usable FP16 baseline.

Goal:

  • Probe whether quantization fragility appears in context-depth stress without relying on a ceiling baseline.

Acceptance criteria:

  • Add probe generation and validation code without mutating frozen probe sets.
  • Verify FP16 calibration before using it in headline comparisons.
  • Document scorer limitations and avoid claims from weak/floor/ceiling baselines.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions