Skip to content

Wire calibration into the analyzer so confidence is earned (and --check can enforce) #48

Description

@hyperpolymath

crates/oikosbot-analysis/src/calibration.rs provides estimate_operation() returning min/typical/max ResourceRanges with Confidence::Calibrated for known operation kinds (HashLookup, Sort, FileIO, NetworkCall, Allocation, StringOp, MathCompute). Its module doc says it "replaces naive complexity * 0.1 J with pattern-based resource profiles".

It was never wired in. Analyzer::estimate_resources() is still exactly that naive form:

let energy = Energy::joules(complexity as f64 * 0.1);
let duration = Duration::milliseconds(complexity as f64 * 0.5);
let memory = Memory::kilobytes(complexity * 2);

and every result is hardcoded Confidence::Estimated.

Consequences

  1. Every published figure is a guess — energy, carbon, duration, memory, and therefore EcoScore, EconScore, ParetoScore and the health index.
  2. --check can never enforce. Only Measured/Calibrated inputs may block, so the enforcement path is inert. fix(check): refuse loudly instead of passing silently #47 makes that refusal loud rather than silent, but the underlying capability is still missing.

What to do

  • Map detected patterns (patterns.rs: nested-loops, busy-wait, string-concat-in-loop, clone-in-loop, unbuffered-io, large-allocation) onto OperationKinds.
  • Use estimate_operation() for recognised patterns; keep the naive path for unrecognised code and label it Estimated — confidence must be earned per finding, never blanket-assigned.
  • Propagate the real ResourceRange (min/typical/max) rather than collapsing to a point estimate; a range is what makes "Calibrated" meaningful.
  • Add tests proving (a) a recognised pattern yields Calibrated, (b) unrecognised code stays Estimated, (c) --check can now exit 1 on an undocumented, calibrated regression — a falsifier for the gate.

Caution

This changes every resource figure and every downstream score, including for the 17 estate consumers. It needs its own reviewed change with before/after numbers, not a drive-by. Do not relabel existing estimates as Calibrated to unblock --check — that would be overclaiming.

Blocks: real regulator mode.
Related: #42 (Pareto engine), #47 (loud refusal).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew capability or improvement to existing behaviour

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions