[BREAKING] [FIX]: Return UNDETERMINED when an evaluator cannot observe the evidence it needs - #141

Merged

Nina Chikanov (nina-msft) merged 34 commits into

microsoft:mainfrom

mahdi-al-hakim:fix/observability-aware-evaluators

Aug 25, 2026

Commits on Aug 21, 2026

[FIX]: Return UNDETERMINED when an evaluator cannot observe the evidence it needs
Mahdi Alhakim (mahdi-al-hakim)
committed
9ac85b7
[FIX]: Evaluate the right operand of & when the left is undetermined
Mahdi Alhakim (mahdi-al-hakim)
committed
8bf0b62
[TEST]: Apply the _async naming convention to the tests added here
Mahdi Alhakim (mahdi-al-hakim)
committed
5c54e68
[FIX]: Keep operand evidence on an undetermined conjunction
Mahdi Alhakim (mahdi-al-hakim)
committed
a5d9a07
[FIX]: Filter the XPIA undetermined summary to undetermined rationales
Mahdi Alhakim (mahdi-al-hakim)
committed
c8287c0
[DOCS]: Correct the observability wording the declared level made stale
Mahdi Alhakim (mahdi-al-hakim)
committed
824b37f
[TEST]: Pin how a blind evaluator composes, and tidy the tests added here
Mahdi Alhakim (mahdi-al-hakim)
committed
8569274
[TEST]: Restore the metadata test that a new class had reparented
Mahdi Alhakim (mahdi-al-hakim)
committed
72b750e
[FIX]: Carry the operand rationale through an undetermined disjunction
Mahdi Alhakim (mahdi-al-hakim)
committed
4d8fb2d
[DOCS]: State the composition contract explicitly and fix the note rendering
Mahdi Alhakim (mahdi-al-hakim)
committed
d36cf73
[DOCS]: Tighten the wording carried over from the review round
Mahdi Alhakim (mahdi-al-hakim)
committed
9cca8da
[FIX]: Name the definitive turn in an unsafe probe summary
Mahdi Alhakim (mahdi-al-hakim)
committed
7baa132
[DOCS]: Say that the observability guarantee is per channel
Mahdi Alhakim (mahdi-al-hakim)
committed
b2ff1b4
[FIX]: Take the unsafe XPIA evidence only from detected results
Mahdi Alhakim (mahdi-al-hakim)
committed
a5023ec
[FEAT]: Carry an undetermined-operand signal through composition
Mahdi Alhakim (mahdi-al-hakim)
committed
72ebd3c
[FIX]: Build the undetermined summary from the propagated operand reasons
Mahdi Alhakim (mahdi-al-hakim)
committed
4b43052
[BREAKING] [FEAT]: Require observability_level on the public evaluation APIs
Mahdi Alhakim (mahdi-al-hakim)
committed
b86a67f
[FIX]: Report the real observability level on a truncated xdist result
Mahdi Alhakim (mahdi-al-hakim)
committed
82f7926
[FIX]: Contain an evaluator value that cannot be rendered
Mahdi Alhakim (mahdi-al-hakim)
committed
93a1312
[FIX]: Explain an undetermined verdict only from the results that caused it
Mahdi Alhakim (mahdi-al-hakim)
committed
df47f35
[DOCS]: Match the argument docs to the field order and cover `|` ordering
Mahdi Alhakim (mahdi-al-hakim)
committed
7e86498
[FIX]: Report the observability level in the JSON run report
Mahdi Alhakim (mahdi-al-hakim)
committed
bbaa923
[FIX]: Normalize evaluator collections before iterating them
Mahdi Alhakim (mahdi-al-hakim)
committed
830f838

Commits on Aug 22, 2026

Commits on Aug 24, 2026

Drop blank carried reasons in _merge_undetermined

Nina Chikanov (nina-msft)
and
Copilot
committed
38dd617
Share safe_float and harden confidence/rationale serialization

Nina Chikanov (nina-msft)
and
Copilot
committed
c93f611
Use pytest.approx in safe_float equality assertions

Nina Chikanov (nina-msft)
and
Copilot
committed
7643ac8

Commits on Aug 25, 2026