diff --git a/docs/reference/task-context-refinement-consumer-plan-v0.1.md b/docs/reference/task-context-refinement-consumer-plan-v0.1.md new file mode 100644 index 00000000..88548635 --- /dev/null +++ b/docs/reference/task-context-refinement-consumer-plan-v0.1.md @@ -0,0 +1,292 @@ +# Task Context External Refinement Consumer Proof v0.1 + +**Status:** Proof Complete / Schema Promotion HOLD +**Date:** 2026-08-18 + +## 0. Verdict + +The external-consumer proof passes at the tested boundary: + +> A caller that retains the original `TaskFrame + ContextRequirement[]` can consume the existing `TaskContext.gap_requirement_ids` / `refinement_requirement_ids`, acquire additional candidates from an external provider, and reassess Task Context sufficiency without any new GeoTask Core public object or field. + +Therefore: + +```text +Refinable critical-gap closure PASS +False acquisition 0 +Unresolved refinement preservation PASS +Direct reassessment parity PASS +Sunk acquisition cost preservation PASS +Core code delta 0 +Core public schema delta 0 +ContextGap / RefinementRequest HOLD +``` + +This does **not** prove that a bare persisted `TaskContext` ids-only artifact is sufficient for a later asynchronous worker that no longer has the original requirement objects. + +--- + +## 1. Problem + +GeoTask has real spatial and temporal evidence for **Sufficiency-Guided Refinement**, but method evidence alone does not prove that the current public Task Context objects are sufficient across a component boundary. + +The concrete question is: + +> Can a caller outside `geotask_core` consume the existing `gap_requirement_ids` / `refinement_requirement_ids`, acquire additional context through a provider, and reassess to a sufficient Task Context without a new public `ContextGap` or `RefinementRequest` object? + +--- + +## 2. Frozen Method + +The external consumer retains the original: + +```text +TaskFrame +ContextRequirement[] +selected ContextCandidate[] +``` + +It may inspect the returned `TaskContext`, but it may not reach into Core internals. + +For each `refinement_requirement_id`: + +```text +id + -> resolve original ContextRequirement + -> call external provider with TaskFrame + ContextRequirement + -> receive ContextCandidate[] + -> append newly acquired candidates + -> call assess_task_context(...) again +``` + +The consumer must not automatically acquire for a `gap_requirement_id` that is **not** also a refinement requirement. Missing context and refinable context remain distinct. + +--- + +## 3. Reference Scenario + +Reuse the existing fictional low-altitude mission semantics: + +```text +weather usable +airspace usable +obstacles applicable but 100 m > required 10 m +poi_labels optional / absent +``` + +Initial state: + +```text +status insufficient +gap_requirement_ids (obstacles, poi_labels) +refinement_requirement_ids (obstacles) +``` + +A fictional external obstacle provider returns a 5 m candidate for the same task scope. + +Final state: + +```text +status sufficient_with_gaps +critical gaps none +refinement_requirement_ids none +optional poi_labels gap remains +``` + +The proof concerns context closure only; it does not authorize or assess a real flight. + +--- + +## 4. Minimum Carrier + +The reference consumer lives outside Core at: + +```text +examples/task_context/refinement_consumer.py +``` + +It defines only: + +```text +RefinementProvider +RefinementCycle +run_refinement_cycle(...) +``` + +It does not add a provider registry, scheduler, Agent loop, Harness dependency, or second context-selection algorithm. + +The consumer resolves emitted requirement ids against the caller's original structured `ContextRequirement[]`; it never parses `what` / `reason` prose and never reads hidden Core state. + +--- + +## 5. Proof Cases + +### 5.1 Successful Critical-Gap Closure + +Provider returns a 5 m obstacle candidate. + +Observed/frozen expectations: + +```text +provider call count 1 +provider call requirement obstacles +false acquisition count 0 +critical obstacle gap closed yes +optional poi_labels gap preserved yes +final refinement ids none +final acquisition cost 6 credits +``` + +The original 100 m obstacle acquisition remains in the total cost. Refinement does not erase sunk context-preparation cost. + +The consumer's final `TaskContext` is asserted equal to a direct call to `assess_task_context()` over the identical final candidate set. The consumer therefore contains no second sufficiency algorithm. + +### 5.2 Too-Coarse Refinement Remains Unresolved + +Provider returns 50 m while the requirement needs <=10 m. + +Expected and tested: + +```text +final status insufficient +obstacles gap preserved +obstacles refinement signal preserved +``` + +The external consumer cannot turn acquisition activity into fake sufficiency. + +### 5.3 Missing Non-Refinement Critical Gap Does Not Trigger Provider + +Remove the airspace candidate while preserving the refinable obstacle gap. + +Expected and tested: + +```text +initial gaps airspace, obstacles, poi_labels +refinement ids obstacles +provider calls obstacles only +false airspace acquisition 0 +final status insufficient +``` + +The obstacle refinement may close, but the unrelated missing critical airspace gap remains authoritative. + +### 5.4 No Refinement Signal Means No Provider Call + +When a 5 m obstacle candidate is already present: + +```text +refinement ids none +provider calls 0 +final == initial yes +``` + +This prevents a generic "always acquire more context" behavior from masquerading as refinement intelligence. + +--- + +## 6. Measures + +### Primary + +**Refinable Critical Gap Closure Rate** + +The one declared refinable critical requirement in the success scenario closes after the provider returns a valid finer candidate. + +### Counter-metrics + +**False Acquisition Count** + +Provider calls for non-refinement gaps remain zero. + +**Unresolved Refinement Preservation** + +A still-too-coarse provider response preserves the gap and refinement signal. + +**Reassessment Parity** + +The consumer final result equals direct Core assessment over the same final candidate set. + +**Schema Delta** + +```text +new Core code 0 +new Core public objects 0 +new TaskContext fields 0 +``` + +**Provider Call Count** + +Exactly one call for exactly one refinable requirement in the success case. + +**Sunk Cost Preservation** + +Previously acquired coarse context remains in the declared total acquisition cost. + +--- + +## 7. Exact-Head Validation + +Implementation head `917053e614ff68d40b33269650169d26da8325b3` passed combined `main` CI run #226: + +```text +Python 3.10 pytest + RC attestation PASS +Python 3.11 pytest + RC attestation PASS +Python 3.12 pytest + RC attestation PASS +Python 3.13 pytest + RC attestation PASS +artifact roundtrip PASS +public boundary PASS +build / twine PASS +provider-neutral Adapter build PASS +OpenAI Adapter build + smoke PASS +public scan PASS +RC build / Reference Agent evidence PASS +RC evidence merge / readiness audit PASS +workflow completed / success +``` + +This document-close commit is intentionally revalidated separately so the final branch head remains bound to an exact CI result. + +--- + +## 8. Applicability Boundary + +This proof is intentionally narrower than a generic asynchronous job protocol. + +It proves only the boundary where the external consumer retains: + +```text +TaskFrame +ContextRequirement[] +TaskContext +``` + +through the refinement cycle. + +It does **not** prove that a bare persisted `TaskContext` artifact containing only requirement ids is sufficient for a later asynchronous worker that no longer has the original `ContextRequirement[]`. + +That future boundary should be tested only when a real Harness / AgentReality / Lowa consumer requires it. If the original requirement objects cannot be preserved or re-resolved there, that failure may justify a structured `ContextGap` / `RefinementRequest` contract. + +--- + +## 9. Ownership Boundary + +This reference consumer is Harness-neutral and lives outside Core. + +Actual DeepSeek Harness / AgentReality integration belongs in the AgentReality adapter/extension layer, not in GeoTask Core. + +GeoTask should expose context semantics and reassessment; it should not become an Agent job system, provider scheduler, or second Harness runtime. + +--- + +## 10. Promotion Decision + +At the tested synchronous/caller-retained boundary: + +> **Keep `ContextGap` / `RefinementRequest` schema promotion on HOLD.** + +The current public objects already support the required acquire -> reassess loop. + +A future new contract must earn promotion from a concrete interoperability failure, not from architectural aesthetics. + +The most plausible next trigger is an actual asynchronous Harness / AgentReality handoff in which the original `ContextRequirement[]` cannot remain available across the component boundary. diff --git a/examples/task_context/refinement_consumer.py b/examples/task_context/refinement_consumer.py new file mode 100644 index 00000000..8d309cbf --- /dev/null +++ b/examples/task_context/refinement_consumer.py @@ -0,0 +1,112 @@ +"""Harness-neutral example consumer for Task Context refinement signals. + +This module deliberately lives outside ``geotask_core``. It demonstrates how a +caller that already owns the TaskFrame, ContextRequirement set, and selected +ContextCandidate set can consume the existing TaskContext gap/refinement ids, +ask an external provider for more context, and reassess sufficiency. + +It is a reference integration pattern, not a provider registry, scheduler, +agent loop, or automatic refinement algorithm. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Protocol, Sequence + +from geotask_core.task_context import ( + ContextCandidate, + ContextRequirement, + TaskContext, + TaskFrame, + assess_task_context, +) + + +class RefinementProvider(Protocol): + """Minimal external provider surface required by this reference consumer.""" + + def acquire_refinement( + self, + task: TaskFrame, + requirement: ContextRequirement, + ) -> Sequence[ContextCandidate]: + """Return additional candidates for one explicit refinement requirement.""" + + +@dataclass(frozen=True) +class RefinementCycle: + """Observable result of one external acquire -> reassess cycle.""" + + initial_context: TaskContext + final_context: TaskContext + provider_call_requirement_ids: tuple[str, ...] + skipped_non_refinement_gap_ids: tuple[str, ...] + acquired_candidate_ids: tuple[str, ...] + + @property + def provider_call_count(self) -> int: + return len(self.provider_call_requirement_ids) + + @property + def closed_refinement_requirement_ids(self) -> tuple[str, ...]: + remaining = set(self.final_context.refinement_requirement_ids) + return tuple( + requirement_id + for requirement_id in self.initial_context.refinement_requirement_ids + if requirement_id not in remaining + ) + + +def run_refinement_cycle( + task: TaskFrame, + requirements: Sequence[ContextRequirement], + selected_candidates: Sequence[ContextCandidate], + provider: RefinementProvider, +) -> RefinementCycle: + """Run exactly one caller-owned refinement cycle. + + The consumer never parses requirement prose and never invents provider + parameters. It resolves ids emitted by Core back to the caller's original + structured ContextRequirement objects, asks the external provider only for + requirements explicitly marked refinable, appends returned candidates, and + calls ``assess_task_context`` again. + + Ordinary gaps that are not also refinement requirements are intentionally + skipped. Missing evidence and refinable resolution remain distinct. + """ + + initial = assess_task_context(task, requirements, selected_candidates) + requirements_by_id = {item.requirement_id: item for item in requirements} + if len(requirements_by_id) != len(requirements): + raise ValueError("requirements must not contain duplicate ids") + + refinement_ids = tuple(initial.refinement_requirement_ids) + skipped_gap_ids = tuple( + requirement_id + for requirement_id in initial.gap_requirement_ids + if requirement_id not in set(refinement_ids) + ) + + acquired: list[ContextCandidate] = [] + provider_calls: list[str] = [] + for requirement_id in refinement_ids: + requirement = requirements_by_id.get(requirement_id) + if requirement is None: + raise ValueError( + "TaskContext refinement id is not present in the caller's " + f"ContextRequirement set: {requirement_id}" + ) + provider_calls.append(requirement_id) + acquired.extend(provider.acquire_refinement(task, requirement)) + + final_candidates = tuple(selected_candidates) + tuple(acquired) + final = assess_task_context(task, requirements, final_candidates) + + return RefinementCycle( + initial_context=initial, + final_context=final, + provider_call_requirement_ids=tuple(provider_calls), + skipped_non_refinement_gap_ids=skipped_gap_ids, + acquired_candidate_ids=tuple(item.candidate_id for item in acquired), + ) diff --git a/tests/test_task_context_refinement_consumer.py b/tests/test_task_context_refinement_consumer.py new file mode 100644 index 00000000..f6a18400 --- /dev/null +++ b/tests/test_task_context_refinement_consumer.py @@ -0,0 +1,260 @@ +from __future__ import annotations + +from dataclasses import dataclass, field +from pathlib import Path +import sys + + +ROOT = Path(__file__).resolve().parents[1] +_ROOT = str(ROOT) +sys.path.insert(0, _ROOT) +try: + from examples.task_context.refinement_consumer import run_refinement_cycle +finally: + sys.path.remove(_ROOT) + +from geotask_core.task_context import ( + ContextCandidate, + ContextRequirement, + TaskFrame, + assess_task_context, +) + + +def _task() -> TaskFrame: + return TaskFrame( + task_id="delivery-a-b-1500", + goal="Prepare context for a fictional low-altitude delivery mission", + subject_refs=("uav-logistics-small",), + spatial_scope="corridor-a-b", + temporal_scope="2026-08-19T15:00/16:00", + outputs=("route_risk_input",), + context_budget=12.0, + context_budget_unit="credits", + ) + + +def _requirements() -> tuple[ContextRequirement, ...]: + return ( + ContextRequirement( + requirement_id="weather", + what="wind and precipitation for the mission corridor", + reason="weather can change route feasibility", + spatial_scope="corridor-a-b", + temporal_scope="2026-08-19T15:00/16:00", + max_spatial_resolution=1000.0, + spatial_resolution_unit="meter", + max_temporal_resolution_seconds=1800.0, + ), + ContextRequirement( + requirement_id="airspace", + what="applicable temporary airspace restrictions", + reason="the route must avoid restrictions applicable to this corridor and window", + spatial_scope="corridor-a-b", + temporal_scope="2026-08-19T15:00/16:00", + ), + ContextRequirement( + requirement_id="obstacles", + what="local obstacle context near the candidate corridor", + reason="local clearance checking requires finer spatial detail", + spatial_scope="corridor-a-b", + max_spatial_resolution=10.0, + spatial_resolution_unit="meter", + ), + ContextRequirement( + requirement_id="poi_labels", + what="nearby POI labels", + reason="useful for human explanation but not required for route-risk input", + critical=False, + spatial_scope="corridor-a-b", + ), + ) + + +def _initial_candidates(*, include_airspace: bool = True) -> tuple[ContextCandidate, ...]: + candidates = [ + ContextCandidate( + candidate_id="weather-forecast-500m", + source="fictional-weather-provider", + requirement_ids=("weather",), + spatial_scope="corridor-a-b", + temporal_scope="2026-08-19T15:00/16:00", + spatial_resolution=500.0, + spatial_resolution_unit="meter", + temporal_resolution_seconds=900.0, + acquisition_cost=2.0, + cost_unit="credits", + ), + ContextCandidate( + candidate_id="regional-obstacle-grid-100m", + source="fictional-map-provider", + requirement_ids=("obstacles",), + spatial_scope="corridor-a-b", + spatial_resolution=100.0, + spatial_resolution_unit="meter", + acquisition_cost=1.0, + cost_unit="credits", + ), + ] + if include_airspace: + candidates.append( + ContextCandidate( + candidate_id="airspace-notice", + source="fictional-airspace-provider", + requirement_ids=("airspace",), + spatial_scope="corridor-a-b", + temporal_scope="2026-08-19T15:00/16:00", + acquisition_cost=1.0, + cost_unit="credits", + ) + ) + return tuple(candidates) + + +@dataclass +class ObstacleProvider: + resolution_meters: float + calls: list[str] = field(default_factory=list) + + def acquire_refinement( + self, + task: TaskFrame, + requirement: ContextRequirement, + ) -> tuple[ContextCandidate, ...]: + self.calls.append(requirement.requirement_id) + assert task.task_id == "delivery-a-b-1500" + assert requirement.requirement_id == "obstacles" + assert requirement.spatial_scope == "corridor-a-b" + assert requirement.max_spatial_resolution == 10.0 + assert requirement.spatial_resolution_unit == "meter" + return ( + ContextCandidate( + candidate_id=f"local-obstacle-grid-{self.resolution_meters:g}m", + source="fictional-obstacle-refinement-provider", + requirement_ids=(requirement.requirement_id,), + spatial_scope=requirement.spatial_scope, + spatial_resolution=self.resolution_meters, + spatial_resolution_unit=requirement.spatial_resolution_unit, + acquisition_cost=2.0, + cost_unit="credits", + ), + ) + + +class NeverCallProvider: + def acquire_refinement(self, task, requirement): # pragma: no cover - failure path + raise AssertionError("provider must not be called when no refinement is requested") + + +def test_external_consumer_closes_refinable_critical_gap_without_new_core_schema() -> None: + task = _task() + requirements = _requirements() + initial_candidates = _initial_candidates() + provider = ObstacleProvider(resolution_meters=5.0) + + cycle = run_refinement_cycle(task, requirements, initial_candidates, provider) + + assert cycle.initial_context.status == "insufficient" + assert cycle.initial_context.gap_requirement_ids == ("obstacles", "poi_labels") + assert cycle.initial_context.refinement_requirement_ids == ("obstacles",) + + assert cycle.provider_call_requirement_ids == ("obstacles",) + assert cycle.provider_call_count == 1 + assert provider.calls == ["obstacles"] + assert cycle.skipped_non_refinement_gap_ids == ("poi_labels",) + assert cycle.acquired_candidate_ids == ("local-obstacle-grid-5m",) + + assert cycle.final_context.status == "sufficient_with_gaps" + assert cycle.final_context.gap_requirement_ids == ("poi_labels",) + assert cycle.final_context.refinement_requirement_ids == () + assert cycle.closed_refinement_requirement_ids == ("obstacles",) + assert cycle.final_context.total_acquisition_cost == 6.0 + assert not cycle.final_context.budget_exceeded + + direct = assess_task_context( + task, + requirements, + initial_candidates + + ( + ContextCandidate( + candidate_id="local-obstacle-grid-5m", + source="fictional-obstacle-refinement-provider", + requirement_ids=("obstacles",), + spatial_scope="corridor-a-b", + spatial_resolution=5.0, + spatial_resolution_unit="meter", + acquisition_cost=2.0, + cost_unit="credits", + ), + ), + ) + assert cycle.final_context == direct + + +def test_provider_that_remains_too_coarse_cannot_fake_gap_closure() -> None: + provider = ObstacleProvider(resolution_meters=50.0) + + cycle = run_refinement_cycle( + _task(), + _requirements(), + _initial_candidates(), + provider, + ) + + assert provider.calls == ["obstacles"] + assert cycle.final_context.status == "insufficient" + assert cycle.final_context.gap_requirement_ids == ("obstacles", "poi_labels") + assert cycle.final_context.refinement_requirement_ids == ("obstacles",) + assert cycle.closed_refinement_requirement_ids == () + + +def test_missing_non_refinement_critical_gap_does_not_trigger_false_acquisition() -> None: + provider = ObstacleProvider(resolution_meters=5.0) + + cycle = run_refinement_cycle( + _task(), + _requirements(), + _initial_candidates(include_airspace=False), + provider, + ) + + assert cycle.initial_context.gap_requirement_ids == ( + "airspace", + "obstacles", + "poi_labels", + ) + assert cycle.initial_context.refinement_requirement_ids == ("obstacles",) + assert cycle.provider_call_requirement_ids == ("obstacles",) + assert cycle.skipped_non_refinement_gap_ids == ("airspace", "poi_labels") + assert provider.calls == ["obstacles"] + + # Obstacle refinement succeeds, but the unrelated missing critical airspace + # evidence remains a real gap and keeps the context insufficient. + assert cycle.final_context.status == "insufficient" + assert cycle.final_context.gap_requirement_ids == ("airspace", "poi_labels") + assert cycle.final_context.refinement_requirement_ids == () + + +def test_no_refinement_signal_means_no_provider_call() -> None: + task = _task() + requirements = _requirements() + candidates = _initial_candidates() + ( + ContextCandidate( + candidate_id="local-obstacle-grid-5m", + source="fictional-obstacle-refinement-provider", + requirement_ids=("obstacles",), + spatial_scope="corridor-a-b", + spatial_resolution=5.0, + spatial_resolution_unit="meter", + acquisition_cost=2.0, + cost_unit="credits", + ), + ) + + cycle = run_refinement_cycle(task, requirements, candidates, NeverCallProvider()) + + assert cycle.initial_context.status == "sufficient_with_gaps" + assert cycle.initial_context.refinement_requirement_ids == () + assert cycle.provider_call_count == 0 + assert cycle.acquired_candidate_ids == () + assert cycle.final_context == cycle.initial_context