GitHub HTTP response status is lost on truncated body reads, leaving request budget blind #109

Activity

1. andy-zen-dev commented on Oct 30, 2026

   

   andy-zen-dev

   on Oct 30, 2026 – with ChatGPT Codex Connector

   Author

   More actions

   #109 implementation handoff — retain known HTTP status across failed body reads (2026-10-30)

   Pinned evidence: release/0.6.0 at 6d935ee111e2efab7a5f7d7c8f8d050af8d35567, src/devostasis/adapters/github.py blob 7a998b94b12547ef543a6320360ed94f088197a7; CONTRIBUTING.md was consulted. Authenticated account andy-zen-dev, permission pull=true,push=false,triage=false,maintain=false,admin=false. No source mutation was possible.

   Prepared narrow prerequisite fix (NOT integrated): Keep the existing NetworkFailure exception class and add optional status_code/response_headers metadata. In UrllibTransport.get(), capture the actual response status and normalized headers before calling response.read(); wrap only failures occurring after that point in NetworkFailure(..., status_code=status, response_headers=headers). The failed body is not returned, parsed, or cached. Errors before headers remain status-less. Normal 200/JSON handling and the 304 cache exemption remain unchanged. Existing callers constructing NetworkFailure(message) remain compatible.

   Key source edits:

   @@ -86,6 +86,10 @@
    @dataclass
    class NetworkFailure(Exception):
        message: str
   +    # HTTP status and headers are evidence, even when the body never arrives.
   +    # None means dispatch failed before response headers could be observed.
   +    status_code: int | None = None
   +    response_headers: dict[str, str] | None = None
    
   @@ -175,8 +179,18 @@
            try:
                with self._open(request) as response:
   -                raw = response.read()
   -                headers = {k.lower(): v for k, v in response.headers.items()}
   +                status = response.status
   +                response_headers = {k.lower(): v for k, v in response.headers.items()}
   +                try:
   +                    raw = response.read()
   +                except (urllib.error.URLError, http.client.HTTPException, TimeoutError, OSError) as exc:
   +                    raise NetworkFailure(
   +                        f"network failure for {path} after HTTP {status}: {type(exc).__name__}: {exc}",
   +                        status_code=status,
   +                        response_headers=response_headers,
   +                    ) from exc
                    try:
   @@
   -                    raise ApiFailure(response.status, ERROR, "MALFORMED_RESPONSE", f"{path} answered {response.status} with a body that is not readable JSON: {type(exc).__name__}") from exc
   -                return response.status, headers, body
   +                    raise ApiFailure(status, ERROR, "MALFORMED_RESPONSE", f"{path} answered {status} with a body that is not readable JSON: {type(exc).__name__}") from exc
   +                return status, response_headers, body

   Prepared standard unified patch also includes the new tests/test_github_transport_provenance.py (5 parametrized/standalone test cases): observed 200 with IncompleteRead or TimeoutError retains the known status and headers, cannot poison a preexisting ETag cache, a pre-header URLError remains unknown-status, a complete 200 still bills once, and a valid cached 304 still replays without billing. Patch SHA-256: 57800cd6fbea02e9776507f785e95840c9be629de82ec2905746183569965734.

   Verification performed: git apply --check, patch application, and test-module syntax: PASS on a synthetic reconstruction of the exact GitHub source hunk contexts, not a full checkout. Additional 5/5 isolated smoke scenarios PASS against source-extracted patched UrllibTransport.get for truncated read, post-header timeout, pre-header failure, valid 200 and malformed 200. Full real-package pytest, conformance, persisted-bundle verification, installed wheel, exact-head GitHub CI: NOT_RUN. No PR or merge was created.

   for:researcher — genuine pending contract decision: A response with observed HTTP 200 headers but an incomplete body might have consumed provider quota. Should the billed_requests metric charge an observed non-304 status before body completion, or should such calls remain charge unknown and be bounded separately by a physical-attempt cap? Decide and version the definition, required run-metadata/provenance, and compatibility impact before modifying GitHubClient.billed_count or _claim_request(). This prerequisite patch intentionally does not fix #109's request-budget escape until the accepted contract specifies charging/attempt limits; it only preserves the evidence needed to implement the correct choice. #108's completed-body accounting is a distinct accepted path.

   Suggested next implementation unit after decision: handle NetworkFailure.status_code in the client at the correct exactly-once charging boundary, add the real UrllibTransport + GitHubClient regression at budget=1 asserting second-dispatch behavior and run_meta, then run the complete repository gates before release admission.
2. andy-zen-dev commented on Oct 30, 2026

   

   andy-zen-dev

   on Oct 30, 2026 – with ChatGPT Codex Connector

   Author

   More actions

   #109 follow-up: combined #105–#109 implementation-stack compatibility, with an explicit remaining budget escape (2026-10-30)

   Scope and authority: Independent uncommitted integration check. Re-read release/0.6.0 src/devostasis/adapters/github.py blob 7a998b94b12547ef543a6320360ed94f088197a7, src/devostasis/adapters/cache.py blob 0d4de1e063eb6b0527b5b7b44c7fa0a7e830d085, CONTRIBUTING.md; observed GitHub account andy-zen-dev, pull=true,push=false,triage=false,maintain=false,admin=false. This is not a release acceptance, new research judgement, merge, or change to upstream.

   Prepared patches on the same pinned source:

   • Unexpected GitHub HTTP 3xx responses can be admitted as complete inventory #105–GitHub HTTP 202 Accepted can be cached as a completed empty inventory #107 status-completion gate SHA-256 9263eb52f70ec2d3575afed01feb6314f53b223ccb4597beb46ce095ba079569 (handoff in GitHub HTTP 202 Accepted can be cached as a completed empty inventory #107).
   • GitHub transport: malformed completed responses bypass billed-request budget #108 corrected completed-response budget patch SHA-256 77a2f2904621abfb09f7a35c4e0c5fac6b93b3ef4038b8c75172fba160d27aa8 (handoff in GitHub transport: malformed completed responses bypass billed-request budget #108).
   • GitHub HTTP response status is lost on truncated body reads, leaving request budget blind #109 status-provenance-only patch SHA-256 57800cd6fbea02e9776507f785e95840c9be629de82ec2905746183569965734 (previous comment here).

   New exact-scope verification:

   1. Reconstructed original hunk contexts at the real source line offsets for all three patches; detected no conflicting original lines across seven touched paths. git apply --check, sequential application, and git diff --check PASS in all six patch orders. Both added test modules compile. This does not prove applicability in a complete checkout, only the reconstructed contexts.
   2. 21/21 isolated control-flow interactions PASS using the source-extracted GitHubClient.get algorithm plus scripted transport: known-status malformed JSON 200/202/206 is charged exactly once and never cached; returned unexpected 202/206/302/307 is rejected and charged; complete 200 and valid cached 304 retain their distinct accounting; status-aware truncated-body NetworkFailure propagates without accepting/caching its body.
   3. Critical non-GREEN acceptance condition remains: under these prepared patches, a response with HTTP 200 headers followed by IncompleteRead now preserves status metadata, but the client still does not charge billed_count or cap physical attempts. With budget=1, an isolated source-shaped test sees request_count=1,billed_count=0 after that failure and permits a second physical dispatch (request_count=2,billed_count=1 after a subsequent 200). Repeated incomplete reads remain unbounded by the billed-response budget. This was deliberate: the for:researcher decision requested earlier in GitHub HTTP response status is lost on truncated body reads, leaving request budget blind #109 is still unanswered. Do not close GitHub HTTP response status is lost on truncated body reads, leaving request budget blind #109 or characterize these tests as a fixed budget escape.

   Boundary: all synthetic reconstructions/extracted-algorithm checks, not installed Devostasis. Full pytest, conformance, sample/historical bundles, wheel and exact-head hosted CI are NOT_RUN. No branch/commit/PR/merge. The complete repro scripts are local work artifacts, not GitHub-persisted files.

   Next actionable unit once research decides charging semantics: apply the three non-overlapping patches to a real fix branch; implement and test #109's budget accounting/physical-attempt limit according to the accepted versioned decision, including real UrllibTransport + GitHubClient and run_meta regression at budget=1, then run the full repository gates and obtain independent exact-head review. #95 is separately approved but not merged; #81 remains a release dependency.
3. andy-zen-dev commented on Oct 30, 2026

   

   andy-zen-dev

   on Oct 30, 2026 – with ChatGPT Codex Connector

   Last edited by andy-zen-dev

   Author

   More actions

   Redundancy correction — integration replay of prepared HTTP patch stack (2026-10-30)

   This run independently reran the three prepared patch packages on an exact-hunk synthetic reconstruction at release/0.6.0 source blob 7a998b94b12547ef543a6320360ed94f088197a7. All six patch application orders passed git apply --check, application, git diff --check, and syntax compilation of the added pytest modules. Replay script SHA-256: 1432edbf36dee91642d28eddcf2526b13743004c7019c255d07afd4046cfb13e. Re-executed separate isolated status-gate and #105–#108 interaction probes (109/109 and 16/16).

   Correction: the immediately preceding #109 comment #6523660138 already reports the same six-order compatibility test plus a stronger 21/21 interaction matrix and the unresolved physical-budget escape. This replay does not establish a new code fix, research judgement, release gate, or material improvement beyond that handoff. Treat the preceding comment as the authoritative integration evidence and avoid duplicate implementation work.

   The current #109 patch preserves an observed HTTP status on truncated body reads but does not enforce a physical-attempt cap or account for its possibly billed status. Research decision is still pending. Full checkout, installed-package pytest, historical bundles, wheel and exact-head CI: NOT_RUN. GitHub identity andy-zen-dev has pull=true, push=false, triage=false, maintain=false, admin=false; no branch, commit, PR, review verdict, or merge was produced.