Skip to content

feat(tools/discourse): bound read requests and response bodies - #32

Draft
MakerFlyFly wants to merge 3 commits into
audit/discourse-read-budget-basefrom
feat/discourse-read-budget
Draft

MakerFlyFly wants to merge 3 commits into
audit/discourse-read-budget-basefrom
feat/discourse-read-budget

Conversation

@MakerFlyFly

@MakerFlyFly MakerFlyFly commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Share 64 HTTP request attempts and 32 MiB of consumed response bodies across each Discourse read command, including discovery, pagination, post batches, lookahead and redirects.
  • Reject a known batch shortfall before fetching missing posts. Bound success/error-body reads, close unused redirect bodies, refuse requests when bytes are spent, and preserve HTTP status when error bodies exceed the allowance.
  • Keep count guidance in the CLI, separate from library errors and unrelated failures. Preserve count-based truncation, GET-only credential boundaries and partial archive coverage, with no new settings, retries or runtime dependencies.

Type of change

  • Tool / bridge contract
  • Python package
  • Documentation

Test plan

Candidate: ae938cbec969c37e7bf6489f7672e0193b18ab2f, based on dbaa84f3e6c294be30c68cb2abba36c623d45d47.

  • 427 package tests pass on Linux/Python 3.12.12 and Windows/Python 3.12.13: 370 view/list cases and 57 budget cases. Ruff, format and mypy pass.
  • With 3000 posts and 20 initially returned, max-posts 1280 succeeds in 64 requests; 1281 and 2000 fail after one request without successful JSON. Cover short topics, larger initial batches, initial redirects, no missing posts, and runtime exhaustion after a successful batch.
  • Verify library preflight messages contain the Python parameter name without CLI flags. View/list CLI hints appear once for request-budget failures, including with or without since; permission, connection, parsing and byte-budget errors do not receive those hints.
  • Cover dynamic list capacity, required lookahead, config reuse/copying, exact byte exhaustion, retained 403/502 status, and valid versus unterminated chunked responses. The 3000-topic/two-match/two-request cutoff remains covered.
  • Integration members pass from the repository root: vendor-neutrality-score 42, skill-and-tool-validator 493, dev 380, spec-validator 87 and spec-inventory 31.
  • prek run --all-files --skip workspace-pytest: 42 Passed. Committed-diff pytest against the fixed base selects only Discourse and passes 427 cases.
  • Magpie complete-PR A–E: 0 blocking / 0 advisory for three commits and eight files. Full/incremental patch application checks pass; all eight changed-file digests match Linux and Windows.
  • This guidance-only follow-up makes no live forum requests. Historical anonymous verification on f8eb46e: one HTTPS GET to topic 183261, 104,226 consumed body bytes, one returned post and truncated: true, with no writes; that result is not relabelled as a smoke test of the current head.
  • zizmor 1.31.1 offline has no unsuppressed findings (1 ignored, 18 suppressed); lychee passes offline. Optional local native Claude validation is skipped because its CLI is absent. Full-workspace pytest and online API-dependent audits were not run. Test collections overlap and are not summed; cloud CI is recorded separately.

RFC-AI-0004 compliance

  • Sandbox — preserve configured HTTPS origin/path restrictions; no host-allowance changes.
  • Vendor neutrality — Discourse behavior stays in its partial adapter.
  • Write-access discipline — GET only, including redirects; automated HTTP tests use mocks.
  • Privacy LLM — no model calls; callers establish visibility and apply existing privacy rules.

Linked issues

Owner audit: MakerFlyFly/magpie#31.
Refs apache/magpie#307, following merged #1755 and #1841.

Notes for reviewers

  • This Draft PR targets audit/discourse-read-budget-base: three commits, eight files, +775/-27. The audit history is preserved.
  • Limits are engineering defaults without CLI/environment overrides in this minimal slice. Larger legitimate reads may fail. Capacity depends on initial posts, page sizes, uniqueness, redirects, end signals and remaining bytes; it is not a fixed argument ceiling or a guaranteed maximum-post recommendation.
  • Preflight computes a minimum batch count without reserving requests. Later redirects, byte overflow or fetch errors can still fail the read; truncated describes only the output-count limit.
  • The CLI creates a fresh config per invocation. Python callers reusing a config, including dataclasses.replace without a new budget, share its mutable allowance; independent operations require a fresh config.
  • Body bytes are counted before decoding/JSON parsing, with at most one extra byte to detect overflow. Headers, transport overhead and unread data are excluded; this is not a process-memory cap.
  • The 30-second blocking-I/O timeout is not a total command deadline. Search, user activity, writes and the remaining archive verbs are outside this patch; coverage remains partial.

AI disclosure

AI-generated PR description. Codex (GPT-6) assisted implementation, review-response analysis, tests, validation and preparation. Owner review of the updated candidate is pending; this remains a Draft for human audit.

Topic and category reads can accumulate requests and response bodies even
when their output counts are capped. Share a command-level allowance of
64 HTTP attempts and 32 MiB of consumed response bodies across discovery,
pagination, post batches and redirects. Fail incomplete reads without
successful JSON and preserve count-based truncation.

Read success and error bodies in bounded chunks, close unused redirect
bodies without draining them, and preserve urllib routing and credential
boundaries. Keep the existing I/O timeout without claiming a total deadline.

Refs: apache#307
Generated-by: Codex (GPT-6)
@MakerFlyFly MakerFlyFly added contract:mail-archive Public mailing-list and forum archive reads enhancement New feature or request family:docs family:tools labels Oct 31, 2026 — with ChatGPT Codex Connector
Once the topic stream is known, reject a minimum batch count that cannot
fit the remaining request budget before fetching any missing posts. Keep
runtime accounting for redirects and report list-specific count hints
only for request-budget failures.

Retain HTTP status codes when an error body exhausts the byte allowance,
and refuse a new request after the byte allowance is spent. Require an
explicit budget in the transport helper and document config lifetime,
conditional capacities and the existing timeout boundary.

Refs: apache#307
Generated-by: Codex (GPT-6)
Keep topic-library errors focused on max_posts and the request allowance.
Add command-line count guidance only in the view/list CLI handlers for
request-budget failures, without attaching hints to unrelated errors.

Clarify since guidance, shared budget accounting and the stream-prefix
condition in the capacity example. Preserve all budget limits and read
semantics.

Refs: apache#307
Generated-by: Codex (GPT-6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contract:mail-archive Public mailing-list and forum archive reads enhancement New feature or request family:docs family:tools

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant