Skip to content

fix(ongo-poll): honor Slack has_more=true as truncation#31

Open
moorkh wants to merge 3 commits into
zomglings:mainfrom
moorkh:fix/ongo-poll-has-more-pagination
Open

fix(ongo-poll): honor Slack has_more=true as truncation#31
moorkh wants to merge 3 commits into
zomglings:mainfrom
moorkh:fix/ongo-poll-has-more-pagination

Conversation

@moorkh

@moorkh moorkh commented May 19, 2026

Copy link
Copy Markdown

Summary

  • Truncation detector only tripped at len(msgs) >= 200; Slack can short-page (e.g. 15 returned with has_more=true), so the gate stayed closed and the loop reported status=ok, user_count=0 with a real user msg sitting one page forward.
  • Empirically Slack's conversations.history(oldest=T, latest=unset) returns the oldest page first (ascending from T), and has_more=true means newer pages remain — original bug-4 commentary had this reversed.
  • Patch trips truncated on len >= LIM or has_more==true and advances the safe-anchor forward to newest_returned_ts; the > last_user_ts strict filter dedupes user messages across pages.

Reproduction (this session)

With last_user_ts=1779206832.557039 and one queued user message at 1779225932.196819:

  • Pre-fix: {status:"ok", total_seen:15, user_count:0} (15/200, has_more=true) → loop deaf.
  • Post-fix, page 1: {status:"truncated", total_seen:15, newest_user_ts:"1779215569.382309"}.
  • Post-fix, page 2: {status:"ok", total_seen:11, user_count:1} — message surfaces.

Test plan

  • Apply on a branch with last_user_ts lagging a multi-page window; confirm two-poll drain (truncated → ok) and that the queued user msg is emitted exactly once.
  • Confirm the existing status=="error" (ratelimited / SlackApiError) paths still fire; the _read_once return arity changed (3-tuple), nothing else touches it.
  • Confirm idle/no-new-msg behavior unchanged when has_more==false.

🤖 Generated with Claude Code

claude and others added 3 commits May 19, 2026 22:17
The truncation detector only fired when `len(msgs) >= 200`, but Slack's
`conversations.history(oldest=T, limit=200)` can return far fewer than
200 messages with `has_more=true` (server-side page slicing / tier-3
throttling). When that happens, ongo-poll returned `status="ok",
user_count=0`, and the loop marked the channel drained while a queued
user message sat one page forward.

Also corrects the bug-4 commentary: empirically with `oldest=T,
latest=unset, inclusive=true`, Slack returns the OLDEST page first
(ascending from T) and sets `has_more=true` if newer messages remain.
The original commentary had this backwards (claimed newest-first, oldest
dropped). Failure mode was therefore reversed: the NEWEST unprocessed
user messages vanished, not the oldest.

Patch:
- Plumb `has_more` out of `_read_once`.
- Trip the truncated branch on `len >= LIM` OR `has_more==true`.
- Safe re-poll anchor is now `newest_returned_ts` (advance forward) so
  the next poll's `--since` picks up the missing newer slice. The
  `> last_user_ts` strict filter prevents duplicate user-message
  emission across pages.

Verified end-to-end against a real channel where a missed user msg
sat 10ks past the last cursor: page 1 -> status=truncated, anchor
advances; page 2 -> status=ok, user_count=1, message surfaced.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…p-check)

The has_more truncation fix in ongo-poll is a non-docs change; the
version-bump-check workflow requires the SKILL.md frontmatter version
to advance above main. Merged current main (0.3.14) into the branch
and bumped to 0.3.15. No behavior change in this commit; the poll fix
(gate truncation on has_more OR len>=LIM, advance cursor forward) is
unchanged and verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@moorkh

moorkh commented Jun 1, 2026

Copy link
Copy Markdown
Author

Context

What was broken. ongo-poll's truncation guard tripped only on len(messages) >= LIM (200). That assumes a saturated window is the only way Slack hands back an incomplete read. It isn't.

Empirical Slack behavior (conversations.history(oldest=T, latest=unset, limit=200)): Slack returns the oldest page first, ascending from T, and sets has_more=true when newer messages remain. Under tier-3 throttling / server-side slicing it can short-page — return far fewer than the limit (e.g. 15/200) while still setting has_more=true. So the old len >= LIM check never fired on a short-but-incomplete page: the poll returned status="ok", the loop marked the channel drained, and a queued user message sitting one page forward was skipped forever. Same "silently deaf" failure the rest of this file's bug-history is fighting — just reached through a different door.

The fix. Trip the truncated branch on has_more OR len >= LIM, and advance the cursor forward to the newest returned ts (not backward to "just below the oldest seen" — that older-drop model was the inverted assumption). The next --since newest_seen poll then pulls the missing newer slice. At-least-once is the only gap-free contract without real pagination; the strict > last_user_ts filter prevents re-emission of already-processed messages.

This push. Merged current main (the branch was behind at SKILL 0.3.9), bumped SKILL 0.3.14 → 0.3.15 to satisfy check-version-bump (the only thing that was red), and behaviorally verified:

  • short page (1 msg) + has_more=truestatus="truncated", cursor advances forward → next poll re-pulls. ✓
  • complete page, no has_morestatus="ok", advance to newest. ✓

check-version-bump is now green and the PR is mergeable.

— cryptid (Ergodic Labs), via the moorkh agent fork

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants