Skip to content

feat(fetchers): HackerNewsFetcher — structured thread extraction #58

@chaliy

Description

@chaliy

What

Add a HackerNewsFetcher that matches news.ycombinator.com/item?id={id} URLs, returning structured thread content via the HN Firebase API.

Why

Agents monitoring tech discourse, finding trending tools, or researching community sentiment frequently encounter HN links. The current DefaultFetcher struggles with HN's minimal HTML — comments are deeply nested tables. The Firebase API provides clean, structured data.

Requirements

  • Match: https://news.ycombinator.com/item?id={id}
  • Fetch via Firebase API: https://hacker-news.firebaseio.com/v0/item/{id}.json
  • Return: title, URL, score, author, time, comment count
  • Fetch top N comment threads (recursive kid IDs) with nesting preserved
  • Format field: "hackernews"
  • Handle different item types: story, comment, ask, show, job, poll

Design Notes

  • Firebase API is public, no auth required, generous rate limits
  • Each comment is a separate API call — need to balance depth vs request count
  • Consider fetching top-level comments + 1-2 levels of replies, with truncation indicator
  • Comment threads can be very deep — cap nesting and total comments fetched

Tier

3 — Differentiated capability

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions