test(fetchers): add live integration tests behind feature flag#84
Merged
test(fetchers): add live integration tests behind feature flag#84
Conversation
Adds `live-tests` feature flag with tests hitting real endpoints for all 13 fetchers. Tests assert structural properties (non-empty, expected substrings) rather than exact content. CI job detects changed fetcher files and runs only the relevant live tests (continue-on-error: true). - `cargo test --features live-tests` runs all live tests - `cargo test --features live-tests live_github_repo` runs one fetcher - Normal `cargo test` is unaffected (tests don't compile without flag)
…ertions Live tests now skip (not fail) when network is unavailable — DNS resolution failures, timeouts, and blocked URLs are treated as infra issues. Twitter test accepts any status code since the API is notoriously unreliable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds live integration tests that hit real endpoints for all 13 fetchers, gated behind a
live-testscargo feature flag. Includes a CI job that selectively runs only the tests for fetchers whose source files changed.Why
Existing tests use wiremock exclusively — good for reliability but no validation that fetchers produce correct output against live sites. If an upstream API changes (GitHub HTML layout, Twitter API shift), we wouldn't know until someone runs examples manually.
How
live-testsincrates/fetchkit/Cargo.toml— tests don't compile without it, socargo testis unaffectedtests/fetcher_live.rswith one module per fetcher (live_github_repo,live_twitter, etc.), using structural assertions (non-empty content, expected substrings)fetch_or_skip()helper treats DNS/timeout/blocked errors as skips, not failures — live tests only fail on unexpected response structurelive-testsinci.ymlruns on PRs only, usesscripts/changed-fetcher-tests.shto detect changed fetcher files and map them to test filters.continue-on-error: trueso flaky endpoints don't block mergesscripts/changed-fetcher-tests.shdiffs against base branch, mapsfetchers/<name>.rs→live_<name>test filterUsage:
Risk
continue-on-error: true)Checklist