Skip to content

feat(core): provide requests to skipped callbacks#3867

Open
AkhilTrivediX wants to merge 1 commit into
apify:masterfrom
AkhilTrivediX:agent/expose-skipped-request
Open

feat(core): provide requests to skipped callbacks#3867
AkhilTrivediX wants to merge 1 commit into
apify:masterfrom
AkhilTrivediX:agent/expose-skipped-request

Conversation

@AkhilTrivediX

Copy link
Copy Markdown

Summary

  • pass the full Request to onSkippedRequest instead of a URL string
  • preserve request metadata for skips caused by filters, robots.txt, limits, crawl depth, and redirects
  • update the crawl-depth coverage to verify userData reaches the callback

Why

onSkippedRequest previously discarded request metadata, which made it impossible for handlers to inspect fields such as userData. This follows the v4 API direction in #3801: callers receive the request object and can read its URL from request.url.

Validation

  • corepack yarn eslint packages/basic-crawler/src/internals/basic-crawler.ts packages/browser-crawler/src/internals/browser-crawler.ts packages/core/src/enqueue_links/enqueue_links.ts packages/core/src/enqueue_links/shared.ts packages/http-crawler/src/internals/http-crawler.ts packages/playwright-crawler/src/internals/adaptive-playwright-crawler.ts test/core/crawlers/basic_crawler.test.ts
  • corepack yarn tsc-check-tests
  • corepack yarn vitest run test/core/crawlers/basic_crawler.test.ts --silent (68 passed)

Closes #3801

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provide Request instance in onSkippedRequest callback

3 participants