feat: backwards-compatibility shim for globs/regexps/pseudoUrls (proposal)#3871
Draft
l2ysho wants to merge 1 commit into
Draft
Conversation
Prototype (not part of PR #3533's approved scope) exploring a BC layer for the deprecated `globs`, `regexps`, and `pseudoUrls` enqueueLinks options, so input generated for v3 (e.g. by the Apify Console input components) keeps working on v4 during the migration window. - Map the deprecated options onto `include` via `normalizeDeprecatedPatternOptions`, emitting deprecation warnings. - Per-pattern request options (method/payload/label/userData/headers) are no longer supported; they are ignored with a warning. - Re-adds the `@apify/pseudo_url` dependency for `pseudoUrls` -> regexp conversion (reverses one of the PR's goals; flagged for discussion). - Add tests covering alias mapping, ignored per-pattern options, and combining aliases with `include`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QEq1cCENtNPmFUjQev67wk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
In #3533, @B4nan suggested reconsidering whether we should drop
globs/regexps/pseudoUrlswithout a backwards-compatibility layer, since the Apify Console input components emit these fields and a v3 → v4 migration window will last months (the same input needs to keep working on both).This draft is a concrete prototype of what a lightweight BC layer could look like, so the approach can be evaluated against a hard removal.
What this does
globs,regexps, andpseudoUrlsonEnqueueLinksOptionsas deprecated aliases, mapped ontoinclude:globs→include(1:1)regexps→include(1:1)pseudoUrls→includeviapurlToRegExplog.deprecated(...)warning whenever an alias is used.method,payload,label,userData,headers) that the old pattern objects accepted are not restored — they are ignored with a warning, sincetransformRequestFunctionis now the only supported way to customize per-request options.includepipeline, so the point-1 (strategy AND-ing) and point-5 (empty-array rejection) behavior from feat: simplifyenqueueLinksinterface #3533 is unchanged.Deliberate scope boundaries
enqueueLinksonly; not replicated inenqueueLinksByClickingElements(Playwright/Puppeteer) orSitemapRequestListyet.@apify/pseudo_urldependency, which reverses one of feat: simplifyenqueueLinksinterface #3533's stated goals. That trade-off is the main thing to decide — a BC layer forpseudoUrlsisn't possible without it (or without inlining the PURL → RegExp conversion).Tests
Adds coverage for alias mapping (
globs/regexps/pseudoUrls), per-pattern options being ignored, and aliases combined withinclude.Open questions for the team
@apify/pseudo_urlacceptable, or should the PURL→RegExp conversion be inlined to avoid the dependency?enqueueLinksByClickingElementsandSitemapRequestList?🤖 Generated with Claude Code
https://claude.ai/code/session_01QEq1cCENtNPmFUjQev67wk
Generated by Claude Code