Skip to content

Remove donor-site leftovers from docs, templates and comments - #6

Merged
jehrr merged 1 commit into
mainfrom
fix/donor-leftovers
Sep 28, 2026
Merged

jehrr merged 1 commit into
mainfrom
fix/donor-leftovers

Conversation

@jehrr

@jehrr jehrr commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Text copied from the repos this one was bootstrapped from, describing another site as if it were Vrbo. Replacements are taken from this repo's own README, TROUBLESHOOTING, product_parser and canary; nothing new was measured.

Fixed

  • captcha_solver.py — docstring said no challenge had ever been observed and a scored address gets no response; the "What is deliberately NOT here" block described a headless browser refused by Akamai with a 394-byte "Access Denied". Both belong to other sites. Now: Expedia's "Bot or Not?" handler (HTTP 429) picked DataDome on every recorded refusal, and there is deliberately no DataDome path ("this repo does not implement it", per the README). The "No DataDome solver" section the docstring referred to now exists. "lots / lot links" → properties.
  • CONTRIBUTING.md — site-change anchors were another site's (divSRPContentProducts, imgLeg-c, span.flip, a DETAIL-page window.__cache); the pinned-properties list was an auction site's (bid kinds, reserve lots, zero-width price placeholder, favorite_count, bid_count, 100-page cap, 24 suggested lots, headless 403); the akamai note said the string is only in a response header, contradicting this repo's own 899 KB-page measurement; the canary note said the browser build is the gate (the README corrects that); the live-run note said a datacentre gets no response at all. All rewritten for Vrbo. Pyppeteer anecdote now names its sibling (tokopedia-scraper). Form/scope wording: no "registration form", no category listings.
  • Issue templates — bug_report hints and the whole site_changed body/dropdown described another site's __NEXT_DATA__ payload, lots, reserve flags and /en/ locales; rewritten around Vrbo's cards, the two price-container spellings, storefronts, stop_reasons. Example URL vrbo.com/en/c/333-watches / --out grills / "96 products" replaced.
  • diff_runs.py — donor example filenames (watches, girls_), a sold/sold_is_floor/"terjual" paragraph for a column this repo lacks, and a docstring listing original_price/discount_pct/in_stock and JSON-LD provenance. Docstring now lists the real tracked fields and stay_changed.
  • .gitignore — catawiki_products.* → vrbo_products.* (the engines' argparse --out default).
  • SECURITY.md — protection named as Vrbo's (Akamai, Bot-or-Not, DataDome, reCAPTCHA); "no releases" corrected (v0.1.0 exists); pointer to a non-existent "Legal" README section fixed.
  • CHANGELOG — entry under [Unreleased]; removed an empty duplicate [Unreleased] heading. No version bump.

Kept, deliberately

  • .gitignore craigslist-scraper note and captcha_solver's farfetch-scraper notes (and the Kohl's example) — sibling provenance that names its source.
  • Diff_runs "a sibling repo needs it" notes (FX tolerance, auction lifecycle) — name the sibling as a sibling and explain why Vrbo differs.
  • "Expedia", "Akamai", "DataDome" everywhere — Vrbo's own parent, CDN and challenge vendor.
  • README badges — already in family order; "runs without an account" is honest per the README ("No. An ordinary local Chrome on an ordinary residential connection reads this site fine … no key and no proxy").
  • Released [0.1.0] CHANGELOG section untouched.

Checks

python3 smoke_test.py OK before and after (pyppeteer/selenium groups skipped locally, engine absent). ci_checks.py --secret-check and --help-check --sample-check OK.

🤖 Generated with Claude Code

- captcha_solver.py: docstring claimed no challenge was ever observed and
  the "NOT here" section described another site's headless Akamai 403;
  both now state Vrbo's measured Bot-or-Not/DataDome behaviour, and the
  "No DataDome solver" section the docstring referenced now exists.
  "lots"/"lot links" -> properties.
- CONTRIBUTING.md: site-change anchors, pinned-property list, akamai
  marker note, canary note, live-run note, form and scope wording
  rewritten for Vrbo from README/TROUBLESHOOTING/product_parser.
- Issue templates: bug-report hints and site-change template rewritten
  for Vrbo (cards, storefronts, stop_reasons); example URL/command fixed.
- diff_runs.py: donor example filenames, a note on a `sold` column this
  repo lacks, and a docstring listing absent columns and JSON-LD.
- .gitignore: catawiki_products.* -> vrbo_products.* (engines' --out
  default).
- SECURITY.md: protection named as Vrbo's, releases exist, README
  section name corrected.
- CHANGELOG: entry under Unreleased; removed an empty duplicate
  Unreleased heading.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jehrr
jehrr force-pushed the fix/donor-leftovers branch from e27ac6c to abaa399 Compare September 28, 2026 19:50
@jehrr
jehrr merged commit adc3127 into main Sep 28, 2026
7 checks passed
@jehrr
jehrr deleted the fix/donor-leftovers branch September 28, 2026 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant