Scraper API: send waitFor as an object, read the target status from http_code

[jehrr]

Two defects in the Scraper API client

Measured 2026-09-23 against https://scraper.2captcha.com/tasks/sync.

1. waitFor was sent as a JSON-encoded string. _build_wait_for returned json.dumps({...}), following a docstring that said the API wanted a double-encoded string. The live API refuses that with HTTP 422 ScrapeParser: params.waitFor must be an object — and still bills the call ($0.0005). The same request with an object answers HTTP 200. So every --wait-text / --wait-element / --wait-state run was a paid exit 5. _build_wait_for now returns a dict (Optional[dict]), logged through json.dumps, and the docstring says what was measured.
2. The target status was read from status. The response is {"status": "success", "http_code": <int>, "headers": …, "body": …}: status is the API's own verdict string, http_code is the target site's HTTP code. The client passed "success" into detect_page_state, so a target 403/503 was never seen (a live rakuten run logged Upstream page status success). It now reads http_code, falling back to status only if that is an int.

Regression check

test_scraper_api_waitfor_is_an_object in smoke_test.py: drives the real parse_args() + fetch_html() with --wait-text, requests.post monkeypatched to capture the payload and return {"status":"success","http_code":403,"body":"<html></html>"}. Asserts waitFor is a dict and the status handed onward is the int 403.

Control: with scraper_api_client.py restored from origin/main (edit asserted to change the file) the suite went red on exactly those two checks and nothing else; with the fix it is green.

Live result

One call with the fixed client, key from the environment:

scraper_api_client.py --url "https://www.vrbo.com/search?destination=Orlando,%20Florida,%20United%20States%20of%20America" --wait-text Orlando --retries 0

→ API HTTP 200 (no 422), Upstream page status 200 (target HTTP code, from http_code), 508,559 bytes, real results-page title — 0 cards, exit 4. That is the shell this module's docstring already warns about: the text "Orlando" is in the page chrome, so the wait resolved before any lodging-card painted. The docstring's own recipe is --wait-element '[data-stid="lodging-card-responsive"]', which I did not spend a second paid call on. The point of this call was the 422, which is gone.

Note: the docstring records a 2026-09-14 run that got HTTP 200 while the client was still sending the string form — so the API appears to have tightened its validation between then and 2026-09-23.

Not changed

• No version bump; the CHANGELOG entry is under [Unreleased].
• The browser engines, the parser and detect_page_state are untouched — they already take an int status; they were just never given one.
• CHANGELOG: the file carried a second, empty ## [Unreleased] heading directly above [0.1.0]; removed it so the new ### Fixed sits in the one Unreleased section. No released section touched.

🤖 Generated with Claude Code