Skip to content

fix: send session cookies when fetching x.com for ClientTransaction init - #86

Open
Blacksuite wants to merge 1 commit into
public-clis:mainfrom
Blacksuite:fix/ct-init-authenticated-fetch
Open

fix: send session cookies when fetching x.com for ClientTransaction init#86
Blacksuite wants to merge 1 commit into
public-clis:mainfrom
Blacksuite:fix/ct-init-authenticated-fetch

Conversation

@Blacksuite

@Blacksuite Blacksuite commented Aug 12, 2026

Copy link
Copy Markdown

Problem

_ensure_client_transaction() fetches https://x.com to extract the ondemand.s bundle used to build x-client-transaction-id, but that request goes out without cookies. x.com serves the logged-out landing page to anonymous requests, and that page contains no ondemand.s marker.

So ON_DEMAND_FILE_REGEX.search() returns None, and get_ondemand_file_url() raises:

Failed to init ClientTransaction: 'NoneType' object has no attribute 'group'

_client_transaction stays None, so every subsequent API call is sent without the x-client-transaction-id header. X answers those with HTTP 404 — which is why the failure shows up as a 404 on SearchTimeline and on account-verification endpoints rather than as an auth error, and why retries never help.

Evidence

Same request via curl_cffi with impersonate="chrome", only difference being cookies:

Request Status Body ondemand occurrences Marker matched
Anonymous 200 ~35 KB 0 no
With auth_token + ct0 200 ~278 KB 233 yes

The anonymous response is a valid 200, which is why this fails silently at a warning level instead of surfacing as a request error.

Fix

Set the Cookie header on the init fetch. _build_headers() already constructs exactly this value for API calls, so this reuses the same self._cookie_string or auth_token=…; ct0=… form — no new configuration or code paths.

Verified end-to-end after the change: twitter search returns live results, and ClientTransaction initialized for x-client-transaction-id is logged.

Tests

Two regression tests added to TestEnsureClientTransaction, covering the derived cookie pair and the full _cookie_string passthrough. Both fail on main and pass with the fix:

$ uv run pytest tests/test_client.py -k EnsureClientTransaction -q   # before
2 failed
$ uv run pytest tests/test_client.py -q                              # after
88 passed

uv run ruff check . and uv run mypy twitter_cli both clean.

Fixes #78
Fixes #73
Fixes #69

@Blacksuite
fix: send session cookies when fetching x.com for ClientTransaction init
456c325 
_ensure_client_transaction() fetched https://x.com without cookies, so
x.com returned the logged-out landing page. That page contains no
ondemand.s marker, so ON_DEMAND_FILE_REGEX.search() returns None and
get_ondemand_file_url() raises "'NoneType' object has no attribute
'group'".

ClientTransaction then stays None and every API request goes out without
an x-client-transaction-id header, which X answers with HTTP 404 —
including SearchTimeline and account verification endpoints.

Measured against x.com: anonymous returns ~35KB with zero occurrences of
"ondemand"; the same request with auth_token/ct0 cookies returns ~278KB
with the marker present. _build_headers() already builds exactly this
Cookie value for API calls; this reuses it for the init fetch.

Fixes public-clis#78
Fixes public-clis#73
Fixes public-clis#69
@Tommy00748 Tommy00748 mentioned this pull request Aug 22, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment