Skip to content

fix(tweet): send cookies when fetching the X shell for the transaction ID #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Chilfish merged 5 commits into from
Sep 25, 2026
Merged

fix(tweet): send cookies when fetching the X shell for the transaction ID #21

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3ecfc46
fix(tweet): send cookies when fetching the X shell for the transactio…
Chilfish Sep 25, 2026
94b4102
test(tweet): cover X shell cookie headers and candidate list
Chilfish Sep 25, 2026
d32eed0
docs: document the transaction-ID frontend drift and cookie root cause
Chilfish Sep 25, 2026
da99296
test(tweet): guard the X shell cookie header against regression
Chilfish Sep 25, 2026
0810ab3
docs(tweet): record the AC-TWEET-011 close-out
Chilfish Sep 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/lang/zh-CN/

#### X 前端迁移修复（2026-09-25）

- X 登出首页迁移到 Rolldown/Vite 后不再内嵌 `ondemand.s` webpack chunk map，`x-client-transaction-id` 因此抛 `OnDemandFileUrlResolutionError`，导致所有走 rettiwt-api 的 X 数据接口失败。`FetcherService` 改为在仍提供旧版 webpack 外壳的路由中探测取文档（`X_LEGACY_HOME_URLS` + `isUsableXDocument` 双条件校验），不再固定依赖 `/home`
- 新增 `test/unit/rettiwt-transaction-document.spec.ts`；沉淀 postmortem 013
- X 迁移到 Rolldown/Vite 后不再内嵌 `ondemand.s` webpack chunk map，`x-client-transaction-id` 因此抛 `OnDemandFileUrlResolutionError`，所有走 rettiwt-api 的 X 数据接口失败。**真因是外壳请求漏带登录 cookie**（X 仅对登录态返回仍含该 chunk map 的旧外壳）：新增 `buildXShellHeaders`，配置 API Key 时附带解码后的 cookie，`_fetchXHomePage` 改为使用它
- 外壳候选列表改为 `X_SHELL_URLS`（`/home` 优先，legacy 路由作无 Key 时的 guest 兜底）+ `isUsableXDocument` 双条件校验；新增/扩充 `test/unit/rettiwt-transaction-document.spec.ts`
- 沉淀 postmortem 013 与 `docs/features/tweet/transaction-id.md`（含 `ondemand.s` indices 每构建随机、不可 pin 的实测）
- 补 AC-TWEET-011 回归防护（行为断言：mock `axios.get` 跑真实 `_fetchTransactionDocument`，锁定外壳请求必带 cookie、无 Key 时保持匿名），与 AC 文档 1:1 绑定

#### 验证诚信修复（2026-09-12）

Expand Down
60 changes: 44 additions & 16 deletions app/lib/rettiwt-api/services/public/FetcherService.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,24 +24,52 @@ import { ErrorService } from '../internal/ErrorService'
import { LogService } from '../internal/LogService'

/**
* X routes that still serve the legacy (webpack) client shell.
* Candidate URLs for the X shell document that drives `ClientTransaction`.
*
* X is migrating its frontend to a Rolldown/Vite build ("x-web") that no longer
* embeds the `ondemand.s` webpack chunk map required by `x-client-transaction-id`
* to derive the key byte indices. The landing pages (`/`, `/home`) already serve
* the new build, but the routes below still return the legacy shell, so we probe
* them in order until one exposes the runtime the library needs.
* The `ondemand.s` webpack chunk map required by `x-client-transaction-id` only
* remains in the legacy shell, and X serves that legacy shell only to an
* authenticated session (see `buildXShellHeaders`). The canonical homepage comes
* first; the remaining routes still return the legacy shell anonymously, as a
* fallback for deployments without an API key.
*
* @internal
*/
export const X_LEGACY_HOME_URLS = [
export const X_SHELL_URLS = [
'https://x.com/home',
'https://x.com/i/topics',
'https://x.com/i/display',
'https://x.com/i/timeline',
'https://x.com/i/communitynotes',
'https://x.com/home',
]

/**
* Builds the headers used to fetch the X shell document.
*
* X only serves the shell containing the `ondemand.s` chunk map to an
* authenticated session, so the decoded cookie must be attached whenever an API
* key is configured. An anonymous request returns the new Rolldown/Vite build
* (which dropped the chunk map), making `ClientTransaction` uninitializable.
*
* @param headers - The base request headers.
* @param apiKey - The API key whose cookies are to be attached, if any.
*
* @returns A copy of the given headers, with the cookie attached when applicable.
*
* @internal
*/
export function buildXShellHeaders(
headers: { [key: string]: string },
apiKey?: string,
): { [key: string]: string } {
const shellHeaders = { ...headers }

if (apiKey) {
shellHeaders.cookie = AuthService.decodeCookie(apiKey)
}

return shellHeaders
}

/**
* Whether the given document is a usable legacy X shell for client transaction
* ID generation: it must expose both the site verification key and the
Expand Down Expand Up @@ -162,19 +190,19 @@ export class FetcherService {
/**
* Resolves a DOM document that can drive `ClientTransaction` initialization.
*
* X's new Rolldown/Vite shell dropped the `ondemand.s` webpack chunk map, so
* the previous fixed `/home` fetch fails with `OnDemandFileUrlResolutionError`.
* We probe the legacy routes in `X_LEGACY_HOME_URLS` and return the first
* usable shell, falling back to the last parsed document so the library can
* raise its own specific initialization error.
* X's new Rolldown/Vite shell dropped the `ondemand.s` webpack chunk map, which
* is why the shell must be fetched with the session cookie (see
* `buildXShellHeaders`). We probe the candidates in `X_SHELL_URLS` and return the
* first usable shell, falling back to the last parsed document so the library
* can raise its own specific initialization error.
*
* @returns The parsed X document.
*/
private async _fetchTransactionDocument(): Promise<Document> {
let lastDocument: Document | undefined
let lastError: unknown

for (const url of X_LEGACY_HOME_URLS) {
for (const url of X_SHELL_URLS) {
try {
const document = await this._fetchXHomePage(url)

Expand Down Expand Up @@ -207,9 +235,9 @@ export class FetcherService {
* @returns The parsed X document.
*/
private async _fetchXHomePage(url: string): Promise<Document> {
// Fetch the X page shell
// Fetch the X page shell (with the session cookie, when available)
const homePageResponse = await axios.get<string>(url, {
headers: this.config.headers,
headers: buildXShellHeaders(this.config.headers, this.config.apiKey),
httpAgent: this.config.httpsAgent,
httpsAgent: this.config.httpsAgent,
})
Expand Down
45 changes: 45 additions & 0 deletions docs/development-log/2026-09-25.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,48 @@

**文档**：[postmortem 013](../postmortem/013-upstream-frontend-drift.md) +「高频雷区」第 11 条 +
`CHANGELOG.md`。

## 真正根因：外壳请求缺 cookie（✅ 已完成，同日追加）

**背景**：owner 质疑「X 的重构是逐步的，`/i/topics` 只是还没轮到，库依赖的东西才是核心」——
顺着这个质疑复测，发现根因不在「换哪个页面」。

**决定性实测**（同一 URL `x.com/home`，仅切换 `Cookie`）：

| 请求 | 外壳 | `ondemand.s` |
| ----------------- | -------------- | ------------ |
| 匿名 `/home` | 16.8 KB 新外壳 | ❌ |
| 带 cookie `/home` | 305 KB 旧外壳 | ✅ |

- 上游/此前的 `_handleXMigration()` 只发 `config.headers`，cookie 只在 `request()` 组装 API 请求时
注入 → **外壳请求永远匿名**，即便配置了 API Key。
- 这是**回归**：Rettiwt #885 曾以「抓外壳时附 cookie」修好同一错误，被 #888 合并时丢失；
上游 Lqm1 issue #20 亦归因为「需要有效登录 session」。
- 端到端：`TweetDetail` 与 `TweetLikers`（#908 的复现接口）均成功返回。

**实现**：新增纯函数 `buildXShellHeaders(headers, apiKey)`（存在 key 时附
`AuthService.decodeCookie(apiKey)`），`_fetchXHomePage` 改为使用它；候选列表改名 `X_SHELL_URLS`
并把 `/home` 提到首位（有 cookie 时首个即命中，legacy 路由降级为无 Key 时的 guest 兜底）。

**另**：实测 `ondemand.s` 的 key byte indices **每次构建随机**（30 个日更版本 30/30 不同），
故只可缓存 URL（内容哈希、不可变），不可 pin / 缓存 indices。

**验证**：`test/unit/rettiwt-transaction-document.spec.ts` 扩到 8 用例（含 cookie 头三项断言）；
真实上游 E2E 复核通过。详见 [postmortem 013](../postmortem/013-upstream-frontend-drift.md)
「后续修正」与 [transaction-id.md](../features/tweet/transaction-id.md)。

## AC 收口：外壳 cookie 回归防护（AC-TWEET-011，✅ 已完成）

**动机**：纯函数 `buildXShellHeaders` 的单测只能证明「这个函数对」，**发现不了「调用点不再用它」**——
而这正是本次 bug 的形态（上游 #885 修好、#888 合并时丢失的回归）。所以补一条**行为断言**：
mock `axios.get` 后跑真实 `_fetchTransactionDocument`，断言**实际发出的**外壳请求头 `cookie`
等于解码后的 cookie；无 `apiKey` 时保持匿名。

**落位**：`test/acceptance/ac-tweet.spec.ts`（`describe('AC-TWEET-011: …')`），与
`verify/acceptance-criteria/AC-tweet.md` v1.2 的 AC-TWEET-011 按 ac-contract 契约 1:1 绑定。

**Falsifiability 复核**：临时把 `_fetchXHomePage` 回退为 `this.config.headers` 后，该用例报
`expected undefined to be 'auth_token=…'`——确认它真能捕获本类回归，而非恒真（对齐 postmortem 011
「门禁先问『改坏它会不会红』」）。

**范围**：仅补验证与文档，运行时语义不变——PR #21 的 cookie 修复即最终实现。
30 changes: 15 additions & 15 deletions docs/development-log/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,21 @@

## 时间线（新 → 旧）

| 日期 | 主题 |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| [2026-09-25](2026-09-25.md) | 修复 X 前端迁移（webpack→Rolldown/Vite）击穿 client transaction id 导致 get tweet 全站失败：改从仍提供旧外壳的路由探测取文档（`X_LEGACY_HOME_URLS`）+ `isUsableXDocument` 双条件校验；沉淀 postmortem 013 |
| [2026-09-16](2026-09-16.md) | X Space 卡片接入（audiospace 空壳卡片 → `AudioSpaceById` 补全标题/主播/人数/时长；范围裁定「卡片 + 跳转 X 播放」；AC-SPACE-001~008；不改高危解析器） |
| [2026-09-12](2026-09-12.md) | 验证诚信修复落地（F1 dev server `NODE_ENV` 泄漏 / F2 零匹配守卫 + 模块别名 / F3 死断言 / F4 AC↔测试名元测试 / F5 workflow 步骤解析 / F8 去 fixture 自证 / F9 AC-TRANS 命名 / F10 断言 lint / F12 README / F13 deprecated 参数）+ postmortem 011 |
| [2026-09-11](2026-09-11.md) | anon-tweet Agent Skill 升级 v1.2.0（随附 PowerShell 封装 + OpenAPI 快照；随附 spec 与 llms.ts 14/14 奇偶一致）+ 版本单源治理（应用版本归 package.json / skill 版本归 SKILL.md，去手写副本）+ 验证套件诚信审查（AC 空跑报绿 / 静态扫描冒充行为断言；修复 backlog F1~F13） |
| [2026-09-06](2026-09-06.md) | 可安装 PWA + Web Share Target（系统分享直达推文/IG；极简网络透传 SW + 首页 GET 接收复用 TweetInputForm） |
| [2026-09-04](2026-09-04.md) | 媒体图片链接统一为 `/slug.jpg`（去掉 `?name=&format=` 查询参数，normalizeMediaUrl 收口） |
| [2026-08-31](2026-08-31.md) | 推文搜索接入（/search 页面 + /api/tweet/search + rettiwt-api guest 授权补缺）+ llms.txt / openapi.json 接入 + tweet GET 全部入口统一走缓存链（getLocalTweet） |
| [2026-08-19](2026-08-19.md) | 主线推送 34 提交 + baseUrl 白名单改可选加固（默认关闭） |
| [2026-08-18](2026-08-18.md) | 修复 unified_card 解析只认 `details_1` 布局（Trending/topic 卡片丢失 og 信息） |
| [2026-08-17](2026-08-17.md) | 修复「句首补充」显示与持久化链路（index 对齐合并丢实体 + AI 流丢片段 + 保存即同步） |
| [2026-08-14](2026-08-14.md) | 测试基线红灯修复（vitest 冷加载超时，独立任务） |
| [2026-08-13](2026-08-13.md) | AI 视觉描述子系统规划（OpenRouter + MiMo-V2.5，文档先行 Phase 0） |
| [2026-08-09](2026-08-09.md) | 工程规范对标 Float + verify 二期 S5/S6/S8（CI / Screenshot Verifier / --server 自动管理） |
| 日期 | 主题 |
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [2026-09-25](2026-09-25.md) | 修复 X 前端迁移（webpack→Rolldown/Vite）击穿 client transaction id 导致 get tweet 全站失败：改从仍提供旧外壳的路由探测取文档 → **真因 = 外壳请求漏带 cookie**（`buildXShellHeaders`，仅登录态返回含 `ondemand.s` 的旧外壳）+ `X_SHELL_URLS` 候选校验作 guest 兜底；沉淀 postmortem 013 + AC-TWEET-011 外壳 cookie 回归防护 |
| [2026-09-16](2026-09-16.md) | X Space 卡片接入（audiospace 空壳卡片 → `AudioSpaceById` 补全标题/主播/人数/时长；范围裁定「卡片 + 跳转 X 播放」；AC-SPACE-001~008；不改高危解析器） |
| [2026-09-12](2026-09-12.md) | 验证诚信修复落地（F1 dev server `NODE_ENV` 泄漏 / F2 零匹配守卫 + 模块别名 / F3 死断言 / F4 AC↔测试名元测试 / F5 workflow 步骤解析 / F8 去 fixture 自证 / F9 AC-TRANS 命名 / F10 断言 lint / F12 README / F13 deprecated 参数）+ postmortem 011 |
| [2026-09-11](2026-09-11.md) | anon-tweet Agent Skill 升级 v1.2.0（随附 PowerShell 封装 + OpenAPI 快照；随附 spec 与 llms.ts 14/14 奇偶一致）+ 版本单源治理（应用版本归 package.json / skill 版本归 SKILL.md，去手写副本）+ 验证套件诚信审查（AC 空跑报绿 / 静态扫描冒充行为断言；修复 backlog F1~F13） |
| [2026-09-06](2026-09-06.md) | 可安装 PWA + Web Share Target（系统分享直达推文/IG；极简网络透传 SW + 首页 GET 接收复用 TweetInputForm） |
| [2026-09-04](2026-09-04.md) | 媒体图片链接统一为 `/slug.jpg`（去掉 `?name=&format=` 查询参数，normalizeMediaUrl 收口） |
| [2026-08-31](2026-08-31.md) | 推文搜索接入（/search 页面 + /api/tweet/search + rettiwt-api guest 授权补缺）+ llms.txt / openapi.json 接入 + tweet GET 全部入口统一走缓存链（getLocalTweet） |
| [2026-08-19](2026-08-19.md) | 主线推送 34 提交 + baseUrl 白名单改可选加固（默认关闭） |
| [2026-08-18](2026-08-18.md) | 修复 unified_card 解析只认 `details_1` 布局（Trending/topic 卡片丢失 og 信息） |
| [2026-08-17](2026-08-17.md) | 修复「句首补充」显示与持久化链路（index 对齐合并丢实体 + AI 流丢片段 + 保存即同步） |
| [2026-08-14](2026-08-14.md) | 测试基线红灯修复（vitest 冷加载超时，独立任务） |
| [2026-08-13](2026-08-13.md) | AI 视觉描述子系统规划（OpenRouter + MiMo-V2.5，文档先行 Phase 0） |
| [2026-08-09](2026-08-09.md) | 工程规范对标 Float + verify 二期 S5/S6/S8（CI / Screenshot Verifier / --server 自动管理） |

## 记录约定

Expand Down
Loading
Loading