tooling: update line-implement and retire the old commands

.agents/commands/line-analyze.md → .agents/skills/line-analyze/SKILL.md

@@ -1,48 +1,70 @@
-Parse and analyze a LINE Chrome Extension network capture.
+---
+name: line-analyze
+description: Parses and redacts a LINE Chrome Extension network capture (CDP capture or Chrome net-log) and compares it with the beeper/line bridge code. Use after a line-capture run, or when given a LINE net-log or CDP capture file to analyze.
+---
+
+# LINE Analyze
+
+Turns a raw LINE Chrome Extension capture into redacted, readable findings and
+compares them with the bridge. Run the commands below from the repository root.
 
 ## Input
 
-Use the provided path to a net-log JSON file. If no path is provided, default to
+Use the provided path to a capture file. Prefer the CDP capture
+(`line-cdp-capture.json`): it includes SSE messages and request bodies. The
+Chrome net-log is a backup. If no path is provided, default to
+`/tmp/line-chrome-extension-capture/line-cdp-capture.json`, then
 `/tmp/line-chrome-extension-capture/line-net-log.json`.
 
 ## What to do
 
-1. Run the parser:
+1. Run the parser. This extracts LINE-related HTTP requests and writes a
+   `-parsed.json` file:
+
+   ```bash
+   python3 .agents/skills/line-analyze/scripts/parse-line-traffic.py <capture-file> [<parsed-output.json>]
    ```
-   python3 scripts/parse-line-traffic.py <net-log-file>
+
+2. Redact it before reading or quoting anything. This removes tokens, MIDs,
+   `picturePath` and raw body bytes:
+
+   ```bash
+   python3 .agents/skills/line-analyze/scripts/redact-line-capture.py <parsed-output.json>
    ```
-   This extracts LINE-related HTTP requests from the Chrome net-log and writes a `-parsed.json` file.
 
-2. Read the parsed output JSON file.
+   Read the raw files only locally, and only when the redacted output loses
+   information you need. Quote or document only redacted snippets.
 
-3. Analyze the traffic and present findings organized by category:
+3. Read the redacted output and present findings by category:
 
    - **Thrift RPC** (`/api/talk/thrift/`): The main LINE API. Requests use LINE's Thrift-over-HTTP protocol. The `x-lhm` header indicates the Thrift method name. Compare against methods in `pkg/line/methods.go`.
-
    - **Long-poll / SSE** (`/api/talk/long-polling/`, `/api/operation/receive`): Real-time event delivery. LF1 is used during login verification, JQ for login polling, and SSE for ongoing message/operation streaming. Compare against `pkg/line/sse.go` and `pkg/line/client.go`.
-
    - **Auth** (`/api/auth/`): Token refresh and login flows. Compare against login methods in `pkg/line/client.go` (loginV2, confirmE2EELogin, tokenRefresh).
-
    - **Media/OBS** (`obs.line-apps.com`): Media upload/download. Check `x-obs-params` header for operation details, `x-obs-oid` in responses. Compare against `pkg/line/client.go` (UploadOBS, DownloadOBS).
-
    - **Stickers/CDN** (`stickershop`, `scdn`): Sticker image downloads.
-
    - **Secondary API** (`legy-jp.line-apps.com`): Link previews and page info. Compare against `pkg/line/client.go` (GetPageInfo).
 
 4. For each interesting request, explain:
    - What the bridge equivalent is (or note if it's not implemented yet)
    - The request/response structure
    - Any headers that carry protocol-specific meaning
 
-5. Highlight any **unimplemented** endpoints or behaviors that differ from what the bridge currently does. Cross-reference with `pkg/line/methods.go` and `pkg/connector/`.
+5. Highlight any **unimplemented** endpoints or behaviors that differ from what
+   the bridge currently does. Cross-reference with `pkg/line/methods.go` and
+   `pkg/connector/`.
 
-6. After presenting the analysis, ask the user if they want to clean up the capture files (`/tmp/line-chrome-extension-capture/`). Warn that the raw net-log and parsed output contain sensitive data (auth tokens in `X-Line-Access` headers, message IDs, etc.) and should not be left on disk or committed to the repository. If the user agrees, delete the entire capture directory.
+6. After presenting the analysis, ask the user if they want to clean up the
+   capture files (`/tmp/line-chrome-extension-capture/`). Warn that the raw
+   capture and parsed output contain sensitive data (auth tokens in
+   `X-Line-Access` headers, message IDs, etc.) and must not be left on disk or
+   committed to a repository. If the user agrees, delete the entire capture
+   directory.
 
 ## LINE protocol context
 
 - The bridge identifies as LINE Chrome Extension: header `x-line-application` contains the app version string.
 - Thrift RPC uses HTTP POST to `line-chrome-gw.line-apps.com/api/talk/thrift/Talk/<Service>/<Method>`.
 - Auth token is in `x-line-access` header.
 - E2EE messages have `chunks` field containing encrypted content (base64 segments).
-- Operations have numeric `type` codes — see `pkg/connector/sync.go` and `pkg/connector/handle_message.go` for the operation type switch.
+- Operations have numeric `type` codes. See `pkg/connector/sync.go` and `pkg/connector/handle_message.go` for the operation type switch.
 - Body content in the parsed output may be base64-encoded binary (Thrift serialization). Note the method name from headers rather than trying to decode the binary body.

scripts/parse-line-traffic.py → .agents/skills/line-analyze/scripts/parse-line-traffic.py

@@ -2,7 +2,7 @@
 """Parse LINE network captures (CDP or net-log format).
 
 Usage:
-    python3 scripts/parse-line-traffic.py <capture.json> [output.json]
+    python3 .agents/skills/line-analyze/scripts/parse-line-traffic.py <capture.json> [output.json]
 
 Supports two input formats:
 - CDP capture (from line-cdp-capture.py): has SSE message bodies, Thrift response bodies
@@ -231,14 +231,17 @@ def parse_cdp_capture(data: dict) -> dict:
             entry["status"] = req["status"]
 
         # Extract interesting request headers
-        headers = req.get("headers", {})
+        # requestWillBeSentExtraInfo carries the headers Chrome really sent
+        # (Cookie, Origin, Sec-*, X-Line-*), which requestWillBeSent omits.
+        headers = {**req.get("headers", {}), **req.get("request_extra_headers", {})}
         interesting_req = {}
         for k, v in headers.items():
             if k.lower() in (
                 "content-type", "x-line-access", "x-line-application",
                 "x-lhm", "x-lpv", "x-lsr", "x-obs-params",
-                "x-line-channeltoken", "accept",
-            ):
+                "x-line-channeltoken", "accept", "origin", "user-agent",
+                "x-line-chrome-version", "x-line-mid",
+            ) or k.lower().startswith("x-line-") or k.lower().startswith("x-ls"):
                 interesting_req[k] = v
         if interesting_req:
             entry["request_headers"] = interesting_req
@@ -270,7 +273,19 @@ def parse_cdp_capture(data: dict) -> dict:
 
         # Include request body if available
         if "request_body" in req:
-            entry["request_body_text"] = req["request_body"]
+            body = req["request_body"]
+            if isinstance(body, dict):
+                # Raw captures keep the bytes in base64; the parsed output only
+                # keeps the text form and a size + sha256 for binary bodies.
+                if body.get("error"):
+                    entry["request_body_error"] = body["error"]
+                else:
+                    entry["request_body_size"] = body.get("size")
+                    entry["request_body_sha256"] = body.get("sha256")
+                    if body.get("text") is not None:
+                        entry["request_body_text"] = body["text"]
+            else:
+                entry["request_body_text"] = body
 
         entry["timestamp"] = req.get("timestamp", "")
         line_requests.append(entry)

scripts/redact-line-capture.py → .agents/skills/line-analyze/scripts/redact-line-capture.py

@@ -2,7 +2,7 @@
 """Redact sensitive values from LINE capture JSON files.
 
 Usage:
-    python3 scripts/redact-line-capture.py input.json [output.json]
+    python3 .agents/skills/line-analyze/scripts/redact-line-capture.py input.json [output.json]
 
 The default output path is input-redacted.json. This helper preserves JSON
 structure for protocol analysis while removing credentials, cookies, tokens,
@@ -39,6 +39,16 @@
     "refresh_token",
     "session",
     "certificate",
+    # Identifiers that point at a person, group or uploaded object.
+    "mid",
+    "chatmid",
+    "contactmid",
+    "picturepath",
+    "picturestatus",
+    "oid",
+    "x-obs-oid",
+    # Raw request-body bytes from the CDP recorder (uploads, Thrift payloads).
+    "base64",
 }
 
 SENSITIVE_KEY_FRAGMENTS = (
@@ -53,7 +63,8 @@
 
 EMAIL_RE = re.compile(r"(?i)[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,}")
 JWT_RE = re.compile(r"eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}")
-MID_RE = re.compile(r"\b([ucr][0-9a-f]{20,})\b", re.IGNORECASE)
+# Not \b: "_" counts as a word character there, which hid MIDs after "_".
+MID_RE = re.compile(r"(?<![0-9A-Za-z])([ucr][0-9a-f]{20,})(?![0-9A-Za-z])", re.IGNORECASE)
 LONG_SECRET_RE = re.compile(r"^[A-Za-z0-9+/=_-]{96,}$")