diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..e4e073f --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,164 @@ +# CodexMeter — Claude Provider Runbook + +When the iPhone app / display reports **`s:0, w:0` ("0/0") for the Claude +provider while real usage exists,** something in the Claude data pipeline +has broken. This file is the triage runbook. Read it before making +changes — every entry here is from a real outage we already debugged. + +## The Claude data pipeline + +The Claude provider has no first-class API, so the daemon stitches +together four sources in `local_claude_activity_snapshot()` +(`daemon/codex-usage-daemon.py`). They are tried in order; the first to +return a real snapshot wins. + +1. **`~/.claude/plan-limits.json`** — written by + `scripts/claude-statusline-limits.py`, which Claude Code itself runs + from its `statusLine` hook (`~/.claude/settings.json`). Only contains + `rate_limits` when Claude Code is in an interactive Pro/Max session + and Anthropic chooses to populate them; can be silent for hours. + TTL: `CLAUDE_HOOK_TTL_SECONDS` (default 15 min). +2. **`~/.config/codexmeter/claude-plan-usage.json`** — written by + `scripts/claude-plan-usage-firefox.py` (primary) every 60 s via the + `com.justin.codexmeter.claude-plan-scrape` LaunchAgent. Reads + `sessionKey` from Firefox's unencrypted `cookies.sqlite` and calls + `https://claude.ai/api/organizations/{org}/usage` directly. Structured + JSON — immune to claude.ai page-layout changes. + TTL: `CLAUDE_PLAN_TTL_SECONDS` (default 60 min). +3. **`~/.config/codexmeter/claude-statusline.json`** — legacy raw + statusline capture; same data as #1 but read from a different cache. +4. **Stale cached `claude-plan-usage.json`** — past TTL, surfaced with + `ok=False` and `status="stale Nh"`. Intentional: better to show the + user the *last known good* percentages with a stale flag than a + confidently-wrong `s:0, w:0`. The token-count fallback is **off** + for the Claude provider — that path produced false 0% any time the + user exceeded the synthetic 200k-token budget. + +The `claude-plan-usage-cache.py` (Chrome AppleScript) script still exists +as a manual fallback only — it depends on Chrome's "Allow JavaScript +from Apple Events" setting, which Chrome can disable silently on update. +Do not put it back into the LaunchAgent. + +## Triage: iOS app shows 0/0 for Claude + +Always start here: + +```bash +curl -s http://localhost:9596/health | python3 -m json.tool +``` + +This single endpoint tells you which sources are stale, what the last +scrape attempt said, and what snapshot the daemon is currently serving. +Common failure modes and what `/health` shows: + +### A. Firefox session expired +`scrape.ok = false`, message contains `HTTP 403` and +`account_session_invalid`. Fix: + +```bash +./scripts/claude-plan-usage-firefox.py login # opens Firefox to /login +# sign in, then: +./scripts/claude-plan-usage-firefox.py +``` + +The cache should refresh within 60 s on the next LaunchAgent tick. + +### B. Daemon is running stale code +After pulling `main`, the long-running daemon still holds the old bytes. +Restart it: + +```bash +launchctl kickstart -k gui/$UID/com.justin.codexmeter.claude +``` + +### C. LaunchAgent isn't running +```bash +launchctl list | grep codexmeter +launchctl kickstart -k gui/$UID/com.justin.codexmeter.claude-plan-scrape +tail -20 ~/Library/Logs/codexmeter-claude-scrape.err.log +``` + +If the LaunchAgent disappeared (e.g., after rebooting before installing +it persistently), re-run `./install.sh`. + +### D. claude.ai changed the page format +Only relevant to the legacy Chrome scrape. The Firefox path hits the API +and is immune. If somebody re-enables the Chrome path, see commit +`ea0ff80` / PR #32 for the regex update pattern. + +### E. Chrome disabled "Allow JavaScript from Apple Events" +Legacy Chrome-scrape path only. If `claude-scrape-health.json.message` +contains `Executing JavaScript through AppleScript is turned off`, the +permanent fix is to switch the LaunchAgent back to the Firefox script. +Do **not** rely on toggling the Chrome menu; it can flip off again. + +### F. mDNS name collision (rare) +Both Codex and Claude daemons used to register `codexmeter._http._tcp.local.` +with the same instance name, causing the second to fail silently. If you +add a third daemon variant, make sure each gets a unique mDNS instance +name (provider-suffixed). Already tracked as a follow-up task. + +## Why we never show 0% for "unknown" + +The token-counting fallback (`claude_project_files()` aggregation) +existed pre-2026-05 and shipped data like +`"s":0,"sr":NNN,"w":0,"wr":NNN,"st":"12467k tok today"` whenever the +user blew past the hardcoded 200k daily / 1M weekly budget. The iOS +display rendered that as "0% remaining," which is indistinguishable from +"daemon dead." We kept losing hours to that ambiguity. The new contract: + +- If a fresh source has real numbers → return them, `ok=True`. +- If no fresh source but a stale cache exists → return the cached + percentages with `ok=False` and a `"stale Xh"` status. The iOS app + should treat `ok=False` as "data is suspect" and surface that visibly. +- The token-count branch is no longer reachable for the Claude provider. + +## Endpoints + +``` +GET http://localhost:9596/usage # compact firmware payload +GET http://localhost:9596/status # full SharedSnapshot +GET http://localhost:9596/health # diagnostic — source freshness, last scrape, current snapshot +``` + +## Files and processes + +| Path / process | What it is | +|---|---| +| `daemon/codex-usage-daemon.py` | HTTP daemon, polls every 30 s | +| `scripts/claude-plan-usage-firefox.py` | Primary scrape (Firefox cookies → claude.ai API) | +| `scripts/claude-plan-usage-cache.py` | Legacy Chrome AppleScript scrape — manual fallback only | +| `scripts/claude-statusline-limits.py` | Persists Claude Code statusLine `rate_limits` | +| `~/Library/LaunchAgents/com.justin.codexmeter.claude.plist` | Daemon (Claude provider, port 9596) | +| `~/Library/LaunchAgents/com.justin.codexmeter.claude-plan-scrape.plist` | Scrape every 60 s | +| `~/.config/codexmeter/claude-plan-usage.json` | Scrape output | +| `~/.config/codexmeter/claude-scrape-health.json` | Scrape last-run status | +| `~/.claude/plan-limits.json` | Statusline hook output | + +## Adding a new data source + +Follow the pattern: write a `claude_*_snapshot()` function that returns +`UsageSnapshot | None`, and insert it into `local_claude_activity_snapshot()` +**before** the stale-cache fallback. Require both `five_hour` and +`seven_day` (or `session` and `weekly`) windows — a partial payload +silently overriding a complete one is the recurring bug pattern +(see PR #29 review, PR #32 review). + +## Useful one-liners + +```bash +# Force-refresh scrape and see what it'd write +.venv/bin/python scripts/claude-plan-usage-firefox.py print + +# Watch the daemon's view in real time +watch -n 5 'curl -s http://localhost:9596/usage' + +# Re-open Firefox login (if session expired) +./scripts/claude-plan-usage-firefox.py login + +# All Claude-related logs +tail -f ~/Library/Logs/codexmeter-claude*.log + +# Confirm both daemons are alive +launchctl list | grep codexmeter +``` diff --git a/daemon/codex-usage-daemon.py b/daemon/codex-usage-daemon.py index becd846..76e0770 100755 --- a/daemon/codex-usage-daemon.py +++ b/daemon/codex-usage-daemon.py @@ -985,6 +985,45 @@ def claude_limit_message_snapshot(message: str) -> UsageSnapshot | None: ) +def claude_stale_plan_usage_snapshot() -> UsageSnapshot | None: + """Return the last cached plan-usage file even if past TTL. + + Used so that when every fresh source fails we still surface the most + recent real percentages with ok=False, instead of dropping to the + token-count fallback that reports a false 0%. + """ + try: + stat = CLAUDE_PLAN_USAGE_FILE.stat() + except OSError: + return None + try: + data = json.loads(CLAUDE_PLAN_USAGE_FILE.read_text(errors="replace")) + except (OSError, json.JSONDecodeError): + return None + session = data.get("session") if isinstance(data.get("session"), dict) else None + weekly = data.get("weekly") if isinstance(data.get("weekly"), dict) else None + if session is None or weekly is None: + return None + + age_minutes = max(0, round((time.time() - stat.st_mtime) / 60)) + session_used = float(session.get("used_pct") or 0) + weekly_used = float(weekly.get("used_pct") or 0) + session_reset = int(session.get("reset_mins") or -1) + weekly_reset = int(weekly.get("reset_mins") or -1) + if age_minutes >= 60: + stale_text = f"stale {age_minutes // 60}h{age_minutes % 60}m" + else: + stale_text = f"stale {age_minutes}m" + return UsageSnapshot( + session_pct=remaining_pct_from_used(session_used), + session_reset_mins=session_reset, + weekly_pct=remaining_pct_from_used(weekly_used), + weekly_reset_mins=weekly_reset, + status=stale_text, + ok=False, + ) + + def local_claude_activity_snapshot() -> UsageSnapshot: hook_snapshot = claude_hook_limits_snapshot() if hook_snapshot is not None: @@ -998,6 +1037,13 @@ def local_claude_activity_snapshot() -> UsageSnapshot: if rate_limit_snapshot is not None: return rate_limit_snapshot + # All fresh sources failed. Prefer the last known good cached numbers + # (marked ok=False) over the token-count budget path, which produces a + # misleading 0% any time the user blows past the synthetic budget. + stale_snapshot = claude_stale_plan_usage_snapshot() + if stale_snapshot is not None: + return stale_snapshot + day_start = datetime.combine( datetime.now().astimezone().date(), datetime.min.time() ).astimezone() @@ -1493,6 +1539,51 @@ def _payload_timestamp(self) -> float: return parsed.timestamp() if parsed else time.time() +def _file_health(path: Path, ttl_seconds: int | None = None) -> dict: + info: dict[str, Any] = {"path": str(path), "exists": path.exists()} + if not info["exists"]: + return info + try: + stat = path.stat() + age = time.time() - stat.st_mtime + info["age_seconds"] = int(age) + if ttl_seconds is not None: + info["fresh"] = age <= ttl_seconds + except OSError as exc: + info["error"] = f"{type(exc).__name__}: {exc}" + return info + + +def _health_payload() -> str: + health: dict[str, Any] = { + "provider": PROVIDER, + "checked_at": iso_now(), + } + if PROVIDER == "claude": + health["sources"] = { + "hook": _file_health(CLAUDE_LIMITS_FILE, CLAUDE_HOOK_TTL_SECONDS), + "plan_usage": _file_health(CLAUDE_PLAN_USAGE_FILE, CLAUDE_PLAN_TTL_SECONDS), + "statusline": _file_health(CLAUDE_RATE_LIMITS_FILE, CLAUDE_HOOK_TTL_SECONDS), + } + scrape_health_file = CACHE_DIR / "claude-scrape-health.json" + if scrape_health_file.exists(): + try: + health["scrape"] = json.loads(scrape_health_file.read_text(errors="replace")) + except (OSError, json.JSONDecodeError): + pass + try: + snapshot = local_claude_activity_snapshot() + health["current"] = { + "session_pct": snapshot.session_pct, + "weekly_pct": snapshot.weekly_pct, + "status": snapshot.status, + "ok": snapshot.ok, + } + except Exception as exc: # noqa: BLE001 + health["current"] = {"error": f"{type(exc).__name__}: {exc}"} + return json.dumps(health, separators=(",", ":")) + + class CodexMeterHTTPHandler(http.server.BaseHTTPRequestHandler): shared: SharedSnapshot | None = None @@ -1512,6 +1603,13 @@ def do_GET(self) -> None: self.send_header("Cache-Control", "no-cache") self.end_headers() self.wfile.write(payload.encode("utf-8")) + elif path == "/health": + self.send_response(200) + self.send_header("Content-Type", "application/json") + self.send_header("Access-Control-Allow-Origin", "*") + self.send_header("Cache-Control", "no-cache") + self.end_headers() + self.wfile.write(_health_payload().encode("utf-8")) else: self.send_response(404) self.end_headers() @@ -1528,7 +1626,10 @@ async def start_http_server(port: int, shared: SharedSnapshot) -> threading.Thre raise RuntimeError(f"HTTP server failed to bind 0.0.0.0:{port}: {exc}") from exc t = threading.Thread(target=server.serve_forever, daemon=True) t.start() - log(f"HTTP server listening on http://0.0.0.0:{port} (endpoints: /usage, /status)") + log( + f"HTTP server listening on http://0.0.0.0:{port} " + "(endpoints: /usage, /status, /health)" + ) # Advertise via mDNS if zeroconf is available. We use AsyncZeroconf and # async_register_service because calling the sync register_service from diff --git a/install.sh b/install.sh index 40f5814..76ef229 100755 --- a/install.sh +++ b/install.sh @@ -64,7 +64,7 @@ PLIST echo "Installed LaunchAgent: $PLIST_FILE" echo "Logs: $HOME/Library/Logs/codexmeter.log" - SCRAPE_SCRIPT="$SCRIPT_DIR/scripts/claude-plan-usage-cache.py" + SCRAPE_SCRIPT="$SCRIPT_DIR/scripts/claude-plan-usage-firefox.py" SCRAPE_PLIST="$PLIST_DIR/com.justin.codexmeter.claude-plan-scrape.plist" if [ -x "$SCRAPE_SCRIPT" ]; then cat > "$SCRAPE_PLIST" < Path: + if PROFILE_OVERRIDE: + p = Path(PROFILE_OVERRIDE) + if not p.is_absolute(): + p = Path.home() / "Library/Application Support/Firefox/Profiles" / p + if (p / "cookies.sqlite").exists(): + return p + raise SystemExit(f"CODEXMETER_FIREFOX_PROFILE points at {p}, no cookies.sqlite there") + base = Path.home() / "Library/Application Support/Firefox/Profiles" + if not base.exists(): + raise SystemExit(f"Firefox profiles directory not found: {base}") + candidates = sorted( + (p for p in base.iterdir() if (p / "cookies.sqlite").exists()), + key=lambda p: p.stat().st_mtime, + reverse=True, + ) + if not candidates: + raise SystemExit("No Firefox profile with cookies.sqlite found") + # Prefer the *-release profile if multiple exist + for c in candidates: + if c.name.endswith(".default-release"): + return c + return candidates[0] + + +def read_claude_cookies() -> dict[str, str]: + profile = firefox_profile_dir() + src = profile / "cookies.sqlite" + with tempfile.NamedTemporaryFile(suffix=".sqlite", delete=False) as tmp: + tmp_path = Path(tmp.name) + try: + shutil.copy(src, tmp_path) + conn = sqlite3.connect(tmp_path) + cookies: dict[str, str] = {} + for name, value in conn.execute( + "SELECT name, value FROM moz_cookies WHERE host LIKE '%claude.ai%'" + ): + cookies[name] = value + conn.close() + return cookies + finally: + try: + tmp_path.unlink() + except OSError: + pass + + +def claude_org_id() -> str: + env = os.getenv("CODEXMETER_CLAUDE_ORG_ID") + if env: + return env + profile = Path.home() / ".claude.json" + if profile.exists(): + try: + data = json.loads(profile.read_text(errors="replace")) + org = (data.get("oauthAccount") or {}).get("organizationUuid") + if org: + return str(org) + except (OSError, json.JSONDecodeError): + pass + raise SystemExit( + "Set CODEXMETER_CLAUDE_ORG_ID or log in to Claude Code so " + "~/.claude.json has oauthAccount.organizationUuid." + ) + + +def fetch_usage(cookies: dict[str, str]) -> dict: + org = claude_org_id() + url = f"https://claude.ai/api/organizations/{org}/usage" + cookie_str = "; ".join(f"{k}={v}" for k, v in cookies.items()) + req = urllib.request.Request( + url, + headers={ + "Cookie": cookie_str, + "User-Agent": DEFAULT_UA, + "Accept": "application/json", + "Referer": "https://claude.ai/settings/usage", + "Accept-Language": "en-US,en;q=0.5", + }, + ) + with urllib.request.urlopen(req, timeout=12) as resp: + return json.loads(resp.read().decode("utf-8")) + + +def minutes_until_iso(iso: str | None) -> int: + if not iso: + return -1 + try: + when = datetime.fromisoformat(iso.replace("Z", "+00:00")) + except ValueError: + return -1 + return max(0, round((when - datetime.now(timezone.utc)).total_seconds() / 60)) + + +def normalize(data: dict) -> dict: + five = data.get("five_hour") if isinstance(data.get("five_hour"), dict) else {} + week = data.get("seven_day") if isinstance(data.get("seven_day"), dict) else {} + return { + "source": "claude_api_firefox_cookies", + "captured_at": datetime.now().astimezone().isoformat(timespec="seconds"), + "session": { + "used_pct": int(round(float(five.get("utilization") or 0))), + "reset_mins": minutes_until_iso(five.get("resets_at")), + }, + "weekly": { + "used_pct": int(round(float(week.get("utilization") or 0))), + "reset_mins": minutes_until_iso(week.get("resets_at")), + }, + } + + +def write_atomic(path: Path, payload: dict) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_suffix(path.suffix + ".tmp") + tmp.write_text(json.dumps(payload, separators=(",", ":"))) + tmp.replace(path) + + +def write_health(ok: bool, message: str, payload: dict | None = None) -> None: + health = { + "ok": ok, + "checked_at": datetime.now().astimezone().isoformat(timespec="seconds"), + "message": message, + } + if payload: + health["last_payload"] = payload + try: + write_atomic(HEALTH_PATH, health) + except OSError: + pass + + +def cmd_login() -> int: + import subprocess + + subprocess.run(["open", "-a", "Firefox", "https://claude.ai/login"], check=False) + print( + "Opened Firefox to claude.ai/login. After you sign in, run " + f"{sys.argv[0]} scrape to verify." + ) + return 0 + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument( + "command", nargs="?", default="scrape", choices=["scrape", "print", "login"] + ) + args = ap.parse_args() + + if args.command == "login": + return cmd_login() + + try: + cookies = read_claude_cookies() + except Exception as exc: + msg = f"cookie read failed: {type(exc).__name__}: {exc}" + write_health(False, msg) + raise SystemExit(msg) from exc + + if "sessionKey" not in cookies: + msg = "Firefox has no claude.ai sessionKey — log in via 'login' command" + write_health(False, msg) + raise SystemExit(msg) + + try: + data = fetch_usage(cookies) + except urllib.error.HTTPError as exc: + body = exc.read().decode("utf-8", errors="replace")[:300] + msg = f"claude.ai/api HTTP {exc.code}: {body}" + write_health(False, msg) + raise SystemExit(msg) from exc + except Exception as exc: + msg = f"claude.ai/api request failed: {type(exc).__name__}: {exc}" + write_health(False, msg) + raise SystemExit(msg) from exc + + payload = normalize(data) + if args.command == "scrape": + write_atomic(CACHE_PATH, payload) + write_health(True, "ok", payload) + print(json.dumps(payload, indent=2)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main())