From f979e1d090b3cc76beeae87c487a4dc1979589f6 Mon Sep 17 00:00:00 2001 From: allenter Date: Sun, 2 Aug 2026 08:50:02 +0800 Subject: [PATCH 1/3] feat(memory): add persistent vector memory vault alongside consensus.md Layer a zero-dependency, pure-Python vector memory store (memories/vault/) on top of the existing single-file consensus baton: - scripts/core/memory_vault.py: chunk consensus + docs, index into memories/vault/index.json (TF-IDF char n-grams + cosine similarity, no external deps), and semantic search with -top-k / min-score. Backend swappable: replace _embed_chunk() to plug in ChromaDB or a model embedding. - auto-loop.sh: each cycle auto-retrieves the top relevant historical blocks (keyed off Next Action) and injects them into the prompt as '## Highly-relevant past memory'; after a successful/soft-timeout cycle it indexes the latest consensus + docs into the vault. - .gitignore: ignore memories/vault/* runtime data. - Docs updated: README(EN/ZH), CLAUDE.md, PROMPT.md, INDEX.md. consensus.md remains the authoritative running-state baton; the vault adds long-term recall of decisions/context that consensus collapses. --- .gitignore | 2 + CLAUDE.md | 5 + INDEX.md | 3 +- PROMPT.md | 4 + README-ZH.md | 5 +- README.md | 5 +- scripts/core/auto-loop.sh | 45 ++++- scripts/core/memory_vault.py | 380 +++++++++++++++++++++++++++++++++++ 8 files changed, 443 insertions(+), 6 deletions(-) create mode 100755 scripts/core/memory_vault.py diff --git a/.gitignore b/.gitignore index 4ade6b0d..7af4e0f9 100644 --- a/.gitignore +++ b/.gitignore @@ -186,3 +186,5 @@ docs/*/* # Ignore all memories (keep folder marker only) memories/* !memories/.gitkeep +memories/vault/* +!memories/vault/.gitkeep diff --git a/CLAUDE.md b/CLAUDE.md index f2ef2835..d4b77a08 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -181,6 +181,11 @@ All skills are under `.claude/skills/`. Any agent can use any skill when relevan ## Consensus Memory - `memories/consensus.md` - cross-cycle baton; must be updated before cycle end +- `memories/vault/` - persistent vector memory index (`index.json`), auto-built by + `scripts/core/memory_vault.py` from consensus snapshots + `docs/` outputs. Every + cycle auto-loop injects the top semantically-relevant blocks into the prompt so + agents can recall historical decisions long collapsed out of consensus.md. It is + generated automatically; no manual upkeep needed. - `docs//` - agent outputs - `projects/` - all created projects diff --git a/INDEX.md b/INDEX.md index 0ef96caa..d1ef7acc 100644 --- a/INDEX.md +++ b/INDEX.md @@ -57,7 +57,8 @@ | 守护 | `scripts/wsl/uninstall-wsl-daemon.sh` | 卸载 WSL daemon | | 守护 | `scripts/wsl/wsl-daemon-status.sh` | 查询 WSL daemon 状态 | | 守护 | `scripts/macos/install-daemon.sh` | macOS launchd 安装/卸载 | -| 核心 | `scripts/core/auto-loop.sh` | 主循环执行、熔断、日志、共识更新 | +| 核心 | `scripts/core/auto-loop.sh` | 主循环执行、熔断、日志、共识更新、向量记忆召回/沉淀 | +| 记忆 | `scripts/core/memory_vault.py` | 向量记忆库：`index`(沉淀共识+docs)、`search`(语义召回)、`status`、`clear`。纯 Python 零依赖，TF-IDF+余弦 | | 核心 | `scripts/core/monitor.sh` | 核心状态/日志输出 | | 核心 | `scripts/core/stop-loop.sh` | 核心停止/暂停/恢复控制 | diff --git a/PROMPT.md b/PROMPT.md index 02549f76..adc22749 100644 --- a/PROMPT.md +++ b/PROMPT.md @@ -8,6 +8,10 @@ 当前共识已预加载在本 prompt 末尾。如果没有，读 `memories/consensus.md`。 +> **长程记忆（向量库）**：本轮 prompt 还会附带一段从 `memories/vault/` 语义检索出的"相关历史记忆"（`## Highly-relevant past memory`）。它补充了 consensus 里被折叠掉的历史上下文——包括更早的决策、docs 里的方案细节。当前共识 `.md` 是最新/最权威的，检索出的历史仅作为背景参考，两者冲突时以共识为准。 + +每轮 cycle 结束后，脚本会把最新的 consensus 快照与 `docs/` 产出分块向量化并存入 `memories/vault/index.json`，供后续周期语义召回。这是自动进行的，agent 无需手动维护 vault。 + ### 2. 决策 - 有明确 Next Action → 执行它 diff --git a/README-ZH.md b/README-ZH.md index 536daf4e..6eb6889b 100644 --- a/README-ZH.md +++ b/README-ZH.md @@ -41,7 +41,7 @@ daemon (launchd / systemd --user, 崩溃自重启) └── sleep → 下一轮 ``` -每个周期是一次独立的 CLI 调用。`memories/consensus.md` 是唯一的跨周期状态——类似接力赛传棒。 +每个周期是一次独立的 CLI 调用。`memories/consensus.md` 是跨周期运行状态的接力棒。另外系统会自动把 consensus 快照与 `docs/` 产出向量化沉淀到长期记忆库 `memories/vault/`，并在每个周期语义注入相关历史供长期召回（见第 2 层）。 ## 你该看哪一节（按平台） @@ -188,7 +188,8 @@ Auto-Company 并非简单调用 LLM API，而是一个高度解耦的 **多智 ### 第 2 层：编排与状态控制层 (Orchestration & State Machine) * **永续主循环 (The Auto-Loop)**：通过 `scripts/core/auto-loop.sh` 控制的执行循环，让 AI 摆脱“单次对话”，实现 24/7 永续运行。 -* **轻量级状态机 (Consensus Memory)**：放弃复杂的向量数据库或内存管理，将跨周期的上下文压缩为一个 Markdown 文件：`memories/consensus.md`。每次循环开始前读取，结束前必须重写，作为整个系统的“接力棒”。 +* **轻量级状态机 (Consensus Memory)**：将跨周期的运行上下文压缩为一个 Markdown 文件：`memories/consensus.md`。每次循环开始前读取，结束前必须重写，作为整个系统的“接力棒”。 +* **持久化向量记忆 (Long-term Recall)**：在接力棒之上再加一层。`scripts/core/memory_vault.py` 把 consensus 快照与 `docs/` 产出向量化沉淀到 `memories/vault/index.json`（纯 Python 的 TF-IDF + 余弦相似度，零外部依赖、近零成本）。每个新周期自动语义检索最相关的历史知识块注入 prompt，使 consensus 中已被折叠的历史决策与细节依然可召回。存储后端可替换：改 `_embed_chunk()` 即可接上 ChromaDB 或模型 embedding。 * **高可用容错机制 (Resilience & Self-Healing)**：内置熔断器 (连续错误触发冷却)、限流退避 (429 报错自动休眠) 和沙箱重置 (未成功输出有效共识时自动回滚)。 ### 第 1 层：基础设施与执行引擎层 (Execution Engine & Infrastructure) diff --git a/README.md b/README.md index 7169217b..6ded28cf 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ daemon (launchd / systemd --user, auto-restart on crash) └── sleep -> next cycle ``` -Each cycle is an independent CLI call. `memories/consensus.md` is the only cross-cycle state. +Each cycle is an independent CLI call. `memories/consensus.md` is the cross-cycle running-state baton. A supplemental persistent **vector memory** (`memories/vault/`) is auto-built from consensus snapshots + `docs/` and injected semantically each cycle for long-term recall (see Layer 2). ## Where To Start (By Platform) @@ -187,7 +187,8 @@ Auto-Company is not a simple LLM API wrapper, but a highly decoupled **Multi-Age ### Layer 2: Orchestration & State Machine * **The Auto-Loop**: The execution loop controlled by `scripts/core/auto-loop.sh` frees the AI from "single-turn conversations", enabling 24/7 continuous operation. -* **Lightweight State Machine (Consensus Memory)**: Forgoes complex vector databases or memory management, compressing cross-cycle context into a single Markdown file: `memories/consensus.md`. Read before every cycle and rewritten before it ends, acting as the system's "baton". +* **Lightweight State Machine (Consensus Memory)**: Compresses cross-cycle running context into a single Markdown file: `memories/consensus.md` (read before every cycle, rewritten before it ends — the "baton"). +* **Persistent Vector Memory (Long-term Recall)**: Layered on top of the baton. `scripts/core/memory_vault.py` indexes consensus snapshots + `docs/` outputs into `memories/vault/index.json` (pure-Python TF-IDF + cosine, zero external deps, ~zero cost). Each new cycle auto-retrieves the top semantically-relevant historical blocks and injects them into the prompt, so past decisions and details that consensus.md has collapsed away remain recallable. Swappable backend: replace `_embed_chunk()` to plug in ChromaDB or a model embedding. * **Resilience & Self-Healing**: Built-in circuit breakers (cooldown triggered by consecutive errors), rate-limit backoff (auto-sleep on 429 errors), and sandbox reset (auto-rollback if a valid consensus is not output). ### Layer 1: Execution Engine & Infrastructure diff --git a/scripts/core/auto-loop.sh b/scripts/core/auto-loop.sh index dea03df1..52c7a900 100644 --- a/scripts/core/auto-loop.sh +++ b/scripts/core/auto-loop.sh @@ -383,6 +383,42 @@ resolve_engine_bin() { esac } +# === Vector Memory Vault helpers === +# Long-term semantic memory, layered on top of the single-file consensus baton. +# Script: scripts/core/memory_vault.py (pure-python, zero external deps). + +VAULT_PY="$PROJECT_DIR/scripts/core/memory_vault.py" + +# Retrieve the top-K memory blocks most relevant to the next task. +# Prints nothing on failure. Returns "" if no python or no vault. +vault_retrieve_prompt() { + local query="$1" + [ -f "$VAULT_PY" ] || return 0 + [ -n "$query" ] || query="$( + awk '/^## Next Action/{getline; print}' "$CONSENSUS_FILE" 2>/dev/null | + head -n1 | tr -d '\r' + )" + [ -z "$query" ] && return 0 + command -v python3 >/dev/null 2>&1 || return 0 + python3 "$VAULT_PY" --vault "$PROJECT_DIR/memories/vault" search "$query" \ + --top-k "${VAULT_TOP_K:-5}" --min-score "${VAULT_MIN_SCORE:-0.05}" 2>/dev/null +} + +# Index current consensus + per-role docs into the vault. +# Non-fatal: failures only log a guard line, never abort the loop. +vault_index_cycle() { + [ -f "$VAULT_PY" ] || return 0 + command -v python3 >/dev/null 2>&1 || return 0 + local out + out=$(python3 "$VAULT_PY" --vault "$PROJECT_DIR/memories/vault" index \ + --consensus "$CONSENSUS_FILE" \ + --docs-dir "$PROJECT_DIR/docs" \ + --max-chunks "${VAULT_MAX_CHUNKS:-5000}" 2>&1) + if [ -n "$out" ]; then + log_cycle "$loop_count" "VAULT" "$out" + fi +} + run_codex_cycle() { local prompt="$1" local output_file timeout_flag message_file @@ -559,7 +595,7 @@ extract_cycle_metadata() { # === Setup === -mkdir -p "$LOG_DIR" "$PROJECT_DIR/memories" +mkdir -p "$LOG_DIR" "$PROJECT_DIR/memories" "$PROJECT_DIR/memories/vault" # Clean up stale stop file from previous run rm -f "$PROJECT_DIR/.auto-loop-stop" @@ -650,6 +686,10 @@ while true; do # Build prompt with consensus pre-injected PROMPT=$(cat "$PROMPT_FILE") CONSENSUS=$(cat "$CONSENSUS_FILE" 2>/dev/null || echo "No consensus file found. This is the very first cycle.") + + # Semantic retrieval of relevant long-term memory (best-effort, non-fatal) + MEMORY_BLOCK=$(vault_retrieve_prompt "") + FULL_PROMPT="$PROMPT --- @@ -667,6 +707,7 @@ while true; do ## Current Consensus (pre-loaded, do NOT re-read this file) $CONSENSUS +$([ -n "$MEMORY_BLOCK" ] && printf '\n\n---\n## Highly-relevant past memory (from vector vault; use as context, trust current consensus)\n\n%s\n' "$MEMORY_BLOCK") --- @@ -705,12 +746,14 @@ This is Cycle #$loop_count. Act decisively." log_cycle "$loop_count" "SUMMARY" "$(echo "$RESULT_TEXT" | head -c 300)" fi error_count=0 + vault_index_cycle elif [ -z "$cycle_failed_reason" ]; then log_cycle "$loop_count" "OK" "Completed (cost: ${CYCLE_COST}, subtype: ${CYCLE_SUBTYPE})" if [ -n "$RESULT_TEXT" ]; then log_cycle "$loop_count" "SUMMARY" "$(echo "$RESULT_TEXT" | head -c 300)" fi error_count=0 + vault_index_cycle else error_count=$((error_count + 1)) log_cycle "$loop_count" "FAIL" "$cycle_failed_reason (cost: ${CYCLE_COST}, subtype: ${CYCLE_SUBTYPE}, errors: $error_count/$MAX_CONSECUTIVE_ERRORS)" diff --git a/scripts/core/memory_vault.py b/scripts/core/memory_vault.py new file mode 100755 index 00000000..00ee640c --- /dev/null +++ b/scripts/core/memory_vault.py @@ -0,0 +1,380 @@ +#!/usr/bin/env python3 +""" +Auto Company — Persistent Vector Memory Vault +============================================== + +A lightweight, dependency-free long-term memory layer that RUNS ALONGSIDE the +existing `memories/consensus.md` baton (the single-file relay). It adds +semantic retrieval on top of the full history that consensus.md collapses away, +so agents can recall relevant past decisions, project states, and docs without +loading everything into the prompt. + +WHY keep consensus.md? + consensus.md stays the authoritative *running state + next-action baton* that + auto-loop.sh reads/writes every cycle. The vault is a *supplemental read-only + index* built from the changelog of consensus snapshots + per-role docs. + +VECTOR DESIGN (zero external deps — "boring technology first"): + - Each memory chunk is tokenized into character n-grams (covers ENG+中文) + - Term frequency vectors are stored as {term: weight} + - Retrieval = cosine similarity (sparse dot product over a shared vocabulary) + - No numpy / no embedding model / no network required. Pure stdlib. + +To swap in a real vector DB (ChromaDB, etc.) or a model embedding: + - Replace `_embed_chunk(text)` to return a numeric vector + - Replace backend read/write in `vault_read` / `vault_write` + JSON format below is kept backend-agnostic (`embeddings` can be dicts of + sparse terms OR dense lists; both are handled on read). + +Usage: + python3 memory_vault.py index [--consensus FILE] [--docs-dir DIR] [--vault DIR] + Scan consensus + docs, chunk, embed, upsert into vault. + python3 memory_vault.py search QUERY [--top-k N] [--vault DIR] [--min-score F] + Semantic search over the vault; prints markdown snippet block. + python3 memory_vault.py status [--vault DIR] + Print vault stats (chunk count, memory size, distinct terms). + python3 memory_vault.py clear [--vault DIR] + Wipe the vault (destructive; asks unless --force). +""" + +import argparse +import json +import math +import os +import re +import sys +from datetime import datetime, timezone + +# ---------------------------------------------------------------- config ---- + +DEFAULT_VAULT = os.path.join(os.path.dirname(os.path.abspath(__file__)), + "..", "..", "memories", "vault") +INDEX_FILE = "index.json" +DOCS_DEFAULT = os.path.join(os.path.dirname(os.path.abspath(__file__)), + "..", "..", "docs") + +# Chunking: target ~N tokens worth of text per memory unit (split on blank lines). +CHUNK_MIN_CHARS = 120 +CHUNK_MAX_CHARS = 1400 +TOP_K_DEFAULT = 5 +MIN_SCORE_DEFAULT = 0.05 + +# Character n-gram feature extraction (covers English + CJK without word-seg). +NGRAM_MIN = 2 +NGRAM_MAX = 3 +STOP_CHARS = set(" \t\r\n\"'`.,;:!?()[]{}<>-_=+|/\\@#$%^&*~·。，；：！？、（）《》【】—…•") +CJK_RE = re.compile(r'[\u3040-\u30ff\u4e00-\u9fff\uac00-\ud7af]') + + +# ------------------------------------------------------------- features ----- + +def _ngrams(text: str) -> dict: + """Return {term: normalized-weight} for overlapping char n-grams.""" + # Normalize whitespace runs to single space so chunk boundaries are stable. + text = re.sub(r"[\r\n\t]+", " ", text) + text = text.strip() + if not text: + return {} + low = text.lower() + counts: dict = {} + n_chars = len(low) + for n in range(NGRAM_MIN, min(NGRAM_MAX, n_chars) + 1): + for i in range(n_chars - n + 1): + gram = low[i:i + n] + if any(c in STOP_CHARS for c in gram): + continue + # Keep CJK grams, else require at least one alpha char. + if not CJK_RE.search(gram) and not any(c.isalpha() for c in gram): + continue + counts[gram] = counts.get(gram, 0) + 1 + # Normalize to term frequency (raw counts up to sqrt): dampens long doc bias. + tf = {g: 1.0 + math.log(c) for g, c in counts.items()} + # L2 normalize. + norm = math.sqrt(sum(v * v for v in tf.values())) or 1.0 + return {g: v / norm for g, v in tf.items()} + + +def _embed_chunk(text: str): + """Sparse embedding. Replace with a real embedding call to upgrade.""" + return _ngrams(text) + + +# --------------------------------------------------------------- vault io ---- + +def _vault_index_path(vault_dir: str) -> str: + return os.path.join(vault_dir, INDEX_FILE) + + +def vault_read(vault_dir: str) -> dict: + path = _vault_index_path(vault_dir) + if not os.path.exists(path): + return {"version": 1, "chunks": []} + try: + with open(path, "r", encoding="utf-8") as fh: + return json.load(fh) + except (OSError, json.JSONDecodeError): + # Corrupt index -> start fresh rather than crash a cycle. + return {"version": 1, "chunks": []} + + +def vault_write(vault_dir: str, index: dict) -> None: + os.makedirs(vault_dir, exist_ok=True) + tmp = _vault_index_path(vault_dir) + ".tmp" + with open(tmp, "w", encoding="utf-8") as fh: + json.dump(index, fh, ensure_ascii=False) + os.replace(tmp, _vault_index_path(vault_dir)) + + +# ----------------------------------------------------------- chunking ------- + +def _iter_markdown_blocks(text: str): + """Split markdown into semantically-meaningful blocks on blank lines.""" + blocks, cur = [], [] + for line in text.splitlines(): + line = line.rstrip() + if not line.strip() and cur: + blocks.append("\n".join(cur)) + cur = [] + else: + cur.append(line) + if cur: + blocks.append("\n".join(cur)) + # Merge tiny blocks and re-split oversized blocks. + merged: list = [] + for b in blocks: + b = b.strip() + if not b: + continue + if merged and len(merged[-1]) < CHUNK_MIN_CHARS and \ + len(merged[-1]) + len(b) <= CHUNK_MAX_CHARS: + merged[-1] = merged[-1] + "\n\n" + b + else: + merged.append(b) + result: list = [] + for b in merged: + if len(b) <= CHUNK_MAX_CHARS: + result.append(b) + else: + # Hard split oversized block on nearest sentence boundary. + start, b_len = 0, len(b) + while start < b_len: + end = min(start + CHUNK_MAX_CHARS, b_len) + if end < b_len: + cut = b.rfind("。", start, end) + if cut <= start + CHUNK_MIN_CHARS: + cut = b.rfind("\n", start, end) + if cut <= start + CHUNK_MIN_CHARS: + cut = end + end = cut + result.append(b[start:end].strip()) + start = end + return [r for r in result if len(r) >= CHUNK_MIN_CHARS // 2] + + +def _generate_chunks(consensus_text: str, docs_map: dict): + """Yield (source, kind, text) tuples. docs_map: {relpath: text}.""" + if consensus_text.strip(): + yield "consensus.md", "consensus", consensus_text + for rel, text in docs_map.items(): + if not text.strip(): + continue + for block in _iter_markdown_blocks(text): + yield rel, "docs", block + + +# -------------------------------------------------------------- ingest ------- + +def cmd_index(args) -> int: + consensus_text = "" + if args.consensus and os.path.exists(args.consensus): + with open(args.consensus, "r", encoding="utf-8") as fh: + consensus_text = fh.read() + + docs_map = {} + docs_dir = args.docs_dir or DOCS_DEFAULT + if os.path.isdir(docs_dir): + for root, _dirs, files in os.walk(docs_dir): + for fn in sorted(files): + if not fn.endswith((".md", ".txt")): + continue + full = os.path.join(root, fn) + rel = os.path.relpath(full, docs_dir) + try: + with open(full, "r", encoding="utf-8") as fh: + docs_map[os.path.join("docs", rel)] = fh.read() + except (OSError, UnicodeDecodeError): + continue + + vault = vault_read(args.vault) + existing = {c["id"] for c in vault["chunks"]} + now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") + + added = 0 + new_chunks = [] + for source, kind, text in _generate_chunks(consensus_text, docs_map): + cid = _stable_id(source, text) + if cid in existing: + continue + emb = _embed_chunk(text) + if not emb: + continue + new_chunks.append({ + "id": cid, + "source": source, + "kind": kind, + "text": text, + "page": now, + "embeddings": emb, + }) + added += 1 + + # Cap vault growth: keep the most recent N chunks (append new, drop oldest). + max_chunks = int(args.max_chunks or 5000) + vault["chunks"].extend(new_chunks) + if len(vault["chunks"]) > max_chunks: + vault["chunks"] = vault["chunks"][-max_chunks:] + + vault["last_indexed"] = now + vault["count"] = len(vault["chunks"]) + vault_write(args.vault, vault) + + print(f"vault: +{added} added, {len(vault['chunks'])} total chunks " + f"({_size_mb(args.vault)} MB), sources: {len(set(c['source'] for c in new_chunks))}") + return 0 + + +# ------------------------------------------------------------ retrieval ------ + +def _stable_id(source: str, text: str) -> str: + import hashlib + return hashlib.sha1((source + "\x00" + text).encode("utf-8")).hexdigest()[:16] + + +def _cosine(a: dict, b: dict) -> float: + if not a or not b: + return 0.0 + if len(a) > len(b): + a, b = b, a + dot = 0.0 + for term, w in a.items(): + wb = b.get(term) + if wb: + dot += w * wb + return dot # both already L2-normalized + + +def _normalize_embedding(emb): + """Accept sparse dict {term: weight} OR dense list of floats.""" + if isinstance(emb, list): + norm = math.sqrt(sum(x * x for x in emb)) or 1.0 + return {f"d{i}": v / norm for i, v in enumerate(emb)} + if isinstance(emb, dict): + return emb + return {} + + +def cmd_search(args) -> int: + vault = vault_read(args.vault) + chunks = vault.get("chunks", []) + query_emb = _ngrams(args.query) + if not query_emb or not chunks: + print("_no_vault_hit_") + return 0 + + top_k = int(args.top_k or TOP_K_DEFAULT) + min_score = float(args.min_score or MIN_SCORE_DEFAULT) + + scored = [] + for c in chunks: + emb = _normalize_embedding(c.get("embeddings", {})) + if not emb: + continue + s = _cosine(query_emb, emb) + if s >= min_score: + scored.append((s, c)) + scored.sort(key=lambda x: x[0], reverse=True) + scored = scored[:top_k] + + if not scored: + print("_no_vault_hit_") + return 0 + + # Emit a markdown snippet block that auto-loop.sh injects into the prompt. + print(f"**[Retrieved {len(scored)} relevant memory block(s) from vault]**") + for i, (score, c) in enumerate(scored, 1): + text = c["text"].strip().replace("\n", " ") + text = text if len(text) <= 500 else text[:497] + "..." + print(f"---\n[{i}] score={score:.3f} src=`{c['source']}` (idx {c['id']})\n{text}") + return 0 + + +# ------------------------------------------------------------- utilities ----- + +def _size_mb(vault_dir: str) -> float: + path = _vault_index_path(vault_dir) + try: + return round(os.path.getsize(path) / (1024 * 1024), 2) + except OSError: + return 0.0 + + +def cmd_status(args) -> int: + vault = vault_read(args.vault) + chunks = vault.get("chunks", []) + terms = set() + for c in chunks: + emb = _normalize_embedding(c.get("embeddings", {})) + terms.update(emb.keys()) + print(f"vault_dir: {os.path.abspath(args.vault)}") + print(f"chunks: {len(chunks)}") + print(f"distinct_terms: {len(terms)}") + print(f"last_indexed: {vault.get('last_indexed', 'never')}") + print(f"size_mb: {_size_mb(args.vault)}") + return 0 + + +def cmd_clear(args) -> int: + path = _vault_index_path(args.vault) + if not args.force: + sys.stderr.write("Refusing to clear without --force.\n") + return 1 + if os.path.exists(path): + os.remove(path) + print(f"vault cleared: {os.path.abspath(path)}") + return 0 + + +# ------------------------------------------------------------------ main ----- + +def main(argv=None) -> int: + ap = argparse.ArgumentParser(description="Auto Company vector memory vault") + ap.add_argument("--vault", default=DEFAULT_VAULT) + sub = ap.add_subparsers(dest="cmd", required=True) + + p_index = sub.add_parser("index", help="Scan consensus + docs into vault") + p_index.add_argument("--consensus", default=os.path.join( + os.path.dirname(os.path.abspath(__file__)), "..", "..", "memories", "consensus.md")) + p_index.add_argument("--docs-dir", default=DOCS_DEFAULT) + p_index.add_argument("--max-chunks", default="5000") + p_index.set_defaults(func=cmd_index) + + p_search = sub.add_parser("search", help="Semantic search over the vault") + p_search.add_argument("query") + p_search.add_argument("--top-k", default=str(TOP_K_DEFAULT)) + p_search.add_argument("--min-score", default=str(MIN_SCORE_DEFAULT)) + p_search.set_defaults(func=cmd_search) + + p_status = sub.add_parser("status") + p_status.set_defaults(func=cmd_status) + + p_clear = sub.add_parser("clear") + p_clear.add_argument("--force", action="store_true") + p_clear.set_defaults(func=cmd_clear) + + args = ap.parse_args(argv) + args.vault = os.path.abspath(args.vault) + return args.func(args) + + +if __name__ == "__main__": + sys.exit(main()) From bf1f56a3c1e9f119b476038f83fa5901ace8d627 Mon Sep 17 00:00:00 2001 From: allenter Date: Sun, 2 Aug 2026 09:08:49 +0800 Subject: [PATCH 2/3] feat(dashboard): visualize vector memory vault Add a Memory Vault panel to the control deck: - server.py: new GET /api/vault endpoint. Reads memories/vault/index.json directly (no subprocess) and returns stats (chunk/source/term counts, size, last-indexed), per-source breakdown, latest entries, plus optional in-process cosine semantic search via ?q=. - app.js: fetchVault() renderer with stat cards, a canvas bar chart of chunks per source, latest-entry list, and a live semantic-search box (Enter or button) showing scored hits with source tags. - index.html: Memory Vault section between Consensus and Recent Log. - styles.css: dark-theme styles for stats, canvas, entries, tags. Pure stdlib on the server side; canvas drawn client-side. --- dashboard/app.js | 148 +++++++++++++++++++++++++++++++++++++++++++ dashboard/index.html | 23 +++++++ dashboard/server.py | 137 ++++++++++++++++++++++++++++++++++++++- dashboard/styles.css | 144 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 451 insertions(+), 1 deletion(-) diff --git a/dashboard/app.js b/dashboard/app.js index fcf4f4d4..33fa3947 100644 --- a/dashboard/app.js +++ b/dashboard/app.js @@ -23,6 +23,15 @@ const els = { logText: document.getElementById("logText"), rawText: document.getElementById("rawText"), + vaultSubtitle: document.getElementById("vaultSubtitle"), + vaultStats: document.getElementById("vaultStats"), + vaultChart: document.getElementById("vaultChart"), + vaultLatest: document.getElementById("vaultLatest"), + vaultResults: document.getElementById("vaultResults"), + vaultQuery: document.getElementById("vaultQuery"), + btnVaultSearch: document.getElementById("btnVaultSearch"), + btnVaultRefresh: document.getElementById("btnVaultRefresh"), + btnRefresh: document.getElementById("btnRefresh"), btnStart: document.getElementById("btnStart"), btnStop: document.getElementById("btnStop"), @@ -236,6 +245,130 @@ async function fetchStatus() { els.latency.textContent = `Roundtrip: ${elapsed}ms`; } +function drawVaultChart(bySource) { + const canvas = els.vaultChart; + if (!canvas || !bySource) return; + const ctx = canvas.getContext("2d"); + const { width: W, height: H } = canvas; + ctx.clearRect(0, 0, W, H); + + const entries = Object.entries(bySource).sort((a, b) => b[1] - a[1]).slice(0, 8); + if (entries.length === 0) { + ctx.fillStyle = "#5b6b7a"; + ctx.font = "14px 'Rajdhani', sans-serif"; + ctx.fillText("no chunks indexed yet", 16, H / 2); + return; + } + + const max = Math.max(...entries.map((e) => e[1]), 1); + const padL = 14, padT = 14; + const rowH = (H - padT * 2) / entries.length; + const barMax = W - padL - 16; + + // subtle axes + ctx.strokeStyle = "rgba(255,255,255,0.06)"; + ctx.lineWidth = 1; + ctx.beginPath(); + ctx.moveTo(padL, padT - 4); + ctx.lineTo(padL, H - padT); + ctx.moveTo(padL, H - padT); + ctx.lineTo(W - 8, H - padT); + ctx.stroke(); + + entries.forEach(([src, count], i) => { + const y = padT + i * rowH + rowH / 2; + const len = count / max * barMax; + const grad = ctx.createLinearGradient(padL, 0, padL + len, 0); + grad.addColorStop(0, "#5ee7df"); + grad.addColorStop(1, "#7c5cff"); + ctx.fillStyle = grad; + const r = 3; + ctx.beginPath(); + ctx.roundRect(padL, y - 7, Math.max(len, 2), 14, r); + ctx.fill(); + + // label + ctx.fillStyle = "rgba(220,230,240,0.85)"; + ctx.font = "600 12px 'Rajdhani', sans-serif"; + let label = src.replace(/\.md$/i, ""); + if (label.length > 18) label = "…" + label.slice(-17); + ctx.fillText(label, padL + Math.max(len, 2) + 8, y + 4); + ctx.fillStyle = "#5ee7df"; + ctx.font = "700 12px 'Rajdhani', sans-serif"; + ctx.fillText(String(count), W - 10, y + 4); + }); +} + +function truncateMiddle(text, n = 150) { + const t = String(text || "").replace(/\s+/g, " ").trim(); + if (t.length <= n) return t; + return t.slice(0, n) + "…"; +} + +function renderVault(data) { + if (!data.ok || !data.stats) { + els.vaultSubtitle.textContent = "Vault not initialized yet — will appear after the first cycle."; + els.vaultStats.innerHTML = "

Waiting for memories/vault/index.json…

"; + drawVaultChart(null); + els.vaultLatest.innerHTML = ""; + return; + } + + const s = data.stats; + const lastIdx = data.timestamp ? "" : ""; + els.vaultSubtitle.textContent = + `Indexed ${s.lastIndexed || "recently"} · ${s.sizeMb} MB`; + + const cols = [ + ["Chunks", s.chunks], + ["Sources", s.sources], + ["Distinct Terms", s.distinctTerms], + ["Size", `${s.sizeMb} MB`], + ]; + els.vaultStats.innerHTML = cols + .map(([k, v]) => `
${k}
${v}
`) + .join(""); + + drawVaultChart(data.bySource || {}); + + const latest = data.latest || []; + if (latest.length) { + els.vaultLatest.innerHTML = `
Latest entries
` + + latest.map((c) => + `
${escapeHtml(c.source)}` + + `${escapeHtml(truncateMiddle(c.text, 110))}
` + ).join(""); + } else { + els.vaultLatest.innerHTML = ""; + } +} + +function renderVaultSearch(data) { + const hits = (data.search && data.search.hits) || []; + if (!hits.length) { + els.vaultResults.innerHTML = "

No relevant memory found.

"; + els.vaultResults.classList.remove("hidden"); + return; + } + els.vaultResults.innerHTML = + `
Top ${hits.length} semantic matches
` + + hits.map((h) => + `
${h.score.toFixed(3)}` + + `${escapeHtml(h.source)}` + + `${escapeHtml(truncateMiddle(h.text, 140))}
` + ).join(""); + els.vaultResults.classList.remove("hidden"); +} + +async function fetchVault(query) { + const qs = query ? `?q=${encodeURIComponent(query)}&top_k=6` : "?top_k=6"; + const res = await fetch(`/api/vault${qs}`, { cache: "no-store" }); + const data = await res.json(); + renderVault(data); + if (query) renderVaultSearch(data); + return data; +} + async function runAction(action) { const btn = action === "start" ? els.btnStart : els.btnStop; const label = btn.textContent; @@ -280,8 +413,23 @@ els.btnRaw.addEventListener("click", () => { els.autoToggle.addEventListener("change", resetAutoTimer); els.refreshInterval.addEventListener("change", resetAutoTimer); +els.btnVaultRefresh.addEventListener("click", () => { + fetchVault().catch(() => {}); +}); +els.btnVaultSearch.addEventListener("click", () => { + const q = els.vaultQuery.value.trim(); + if (q) fetchVault(q).catch(() => {}); +}); +els.vaultQuery.addEventListener("keydown", (e) => { + if (e.key === "Enter") { + const q = els.vaultQuery.value.trim(); + if (q) fetchVault(q).catch(() => {}); + } +}); + fetchStatus().catch((err) => { const msg = err instanceof Error ? err.message : String(err); els.rawText.textContent = msg; }); +fetchVault().catch(() => {}); resetAutoTimer(); diff --git a/dashboard/index.html b/dashboard/index.html index b6e5ca42..1b8f58e8 100644 --- a/dashboard/index.html +++ b/dashboard/index.html @@ -87,6 +87,29 @@

Consensus Head

+
+
+
+

Memory Vault

+

Persistent vector memory index

+
+
+ + + +
+
+
+
+
+ +
+
+
+ +
+

Recent Log

diff --git a/dashboard/server.py b/dashboard/server.py index 846b03ca..356d5943 100644 --- a/dashboard/server.py +++ b/dashboard/server.py @@ -32,6 +32,8 @@ LOG_FILE = REPO_ROOT / "logs" / "auto-loop.log" STATE_FILE = REPO_ROOT / ".auto-loop-state" CONSENSUS_FILE = REPO_ROOT / "memories" / "consensus.md" +VAULT_FILE = REPO_ROOT / "memories" / "vault" / "index.json" +VAULT_PY = REPO_ROOT / "scripts" / "core" / "memory_vault.py" WINDOWS_HOST = "windows" MACOS_HOST = "macos" @@ -415,7 +417,134 @@ def parse_status_output(raw: str, system_name: str | None = None) -> dict[str, A return profile["parser"](raw) -def gather_status_payload(system_name: str | None = None) -> dict[str, Any]: +def gather_vault_payload(query: str | None = None, top_k: int = 5) -> dict[str, Any]: + """Summarize the vector memory vault + optional semantic search. + + Reads the vault index directly (no subprocess). Stat fields: total chunks, + per-source breakdown, distinct terms, memory footprint, last index time. + If `query` is provided, does an in-process cosine search over the chunks. + """ + payload: dict[str, Any] = { + "timestamp": datetime.now(timezone.utc).isoformat(), + "ok": True, + "exists": VAULT_FILE.exists(), + "stats": None, + "bySource": {}, + "latest": [], + "search": None, + } + + if not VAULT_FILE.exists(): + payload["ok"] = False + return payload + + try: + with open(VAULT_FILE, "r", encoding="utf-8") as fh: + index = json.load(fh) + except (OSError, json.JSONDecodeError): + payload["ok"] = False + return payload + + chunks = index.get("chunks", []) + by_source: dict[str, int] = {} + latest: list[dict[str, Any]] = [] + for c in chunks: + src = c.get("source", "?") + by_source[src] = by_source.get(src, 0) + 1 + latest.append({ + "id": c.get("id", ""), + "source": src, + "kind": c.get("kind", "?"), + "page": c.get("page", ""), + "text": c.get("text", "")[:120], + }) + # Keep "latest" = N most recent by id order (append-ordered in the file). + latest = latest[-5:][::-1] + + try: + size_mb = round(VAULT_FILE.stat().st_size / (1024 * 1024), 3) + except OSError: + size_mb = 0.0 + + payload["stats"] = { + "chunks": len(chunks), + "sources": len(by_source), + "distinctTerms": len({ + t for c in chunks for t in (c.get("embeddings", {}) or {}).keys() + }), + "sizeMb": size_mb, + "lastIndexed": index.get("last_indexed", "never"), + "maxChunks": index.get("count"), + } + payload["bySource"] = by_source + payload["latest"] = latest + + if query and query.strip(): + payload["search"] = memory_search_in_process(chunks, query, top_k=top_k) + + return payload + + +def _cosine(a: dict, b: dict) -> float: + if not a or not b: + return 0.0 + if len(a) > len(b): + a, b = b, a + dot = 0.0 + for term, w in a.items(): + wb = b.get(term) + if wb: + dot += w * wb + return dot + + +def _query_grams(text: str) -> dict: + """Inline copy of memory_vault n-gram embedding for dashboard search.""" + import math as _m + + stop_set = set(" \t\r\n\"'`.,;:!?()[]{}<>-_=+|/\\@#$%^&*~·。，；：！？、（）《》【】—…•") + text = re.sub(r"[\r\n\t]+", " ", text).strip() + if not text: + return {} + low = text.lower() + counts: dict = {} + n_chars = len(low) + for n in range(2, min(3, n_chars) + 1): + for i in range(n_chars - n + 1): + gram = low[i:i + n] + if any(c in stop_set for c in gram): + continue + if not (any(c.isalpha() for c in gram) or re.search( + r"[\u3040-\u30ff\u4e00-\u9fff\uac00-\ud7af]", gram)): + continue + counts[gram] = counts.get(gram, 0) + 1 + tf = {g: 1.0 + _m.log(c) for g, c in counts.items()} + norm = _m.sqrt(sum(v * v for v in tf.values())) or 1.0 + return {g: v / norm for g, v in tf.items()} + + +def memory_search_in_process(chunks: list[dict], query: str, top_k: int = 5) -> dict[str, Any]: + q = _query_grams(query) + if not q or not chunks: + return {"hits": []} + scored = [] + for c in chunks: + emb = c.get("embeddings", {}) + if not emb: + continue + score = _cosine(q, emb) + if score >= 0.05: + scored.append({ + "score": round(score, 3), + "source": c.get("source", "?"), + "id": c.get("id", ""), + "text": c.get("text", "")[:160], + }) + scored.sort(key=lambda x: x["score"], reverse=True) + return {"hits": scored[:top_k]} + + + result = run_status_command(system_name) parsed = parse_status_output(result["output"], system_name) return { @@ -480,6 +609,12 @@ def do_GET(self) -> None: # noqa: N802 if path == "/api/status": self._json(gather_status_payload()) return + if path == "/api/vault": + qs = parse_qs(parsed.query) + query = qs.get("q", [None])[0] + top_k = parse_positive_int(qs.get("top_k", ["5"])[0], default=5) + self._json(gather_vault_payload(query=query, top_k=top_k)) + return if path == "/api/log-tail": qs = parse_qs(parsed.query) lines = parse_positive_int(qs.get("lines", ["180"])[0], default=180) diff --git a/dashboard/styles.css b/dashboard/styles.css index 5c931703..28198d0d 100644 --- a/dashboard/styles.css +++ b/dashboard/styles.css @@ -436,3 +436,147 @@ select { grid-template-columns: 1fr; } } + +/* ===================== Memory Vault ===================== */ + +.panel-head-wrap { + flex-wrap: wrap; + padding-bottom: 8px; +} + +.vault-controls { + display: flex; + align-items: center; + gap: 8px; +} + +.vault-input { + min-width: 220px; + padding: 7px 12px; + border-radius: 10px; + border: 1px solid var(--panel-border); + background: rgba(10, 23, 30, 0.6); + color: var(--text); + font-family: "Rajdhani", sans-serif; + font-weight: 500; + outline: none; + transition: border-color 0.15s ease; +} + +.vault-input::placeholder { + color: var(--muted); +} + +.vault-input:focus { + border-color: var(--cyan); +} + +.vault-row { + display: grid; + grid-template-columns: minmax(160px, 220px) 1fr; + gap: 16px; + padding: 12px 16px 8px; + align-items: stretch; +} + +.vault-stats { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 10px; +} + +.vault-stat { + padding: 12px; + border-radius: 12px; + background: rgba(10, 23, 30, 0.58); + display: flex; + flex-direction: column; + justify-content: center; +} + +.vault-stat dt { + font-size: 11px; + letter-spacing: 0.08em; + text-transform: uppercase; + color: var(--muted); +} + +.vault-stat dd { + margin: 4px 0 0; + font-family: "Rajdhani", sans-serif; + font-size: 22px; + font-weight: 700; + color: var(--cyan); +} + +.vault-bar-wrap { + min-height: 180px; + border-radius: 14px; + background: rgba(10, 23, 30, 0.42); + border: 1px solid rgba(151, 215, 239, 0.12); + padding: 8px; +} + +.vault-chart { + width: 100%; + height: 100%; + display: block; +} + +.vault-latest, +.vault-results { + padding: 4px 16px 12px; +} + +.vault-latest-title { + font-size: 12px; + letter-spacing: 0.1em; + text-transform: uppercase; + color: var(--muted); + margin: 10px 0 8px; +} + +.vault-entry { + display: flex; + align-items: center; + gap: 10px; + padding: 8px 10px; + border-radius: 10px; + background: rgba(10, 23, 30, 0.35); + margin-bottom: 6px; +} + +.vault-entry .mono { + font-size: 12px; + line-height: 1.4; +} + +.tag { + flex: 0 0 auto; + font-family: "Rajdhani", sans-serif; + font-weight: 600; + font-size: 11px; + letter-spacing: 0.05em; + padding: 3px 9px; + border-radius: 999px; + color: #cdeeff; + background: rgba(103, 217, 255, 0.12); + border: 1px solid rgba(103, 217, 255, 0.25); +} + +.tag-score { + color: #c9ffec; + background: rgba(41, 211, 163, 0.12); + border-color: rgba(41, 211, 163, 0.3); +} + +.vault-results.hidden { + display: none; +} + +@media (max-width: 1080px) { + .vault-row { + grid-template-columns: 1fr; + } +} + From 88f301e3d260bd1a63ffc4e846b1eb4bcccacbee Mon Sep 17 00:00:00 2001 From: allenter <15058142+allenter@users.noreply.github.com> Date: Sun, 2 Aug 2026 11:49:20 +0800 Subject: [PATCH 3/3] feat(scripts): add cost monitor with daily summary + threshold alerts Auto-Company burns model quota 24/7 but had zero cost tracking; each cycle's `cost:` only scattered across logs/auto-loop.log. Add a small zero-dependency tool: - cost-monitor.sh: parses per-cycle cost from logs/auto-loop.log and summarizes by calendar day (total/runs/avg/max). Supports --days, --date (incl. "yesterday"), --threshold, and a --check mode that exits non-zero when today's spend exceeds the threshold and appends to logs/cost-alerts.log (cron/launchd friendly). - cost-alert.sh: launchd wrapper running --check that raises a macOS desktop notification on threshold breach and records logs/cost-daily.log. Portability notes baked in: uses match() (macOS awk doesn't set RSTART/RLENGTH for /regex/ patterns), supports both `date -v` and `date -d`, avoids `column -t` (absent on macOS), and keeps the final newline (bash `read` treats a trailing line without \n as EOF). Verified on macOS: today/7-day/single-day summaries, --check under and over threshold (exit 0/1), --help, and no-data path. Co-Authored-By: Claude --- scripts/core/cost-alert.sh | 37 ++++++++++ scripts/core/cost-monitor.sh | 135 +++++++++++++++++++++++++++++++++++ 2 files changed, 172 insertions(+) create mode 100755 scripts/core/cost-alert.sh create mode 100755 scripts/core/cost-monitor.sh diff --git a/scripts/core/cost-alert.sh b/scripts/core/cost-alert.sh new file mode 100755 index 00000000..4f35d70a --- /dev/null +++ b/scripts/core/cost-alert.sh @@ -0,0 +1,37 @@ +#!/bin/bash +# ============================================================ +# Auto Company — Cost Alert wrapper (launchd-triggered) +# ============================================================ +# Runs cost-monitor.sh --check against the daily threshold; when +# today's spend exceeds it, pops a macOS desktop notification and +# records the event. Designed to be scheduled by launchd. +# +# Config: +# COST_THRESHOLD_DAILY daily spend threshold in USD (default 20) +# +# Logs one line per run to logs/cost-daily.log regardless of outcome. +# ============================================================ + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +PROJECT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)" +COST_MONITOR="$SCRIPT_DIR/cost-monitor.sh" +RUN_LOG="$PROJECT_DIR/logs/cost-daily.log" +THRESHOLD="${COST_THRESHOLD_DAILY:-20}" + +[ -f "$COST_MONITOR" ] || { echo "cost-monitor.sh missing at $COST_MONITOR" >&2; exit 1; } +mkdir -p "$PROJECT_DIR/logs" + +rc=0 +output="$("$COST_MONITOR" --check --threshold "$THRESHOLD" 2>&1)" || rc=$? + +echo "$(date '+%Y-%m-%d %H:%M:%S') [rc=$rc] $output" >> "$RUN_LOG" + +if [ "$rc" -ne 0 ]; then + # Threshold exceeded: desktop notification (best effort) + keep log + osascript -e "display notification \"$output\" with title \"Auto-Company 成本告警\" sound name \"Glass\"" 2>/dev/null \ + || echo "$(date '+%Y-%m-%d %H:%M:%S') [notify-failed]" >> "$RUN_LOG" +fi + +exit 0 diff --git a/scripts/core/cost-monitor.sh b/scripts/core/cost-monitor.sh new file mode 100755 index 00000000..2f175b39 --- /dev/null +++ b/scripts/core/cost-monitor.sh @@ -0,0 +1,135 @@ +#!/bin/bash +# ============================================================ +# Auto Company — Cost Monitor (daily summary + threshold alert) +# ============================================================ +# Parses per-cycle cost from logs/auto-loop.log and summarizes +# by calendar day. +# +# Usage: +# ./cost-monitor.sh # today's summary +# ./cost-monitor.sh --days 7 # last 7 days, oldest first +# ./cost-monitor.sh --threshold 25 # daily threshold in USD (default 20) +# ./cost-monitor.sh --check # alert mode: exit 1 if today exceeds +# # threshold, else 0 (cron-friendly) +# ./cost-monitor.sh --date 2026-08-02 # single day (or "yesterday") +# +# Exit codes: +# 0 OK (no alert, or no data) +# 1 threshold exceeded (--check mode) +# 2 usage error +# ============================================================ + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +PROJECT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)" +LOG_FILE="$PROJECT_DIR/logs/auto-loop.log" +ALERT_LOG="$PROJECT_DIR/logs/cost-alerts.log" + +THRESHOLD="${COST_THRESHOLD_DAILY:-20}" +DAYS=1 +MODE="summary" +TARGET_DAY="" + +# macOS `date -v` vs GNU `date -d`: support both. +days_ago() { + local n="$1" + if date -v-"${n}"d '+%Y-%m-%d' >/dev/null 2>&1; then + date -v-"${n}"d '+%Y-%m-%d' + else + date -d "-${n} days" '+%Y-%m-%d' + fi +} +today() { days_ago 0; } + +while [ $# -gt 0 ]; do + case "$1" in + --days) DAYS="$2"; shift 2 ;; + --threshold) THRESHOLD="$2"; shift 2 ;; + --date) TARGET_DAY="$2"; shift 2 ;; + --check) MODE="check"; shift ;; + --help|-h) + grep -E '^# ' "$0" | sed 's/^# //' + exit 0 + ;; + *) echo "Unknown option: $1" >&2; exit 2 ;; + esac +done + +case "$TARGET_DAY" in + ""|today) TARGET_DAY="$(today)" ;; + yesterday) TARGET_DAY="$(days_ago 1)" ;; +esac + +if [ ! -f "$LOG_FILE" ]; then + echo "No log file at $LOG_FILE" + exit 0 +fi + +# Emit " " for every line carrying a real cost +# (skips "cost: N/A" and empty values). +raw="$(awk ' + match($0, /cost: [0-9]+(\.[0-9]+)?/) { + cost = substr($0, RSTART + 6, RLENGTH - 6) + day = substr($0, 2, 10) # [YYYY-MM-DD ... + if (cost != "" && day ~ /^[0-9]{4}-[0-9]{2}-[0-9]{2}$/) { + print day, cost + } + } +' "$LOG_FILE")" + +if [ -z "$raw" ]; then + echo "No cycle cost data found in $LOG_FILE yet." + exit 0 +fi + +# Aggregate per day: +agg="$(echo "$raw" | awk ' + { s[$1] += $2; n[$1]++; if ($2 > m[$1]) m[$1] = $2 } + END { for (d in s) printf "%s %.6f %d %.6f\n", d, s[d], n[d], m[d] } +' | sort)" + +# Build the list of days to show. +days_list="" +for i in $(seq 0 "$((DAYS - 1))"); do + days_list="$days_list $(days_ago "$i")" +done + +# Filter aggregated rows to the requested window. +rows="" +while read -r d total runs maxcost; do + [ -z "$d" ] && continue + if [ "$DAYS" -eq 1 ] && [ "$d" != "$TARGET_DAY" ]; then + continue + fi + case " $days_list " in + *" $d "*) rows="$rows$d $total $runs $maxcost\n" ;; + esac +done <<< "$agg" +# Keep the trailing \n: bash `read` treats a final line without a newline +# as EOF and would skip the row entirely. + +if [ "$MODE" = "check" ]; then + today_total="$(printf '%b' "$rows" | awk -v t="$TARGET_DAY" '$1 == t { print $2; exit }')" + today_total="${today_total:-0}" + today_fmt="$(awk -v t="$today_total" 'BEGIN { printf "%.2f", t }')" + if awk -v t="$today_total" -v th="$THRESHOLD" 'BEGIN { exit !(t > th) }'; then + msg="$(date '+%Y-%m-%d %H:%M:%S') ALERT: $TARGET_DAY cost \$$today_fmt exceeds threshold \$$THRESHOLD" + echo "$msg" + echo "$msg" >> "$ALERT_LOG" + exit 1 + fi + echo "$(date '+%Y-%m-%d %H:%M:%S') OK: $TARGET_DAY cost \$$today_fmt within threshold \$$THRESHOLD" + exit 0 +fi + +# Summary table (pure printf alignment; macOS column lacks -t) +printf '%-12s %9s %6s %9s %9s\n' "Date" "Total\$" "Runs" "Avg\$" "Max\$" +printf '%-12s %9s %6s %9s %9s\n' "----------" "---------" "------" "--------" "--------" +printf '%b' "$rows" | while read -r d total runs maxcost; do + avg=$(awk -v t="$total" -v n="$runs" 'BEGIN { printf "%.2f", (n ? t/n : 0) }') + printf '%-12s %9.2f %6s %9s %9.2f\n' "$d" "$total" "$runs" "$avg" "$maxcost" +done +grand=$(printf '%b' "$rows" | awk '{ s += $2 } END { printf "%.2f", s }') +printf '%-12s %9s\n' "----------" "---------" +printf '%-12s %9s\n' "Total" "\$$grand"