Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
05893ab
Fix competition closeout, scheduling, Ops workflow and acceptance gates
PureStream108 Oct 14, 2026
480df32
Prevent completion and postprocess lock-order deadlocks
PureStream108 Oct 14, 2026
3b11f63
Cover asynchronous transcript repair and safe tool continuation
PureStream108 Oct 14, 2026
dc5e9d0
feat(agent): bounded tool output pipeline and teammate messaging
PureStream108 Oct 17, 2026
e0d8d17
feat(competition): durable control operations and instance lifecycle
PureStream108 Oct 17, 2026
bebb8fe
feat(ops): built-in workflow presets, TSecBench platform and question…
PureStream108 Oct 17, 2026
c415f88
feat(core): runtime telemetry, health details and paged log reads
PureStream108 Oct 17, 2026
1c78cb6
feat(frontend): GZCTF setup form, log search and log pagination
PureStream108 Oct 17, 2026
bfc1e9e
chore(build): install the docker CLI package in the app image
PureStream108 Oct 17, 2026
553edc9
chore(scripts): PCTF run controls and collaboration report
PureStream108 Oct 17, 2026
9766a52
chore: strip source comments and module docstrings
PureStream108 Oct 17, 2026
3be32fd
chore: remove the personal .vscode settings from the repository
PureStream108 Oct 17, 2026
7892046
feat: complete nextchange pentest workflow and TSecBench multi-flag c…
PureStream108 Oct 18, 2026
386482d
feat: classify multi-flag TSecBench challenges as pentest
PureStream108 Oct 18, 2026
13556a6
feat: schedule easy and medium challenges before hard
PureStream108 Oct 18, 2026
f8f315e
fix: ignore timestamps when matching retried recon messages
PureStream108 Oct 18, 2026
d6fa9d1
feat: use idle Members when remote capacity is saturated and fill pen…
PureStream108 Oct 18, 2026
acf3ae5
fix: scope target-instance messages to the assignment epoch
PureStream108 Oct 18, 2026
377ed96
fix: never mint unseated solvers for competition projects
PureStream108 Oct 18, 2026
12acf43
chore(scripts): mark collab report executable
PureStream108 Oct 20, 2026
8b8712b
fix(ci): make e2e diagnostics writable
PureStream108 Oct 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,14 @@
**/__pycache__
**/*.pyc
.pytest_cache
.ruff_cache
coverage.json
.venv
.env
.env.*
.qa-artifacts/
testconfigkey.txt
backend/config/config.yaml
tools/
data/
logs/
Expand All @@ -15,4 +22,4 @@ projects/
memory/
exports/
ipc_check.js
log.txt
log.txt
136 changes: 130 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,14 @@ on:
permissions:
contents: read

concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
timeout-minutes: 45
services:
postgres:
image: postgres:16-alpine
Expand Down Expand Up @@ -46,16 +50,78 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install -e ".[dev,docker]"
python -m pip install -e ".[dev,docker,browser]"

- name: Set up Node for frontend behavior tests
uses: actions/setup-node@v4
with:
node-version: "24"

- name: Install explicit integration dependencies
run: |
sudo apt-get update
sudo apt-get install -y postgresql

- name: Resolve Playwright cache version
id: playwright
run: python -c "import importlib.metadata; print('version='+importlib.metadata.version('playwright'))" >> "$GITHUB_OUTPUT"

- name: Cache Playwright browsers
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-${{ steps.playwright.outputs.version }}

- name: Install Playwright browser and system dependencies
run: |
python -m playwright install --with-deps chromium

- name: Run tests
run: python -m pytest -q
- name: Run tests with branch coverage
run: python -m pytest -q --require-postgres --require-browser --cov=backend --cov-branch --cov-report=term-missing --cov-report=json --junitxml=data/exports/pytest-ci.xml

- name: Static and deployment checks
- name: Enforce architectural coverage floors
run: python scripts/check_coverage.py

- name: Verify temporary PostgreSQL isolation
run: |
native_postgres_ctl="$(pg_config --bindir)/pg_ctl"
test -x "$native_postgres_ctl"
env -u IPC_TEST_DATABASE_URL -u IPC_DATABASE_URL IPC_TEST_POSTGRES_EXECUTABLE="$native_postgres_ctl" \
python -m pytest -q --require-postgres tests/test_postgres_isolation.py tests/test_goal_intent_postgres.py tests/test_ops_stream.py tests/test_competition_monitor.py tests/test_competition_migration_smoke.py

- name: Full lint and deployment checks
run: |
python -m ruff check backend scripts tests
python -m compileall -q backend scripts tests
python -m alembic upgrade head --sql > /tmp/ipc-migration.sql
docker compose config --quiet
docker compose -f docker-compose.e2e.yml config --quiet
node tests/js/ops_chat_checks.mjs .
node tests/js/pentest_checks.mjs .

- name: Upload test and coverage diagnostics
if: always()
uses: actions/upload-artifact@v4
with:
name: ipc-tests-coverage
path: |
coverage.json
data/exports/pytest-ci.xml
data/exports/pentest

package-smoke:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v6
with:
python-version: "3.11"
cache: pip
- name: Install package build dependencies
run: python -m pip install -e ".[docker]" build
- name: Verify installed platform workflows
run: python scripts/check_workflow_package.py

task-image:
runs-on: ubuntu-latest
Expand All @@ -64,8 +130,18 @@ jobs:
- name: Checkout
uses: actions/checkout@v5

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build task image
run: docker build -f docker/member/Dockerfile -t ipc-task:ci .
uses: docker/build-push-action@v6
with:
context: .
file: docker/member/Dockerfile
tags: ipc-task:ci
load: true
cache-from: type=gha,scope=task-image
cache-to: type=gha,scope=task-image,mode=max

- name: Run CTF tool acceptance
run: docker run --rm -v "$PWD/scripts:/acceptance:ro" ipc-task:ci bash /acceptance/c5_task_acceptance.sh
Expand All @@ -82,3 +158,51 @@ jobs:

- name: Run AI CPU smoke
run: docker run --rm -v "$PWD/scripts:/acceptance:ro" ipc-task:ci python3 /acceptance/ai_smoke.py

e2e:
runs-on: ubuntu-latest
timeout-minutes: 30
env:
IPC_INSTALL_WSRX: "0"
steps:
- name: Checkout
uses: actions/checkout@v5

- name: Start deterministic E2E stack
run: |
docker compose build ipc-app
docker compose -f docker-compose.e2e.yml up -d e2e-postgres e2e-platform e2e-app

- name: Run Playwright competition acceptance
run: docker compose -f docker-compose.e2e.yml run --rm e2e-runner

- name: Collect E2E stack logs
if: always()
run: |
mkdir -p data/exports/e2e
sudo chown -R "$(id -u):$(id -g)" data/exports/e2e
docker compose -f docker-compose.e2e.yml ps -a > data/exports/e2e/compose-status.txt
docker compose -f docker-compose.e2e.yml logs --no-color > data/exports/e2e/compose.log

- name: Upload E2E diagnostics
# All validation jobs must be required by the repository's branch rules;
# declaring a job here alone does not configure branch protection.
if: always()
uses: actions/upload-artifact@v4
with:
name: ipc-e2e-diagnostics
path: data/exports/e2e

- name: Stop E2E stack
if: always()
run: docker compose -f docker-compose.e2e.yml down -v

required-checks:
if: always()
needs: [test, package-smoke, task-image, e2e]
runs-on: ubuntu-latest
steps:
- name: Require every validation job to succeed
env:
JOB_RESULTS: ${{ toJSON(needs) }}
run: python -c "import json,os; results=json.loads(os.environ['JOB_RESULTS']); assert all(job['result']=='success' for job in results.values()), results"
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ backend/config/config.yaml
logs/
data/
.ipc/
memory/
/memory/
projects/
Wp/
docs/
Expand Down Expand Up @@ -64,6 +64,7 @@ htmlcov/
.cache
nosetests.xml
coverage.xml
coverage.json
*.cover
*.py.cover
.hypothesis/
Expand Down Expand Up @@ -221,7 +222,7 @@ cython_debug/
# that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
# and can be added to the global gitignore or merged into this file. However, if you prefer,
# you could uncomment the following to ignore the entire vscode folder
# .vscode/
.vscode/
# Temporary file for partial code execution
tempCodeRunnerFile.py

Expand Down
4 changes: 0 additions & 4 deletions .vscode/settings.json

This file was deleted.

22 changes: 16 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ FROM python:3.12-slim

WORKDIR /app

# Debian's docker.io package provides the daemon; the app needs the docker.io
# client command, supplied by the split docker-cli package below.

RUN sed -i 's|http://deb.debian.org/debian|https://mirrors.tuna.tsinghua.edu.cn/debian|g; s|http://deb.debian.org/debian-security|https://mirrors.tuna.tsinghua.edu.cn/debian-security|g' /etc/apt/sources.list.d/debian.sources \
&& printf '%s\n' \
'Acquire::Retries "5";' \
Expand All @@ -12,7 +15,8 @@ RUN sed -i 's|http://deb.debian.org/debian|https://mirrors.tuna.tsinghua.edu.cn/
'Acquire::BrokenProxy "true";' \
> /etc/apt/apt.conf.d/80-ipc-retries \
&& apt-get update && apt-get install -y --no-install-recommends \
ca-certificates docker.io \
ca-certificates docker-cli \
&& command -v docker \
&& rm -rf /var/lib/apt/lists/*

COPY pyproject.toml /app/pyproject.toml
Expand All @@ -33,11 +37,17 @@ RUN python -c "from pathlib import Path; from backend.skills.store import SkillS
# subprocesses are owned by the backend process (Members just connect to
# ipc-app:<port>).
ARG WSRX_VERSION=0.6.1
RUN python -c "import urllib.request; urllib.request.urlretrieve('https://github.com/XDSEC/WebSocketReflectorX/releases/download/${WSRX_VERSION}/wsrx-cli-${WSRX_VERSION}-linux-musl-x86_64.tar.gz', '/tmp/wsrx.tar.gz')" \
&& tar -xzf /tmp/wsrx.tar.gz -C /tmp \
&& install -m 0755 /tmp/wsrx /usr/local/bin/wsrx \
&& rm -f /tmp/wsrx /tmp/wsrx.tar.gz \
&& wsrx --version
ARG INSTALL_WSRX=1
RUN if [ "$INSTALL_WSRX" = "1" ]; then \
python -c "import urllib.request; urllib.request.urlretrieve('https://github.com/XDSEC/WebSocketReflectorX/releases/download/${WSRX_VERSION}/wsrx-cli-${WSRX_VERSION}-linux-musl-x86_64.tar.gz', '/tmp/wsrx.tar.gz')" \
&& tar -xzf /tmp/wsrx.tar.gz -C /tmp \
&& install -m 0755 /tmp/wsrx /usr/local/bin/wsrx \
&& rm -f /tmp/wsrx /tmp/wsrx.tar.gz \
&& wsrx --version; \
else \
printf '#!/bin/sh\necho wsrx disabled in this image\nexit 127\n' > /usr/local/bin/wsrx \
&& chmod +x /usr/local/bin/wsrx; \
fi

# /app/data is bind-mounted by docker-compose.yml. PostgreSQL owns runtime
# facts; filesystem workspaces and generated files share one Artifact tree.
Expand Down
42 changes: 20 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,11 @@


<div align="center">

[![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/Python-3.10%2B-3776AB.svg?logo=python&logoColor=white)](https://www.python.org/)
[![FastAPI](https://img.shields.io/badge/API-FastAPI-009688.svg?logo=fastapi&logoColor=white)](https://fastapi.tiangolo.com/)
[![PostgreSQL 16](https://img.shields.io/badge/Runtime-PostgreSQL_16-4169E1.svg?logo=postgresql&logoColor=white)](#durable-runtime-state)
[![Docker Compose](https://img.shields.io/badge/Deploy-Docker_Compose-2496ED.svg?logo=docker&logoColor=white)](#quick-start)

</div>

<div align="center">
[🚀 快速开始](#quick-start) • [✨ 核心设计](#core-innovations) • [🖥️ 控制台](#agent-workbench) • [🏗️ 系统架构](#system-architecture) • [🧰 工具运行时](#tool-runtime) • [🧪 开发](#development)

</div>

## Current Reward
Expand Down Expand Up @@ -120,6 +113,12 @@ PostgreSQL 只保存运行时事实、协调状态和可查询元数据;文件
- 每个题目最多分配两个解题席位;WP 任务使用原作者 Member/session,远端实例与本地 sandbox 分开计量。
- `POST /api/workflows/{id}/preflight`、`POST /api/workflows/{id}/start` 和 `/api/runs/{id}/...` 提供平台预检、启动、同步、暂停、恢复、停止与事件查询。

#### Built-in GZCTF workflow

GZCTF is available as the built-in default platform workflow. Use **Competition → New GZCTF** or `POST /api/ops/workflows/gzctf` and provide the platform origin, game ID, and optional competition name, team ID, deadline, container concurrency, and attachment CDN origin. Choose either a participant token or username/password; credentials are stored in the workflow secret namespace and are never copied into the run snapshot.

The same preset is available to the IPC MCP tool `ipc_gzctf_workflow`. After saving the draft, run the normal confirmation and preflight flow. Future GZCTF runs reuse the native challenge discovery, attachments, submissions, verdict polling, and container lifecycle adapter; no per-event HTTP field mapping is required. Other platforms and custom HTTP JSON workflows remain available through the generic workflow endpoint.

### <a id="tool-runtime"></a>MCP

Member 在题目容器中使用按分类注册的 CTF 工具,并可通过 MCP 访问:
Expand Down Expand Up @@ -151,7 +150,6 @@ Docker Compose 将数据库放入命名卷,将需要人工访问或跨容器
| `data/artifacts/logs/` | 项目、LLM、工具与 Memory JSONL |
| `data/artifacts/exports/` | Writeup、日志和 Obsidian/Markdown Memory 导出快照 |
| `data/ops-agent/` | IPC 模型配置和工作流密钥等文件型机密 |
| `ipc_claude_home` Docker 卷 | `claudecode` 运行器的原生 JSONL 会话 |

导出采用无损编号命名;同名文件存在时会创建 `名称01`、`名称02` 等新文件,不覆盖历史导出。

Expand All @@ -165,7 +163,7 @@ Docker Compose 将数据库放入命名卷,将需要人工访问或跨容器
| Docker Compose v2 | 必需 | 标准部署与服务编排 |
| Docker Socket | 必需 | App 需要创建题目容器;IPC Runner 同样依赖 Socket |
| Linux Docker 主机 | 推荐/已验证 | Compose 会挂载 Docker Socket 与 Compose 插件 |
| LLM 端点 | 开始解题时需要 | Diamond 与已启用 Member 使用 OpenAI、Anthropic、Claude Code、DeepSeek、Pi 或 Mock 适配器 |
| LLM 端点 | 开始解题时需要 | Diamond 与已启用 Member 使用 OpenAI、Anthropic、DeepSeek、Pi 或 Mock 适配器 |
| 浏览器 | 可选 | 只保留 Playwright Python 包;浏览器二进制不在任务镜像内,由部署环境显式提供路径 |

> [!WARNING]
Expand Down Expand Up @@ -199,7 +197,7 @@ docker compose logs -f ipc-app
docker compose down
```

此命令不会删除 `./data`、`ipc_postgres_data` 或 `ipc_claude_home`。只有主动删除它们或执行 `docker compose down -v` 才会清理命名卷。
此命令不会删除 `./data` 或 `ipc_postgres_data`。只有主动删除它们或执行 `docker compose down -v` 才会清理命名卷。

### 2. Configure the LLM runtime

Expand Down Expand Up @@ -258,7 +256,7 @@ UI 运行在操作者可信的 Docker 网络边界内,默认不增加浏览器

IPC 是面向操作者的持续对话代理,可诊断运行环境、辅助题目沙箱、操作项目生命周期,并生成声明式平台工作流。工作流必须经人工确认才能执行。其对话、运行事件和工作流元数据存入 PostgreSQL,文件型密钥保存在 `data/ops-agent/`。

`claudecode` 运行器会持久化原生会话,以便在容器替换后通过 `--resume` 延续上下文。无论使用哪种 Runner,内部 MCP 请求都应通过 `IPC_RUNNER_TOKEN` 鉴权。
IPC 与比赛 Member 共用 `backend/agent/` 中的统一运行时:会话记录在 `agent_sessions` / `agent_events`,因此容器替换后可以从持久事件重建上下文,无需外部 Runner。

---

Expand All @@ -275,7 +273,7 @@ flowchart TB

subgraph OPS["IPC 行动代理层"]
IPC["IPC 对话代理"]
RUNNER["Claude Code Runner\n或 OpenAI-compatible API"]
RUNNER["统一 Agent 运行时\nagent_sessions / agent_events"]
IMCP["内部 IPC MCP\n项目生命周期与诊断工具"]
IPC <--> RUNNER
RUNNER --> IMCP
Expand Down Expand Up @@ -450,6 +448,16 @@ docker run --rm \

---

## References

Pentest 工作区、渗透图谱、证据与 Agent 工具说明见 [Pentest 使用文档](docs/pentest.md)。
GZCTF 与 TSecBench 固定工作流的个性化配置见 [平台文档](backend/platform/README.md)。
普通题目、独立渗透目标和平台多 Flag 题目分别遵循各自的完成条件,子任务继续使用本项目 Agent Harness。

[Cairn](https://github.com/oritera/Cairn)

[ctf-agent](https://github.com/verialabs/ctf-agent)

## 👥 Contributors

<p>
Expand All @@ -463,16 +471,6 @@ docker run --rm \
<img src="https://github.com/springbot2025.png?size=80" width="80" height="80" style="border-radius: 50%;" alt="springbot" title="springbot" />
</a>
</p>


## 🤝 Contribution

欢迎提交 bug 报告、稳定性测试、工具集成、文档和架构改进。

1. 先在 [Issues](https://github.com/PureStream108/IPC_CTFAgent/issues) 描述问题或设计提案。
2. 创建聚焦的分支,并为修改的运行时边界补充测试。
3. 在 Pull Request 中说明行为变化、迁移影响和验证结果。

---

## 📝 License
Expand Down
1 change: 0 additions & 1 deletion backend/agent/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +0,0 @@
"""Unified agent runtime: one durable action loop for Ops, competition, projects."""
Loading
Loading