English · 中文
A publication-grade meta-analysis & systematic review, run by your coding agent — with you signing off at the points that matter.
Built natively for Claude Code; runs in Codex and any agent that loads the SKILL.md format.
Five skills · seven human sign-off gates · red lights computed by R, not the LLM.
You ask your agent for a meta-analysis or a systematic review. This runs the real pipeline — protocol, PRISMA-S search ingestion, dual-blind screening with Cohen's κ, dual extraction, statistics executed locally in R (metafor and friends), forest/funnel figures, and an APA / PRISMA-2020 draft — and stops at seven irreversible points to hand you an audit card built for that decision. You read the card, sign off or send it back.
The agent does the grunt work that used to eat hundreds of research-assistant hours. You make the calls that can't be delegated — and that carry authorship. This is not a one-click paper machine, and it does not pretend to be (see Honest boundaries).
In your agent's chat, after install:
Do a three-level meta-analysis of empathy on emotion recognition,
with intervention type as the key moderator.
Three design decisions separate this from "ask the model to do a meta-analysis."
1 · Red lights are computed by R, not read by the LLM.
audit_parse.R derives every statistical flag — I² > 90%, a single study carrying > 30% of the weight, PET intercept non-significant, funnel asymmetry — directly from the fitted metafor object's fields. The model never eyeballs a summary() printout to decide what's significant. If the engine says red, it's red.
2 · The seven sign-off gates actually block.
Each gate's verdict lands in a decisions/ file. Before the next stage runs, signoff_preflight.py reads it — unsigned, rejected, or failing a hard check → non-zero exit code, and the downstream stage cannot start. It is not a prompt nudge to "please review." Hard checks (a red light, κ below threshold, an unverified citation) are evaluated before the human sign-off and block even a signed gate — so a rubber-stamp can't wave through a broken result.
3 · Gold-standard calibration answers "how accurate is the AI?" with data.
Hand-code 10–20 studies. The agent extracts the same ones with its real production prompt. compute_kappa_icc.R scores per-field agreement (Cohen's κ for categorical fields, ICC for continuous) and sorts every field into green (trust it, skip review) · amber (spot-check) · red (review every one). That per-field table is your evidence-based answer to which fields you can trust the agent on — not a vibe.
Five interlocking skills. meta-analysis and systematic-review consume the shared SSOT base by reference; academic-ref-check guards the final citation gate; manuscript-typeset renders the finished manuscript for submission.
| Skill | What it owns | Stage |
|---|---|---|
| systematic-review | Strict PRISMA 2020 / Cochrane v6.5: protocol, PRISMA-S search, dual-blind screening + κ, dual extraction, RoB 2 / ROBINS-I, GRADE, AMSTAR 2 | L1–L8 |
| meta-analysis | Quantitative synthesis: effect sizes, FE/RE, I²/τ²/95% PI, subgroup & meta-regression, publication bias, psychology methods (RVE · three-level · MASEM · Hunter-Schmidt) | L9–L12 |
| review-methodology-foundations | The shared base — 20 Single-Source-of-Truth methodology documents the two review skills cite (effect-size decisions, synthesis methods, RoB tool mapping, GRADE, AMSTAR 2, reporting standards, AI-failure defenses, …) | all (referenced) |
| academic-ref-check | End-to-end reference checking, APA 7 formatting, HTTP verification against OpenAlex + CrossRef — drives the final citation gate | L11 |
| manuscript-typeset | Standalone academic typesetting: a finished Markdown manuscript → submission-ready .docx + APA 7 (man/jou) PDF + a clean Markdown copy, behind a fidelity gate that fails the build if any number in the output has no source in the input |
L11–L12 |
Twelve methodology phases (L1–L12). The agent runs them; you stand at seven irreversible gates.
L1–L3 type · PICO · protocol ── you: define the question, confirm scope
L4 search strategy ───────────────────▶ SP1 freeze search + eligibility
L5 ingest exports · dedup (PRISMA-S)
L6 dual-blind screening + κ ──────────▶ SP2 resolve disagreements
L7 calibrate ────────────────────────▶ SP3 approve extraction go-live
dual extraction + numeric check ──▶ SP4 verify key numbers
L8 risk of bias (RoB 2 / ROBINS-I)
L9 EFFECT SIZES → MODEL → I²/τ²/PI ──▶ SP5 accept model + heterogeneity
moderators · sensitivity · pub-bias▶ SP6 accept interpretation
L10 GRADE
L11 APA / PRISMA draft + citations ────▶ SP7 clear flagged citations
L12 AMSTAR 2 self-assessment
↑ AI does the grunt work ↑ seven gates a script enforces
You also do what no agent can: run the database searches that have no API (export RIS/CSV from Web of Science, PsycINFO, CNKI), fetch paywalled PDFs, write the introduction and discussion, and put your name on it. Realistic human effort drops from hundreds of hours to roughly 5–10 working days of judgment concentrated at the gates.
Running this for real is not free and not instant — supervised, publication-grade evidence synthesis has an irreducible floor. Three separate costs, kept separate so you can plan each:
1 · Compute / model API. Order of magnitude tens to a few hundred USD for one full review, depending on how you pay for the model — a flat Max/Pro subscription absorbs most of it; metered API billing sits at the higher end. The R statistics, figure rendering, and preflight run locally and cost nothing.
2 · Agent wall-clock — roughly 10–15 hours for a full run. Most of it is unattended — you leave it running: search ingestion and dedup, batched dual-blind screening, dual extraction, R pooling, figure rendering, drafting. The attended part is the seven sign-off gates — you have to be at the keyboard to read each audit card and decide.
3 · Human labour — a few hours, concentrated where it can't be delegated:
| Human task | Rough effort |
|---|---|
| Run the no-API database searches, export RIS/CSV | 2–4 h |
| Fetch paywalled full text | variable — depends on your access |
| The seven sign-off gates | ~10–30 min each → ~1.5–3.5 h total |
| Write the introduction & discussion, put your name on it | separate scholarly work — hours to days |
A rough scaling handle. The single real run this was validated on screened 23,626 records (of which 4,715 went through dual-blind screening + adjudication), pulled ~1.5 GB of full text, and used ~97 SubAgent dispatches end to end over ~10–15 h. As a blended, whole-pipeline rate that is ≈ 4 SubAgent dispatches per 1,000 records screened — a crude single-run estimate, not a formula. It moves a lot with the number of databases, k (studies that survive), and how many records survive dedup; treat it as an order of magnitude, not a quote.
Why "fast" has a floor. The wall-clock lower bound comes from the methodology, not from slow engineering: dual-blind screening, dual extraction, and seven sign-off gates are structural. Wall-clock does not scale linearly with record count — a review with 10× fewer records is not 10× faster, because the fixed human-gate loop and the longest-pole stages dominate. The only honest ways to spend less are narrower scope (smaller k, fewer databases) or an explicit downgrade (Learning mode, every shortcut disclosed) — never skipping a gate.
Every gate renders a self-contained, offline, bilingual HTML card — numbers from the engine, red/amber/green from R, plain-language explanation of what to look at and what you're deciding. Open one in any browser.
 SP1 · Freeze search — seed-recall check before the search is locked. |
 SP2 · Screening — Cohen's κ gauge; only disagreements surface for you. |
 SP3 · Calibration — per-field reliability table; which fields go live unreviewed. |
 SP4 · Numeric check — every effect-size input checked against the source figure. |
 SP6 · Claims vs data — every claim in the draft checked against what the analysis actually supports. |
 SP7 · Citations — HTTP-verified references; suspected fabrications flagged red. |
The five skills are interdependent — install all five into your agent's skills directory. Three commands: install dependencies, symlink the skills, health-check.
git clone https://github.com/O0000-code/meta-analysis-skill.git
cd meta-analysis-skill
./setup.sh # install dependencies: R packages + node_modules + Python (one command)
./install.sh # symlink the five skills into your agent (default: ~/.claude/skills)
python3 doctor.py # health check — what's present/missing and what each gap costs
# ./install.sh ~/.codex/skills # Codex CLI
# ./install.sh ~/.agents/skills # cross-agent conventionsetup.sh installs what it can without sudo — the R/CRAN packages (metafor, robumeta, clubSandwich, metaSEM, psychmeta, puniform, irr, psych, jsonlite, …) plus dmetar from GitHub, the audit-card renderer's node_modules, and the Python figure/docx dependencies (matplotlib, numpy, python-docx, pyyaml). It only prints the command for things that need sudo or a large download — pandoc and XeLaTeX/TinyTeX (for manuscript-typeset), JAGS (for RoBMA), and CJK/serif fonts.
doctor.py then reports exactly what is present or missing and, for each gap, what capability you lose — so a missing optional dependency degrades a feature loudly instead of failing silently. When JAGS is absent, the pipeline auto-falls back to puni_star + PET-PEESE and says so. The Python helpers (verify_http.py, signoff_preflight.py, ingestion) use only the standard library.
Companion skills (search, figure styling, PDF fetch, full-text conversion) are optional and shipped separately — where to get each, and how the pipeline degrades without it, is in
docs/DEPENDENCY-MATRIX.md.
Want to see the outputs before committing to a real run? The bundled demo runs the L9 → L12 tail on a synthetic 10-study dataset — one command, ~1–2 minutes of compute:
bash demo/run_demo.shIt produces the full shape of a finished run: a forest plot, a funnel plot, the PRISMA 2020 flow diagram, and a risk-of-bias traffic-light figure; two sign-off cards (SP5 model + heterogeneity, SP6 publication bias); and a mini delivery manifest. Open the HTML cards in any browser — they are fully offline. The figures come from the skill's fixed renderers (each with its own QC gate), not ad-hoc plotting code.
Synthetic data, not a real search. The 10 studies, effect sizes, PRISMA counts, and RoB ratings are all invented (
Demo-Study-NNplaceholders) — the demo shows the artefact forms, not a result. What a real, human-supervised run actually costs is in Cost & effort above. Seedemo/README.mdfor details.
Set in L3; the same methodology, three human-involvement profiles.
| Mode | For | Human role |
|---|---|---|
| Strict | Top-journal submission, the fully-automated baseline | Final acceptance, once |
| Learning | Coursework, proposals — simplified, but every shortcut disclosed | Sampled spot-checks |
| Supervised | Publishable output where you sign off at every irreversible point | Decision review at all seven gates |
Supervised mode is the headline: it carries the full strict-mode methodology (nothing is downgraded) and adds the seven structural gates on top. It's what you want when "AI drafts it, a human checks every critical call, and it has to reach publication grade" describes the job.
Responsible use. AI drafts, a human signs off at seven irreversible gates, and the signing author writes the theory and takes full responsibility for every claim — fully-unsupervised auto-submission is out of scope by design, not omission. The full stance, the two-phase supervision model, and the honest boundaries are in
RESPONSIBLE_USE.md.
This project's standing rule is to never claim more certainty than the evidence supports — that applies to the tool itself:
- Not a one-click paper. Search execution (databases without an API need manual export), paywalled full text, the seven sign-off judgments, the introduction and discussion, and authorship are human work by design. The end state is AI does the grunt work + a human signs off at the gates + a human writes the theory + a human takes responsibility.
- Fully-autonomous "no human" mode is out of scope — for technical and accountability reasons, not as a missing feature.
- Mechanism validated on synthetic data; the real-data "last mile" is in progress. The supervised engine (gates, calibration, red-light computation) has been exercised end-to-end on a synthetic pilot. Validating the full chain on real PDFs and real searches is ongoing; the synthetic pilot's accuracy numbers are an optimistic upper bound, not real-world accuracy.
The 20 SSOTs operationalize methodology from the Cochrane Handbook v6.5, Cooper, Borenstein, the JBI Manual, and the APA Publication Manual, and reference the PRISMA 2020, GRADE, AMSTAR 2, and APA JARS standards — each with citations. Statistical methods, formulas, and thresholds are facts and procedures (not subject to copyright); they are operationalized in our own words with attribution, never reproduced as verbatim prose. Standardized instruments (RoB 2, ROBINS-I, AMSTAR 2) are quoted with their official sources because an instrument must keep its exact wording. Full credits — sources, instruments, bundled runtime assets, and dependencies — are in THIRD_PARTY.md.
Source-available, non-commercial — dual-licensed by file type. Code (scripts/**, *.py / *.R / *.mjs / *.sh, setup.sh / install.sh / doctor.py) is under PolyForm Noncommercial 1.0.0; documentation (*.md, SKILL.md, references/**, assets/**) is under CC BY-NC-SA 4.0. LICENSE holds the file-type split; LICENSES/ holds the full texts.
This is source-available, not OSI open source — PolyForm Noncommercial permits use, modification, and sharing for non-commercial purposes, and CC BY-NC-SA lets you fork and improve the docs under the same terms. Commercial licensing is available by separate agreement (contact the copyright holder). GitHub's sidebar shows "Other" because a genuine dual-licensed repo has no single-license badge — that is the correct, honest result, not a misconfiguration. This license applies going forward; earlier versions released under MIT keep those terms (rights already granted are not revoked).
The methodology sources and standardized instruments it cites (RoB 2, PRISMA 2020, GRADE, AMSTAR 2, …) carry their own terms — see THIRD_PARTY.md.