XuanL17 · Jul 30, 2026 · Jul 7, 2026 · Jul 7, 2026 · Jul 7, 2026 · Jul 14, 2026
diff --git a/.gitignore b/.gitignore
@@ -69,6 +69,22 @@ logs/
 .env
 .env.local
 
+# Local pipeline settings may contain API keys
+config/settings.yaml
+config/settings.yml
+config/settings.json
+
+# Generated pipeline outputs and simulation runs
+generation_pipeline/outputs/*
+!generation_pipeline/outputs/.gitkeep
+generation_pipeline/old_outputs/
+generation_pipeline/**/__pycache__/
+runs/
+runs_*/
+experiments/
+experiments_*/
+.cache/stage5/
+
 # OS
 Thumbs.db
 

diff --git a/README.md b/README.md
@@ -1,82 +1,136 @@
 <div align="center">
   <img src="docs/img/new-HS-bench_logo.png" alt="HumanStudy-Bench Logo" width="300">
 
-  <h1>HumanStudy-Bench: Community Edition</h1>
-  <p><em>Open community-driven expansion of the HumanStudy-Bench benchmark</em></p>
-
-  <a href="https://arxiv.org/abs/2602.00685"><img src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv&logoColor=white" alt="Read the Paper" /></a>
-  <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT" /></a>
-  <a href="https://github.com/HumanStudy-Hub/HumanStudy-Bench/pulls"><img src="https://img.shields.io/badge/contributions-welcome-brightgreen.svg?logo=opensourceinitiative&logoColor=white" alt="Contributions Welcome" /></a>
-  <a href="https://www.hs-bench.clawder.ai"><img src="https://img.shields.io/badge/docs-website-blue?logo=readthedocs&logoColor=white" alt="Docs" /></a>
+  <h1>HumanStudy-Bench</h1>
+  <p><em>A benchmark for evaluating AI agents through runnable human studies</em></p>
 
+  <a href="https://arxiv.org/abs/2602.00685"><img src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv&logoColor=white" alt="Read the Paper"></a>
+  <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="MIT License"></a>
+  <a href="https://github.com/HumanStudy-Hub/HumanStudy-Bench/pulls"><img src="https://img.shields.io/badge/contributions-welcome-brightgreen.svg" alt="Contributions welcome"></a>
 </div>
 
 ---
 
-> LLMs are increasingly used to simulate human participants in social science research, but existing evaluations conflate base model capabilities with agent design choices, making it unclear whether results reflect the model or the configuration.
+HumanStudy-Bench is a standardized testbed for replaying human-subject
+experiments with AI agents. It combines:
+
+- an execution engine that reconstructs published experimental protocols;
+- a growing collection of runnable social-science studies; and
+- evaluation at the level of behavior, effects, and scientific findings.
+
+The 12 foundational studies cover cognition, strategic interaction, and social
+psychology. Community studies live alongside them under [`studies/`](studies/).
 
-## <img src="https://api.iconify.design/lucide/book-open.svg?color=%230891b2" width="20" height="20" /> Overview
+## Add a study
 
-HumanStudy-Bench treats participant simulation as an *agent design problem* and provides a **standardized testbed** — combining an **Execution Engine** that reconstructs full experimental protocols from published studies and a **Benchmark** with standardized evaluation metrics — for *replaying human-subject experiments end-to-end* with alignment evaluation at the level of scientific inference.
+There are two contribution paths.
 
-## <img src="https://api.iconify.design/lucide/git-pull-request.svg?color=%230891b2" width="20" height="20" /> How to Contribute a Study
+| Path | Best for | Current status |
+|---|---|---|
+| **Build Study** | Researchers starting from a paper PDF and optional open materials | **Private beta.** HumanStudy-Hub runs this repository's pipeline, persists review decisions, and produces a ZIP or GitHub pull request. |
+| **Direct pull request** | Contributors who want full control over study files | **Available now.** Build the folder locally, run validation, and open a PR. |
+
+The intended Build Study workflow is:
+
+```text
+paper PDF + optional OSF/open materials
+  -> evidence and study extraction
+  -> researcher review at uncertain decisions
+  -> runnable HumanStudy-Bench folder
+  -> download ZIP and/or save to GitHub
+```
 
-### 1. Fork and clone
+Build Study is currently operated through the private HumanStudy-Hub web
+application. The direct contribution documentation remains supported for
+researchers who prefer to author or inspect every file themselves. The manual
+entry point is the [direct pull request guide](docs/submit_study.md).
+
+### Direct pull request
+
+1. Fork this repository and clone your fork.
 
 ```bash
 git clone https://github.com/<your-github-id>/HumanStudy-Bench.git
 cd HumanStudy-Bench
-git checkout -b contrib-<yourgithubid>-013
+git checkout -b contrib-<your-github-id>-<study-name>
 ```
 
-### 2. Create your study folder
-
-Add a new directory under `studies/` with the required folders:
+2. Add a study folder.
 
+```text
+studies/<contributor-id>_<study-id>/
+  index.json
+  README.md
+  source/
+  scripts/
 ```
-studies/<yourgithubid>_013/
-  ├── index.json
-  ├── source/
-  ├── scripts/
-  └── README.md
+
+3. Validate it locally.
+
+```bash
+bash scripts/verify_study.sh <contributor-id>_<study-id>
 ```
 
-See the docs below for what goes inside each folder and the exact schemas:
+4. Commit the study, push your branch, and open a pull request targeting
+   `main`. CI validates the package and contributor attribution; maintainers
+   complete the human review and assign the final study ID.
 
-| # | Guide | Description |
-|---|-------|-------------|
-| 1 | [What Should I Submit?](https://www.hs-bench.clawder.ai/docs/what_to_submit) | Overview of contribution, required folders and files |
-| 2 | [How to Extract Data from a Paper](https://www.hs-bench.clawder.ai/docs/extract_from_paper) | Paper hierarchy, AI extraction prompt, walkthrough example |
-| 3 | [How to Build Your Study Files](https://www.hs-bench.clawder.ai/docs/build_study_files) | Schemas, code examples, and contracts for each file |
-| 4 | [How to Submit Your Study](https://www.hs-bench.clawder.ai/docs/submit_study) | Fork, verify, push, and open a PR |
+Contributor guides:
 
-### 3. Verify locally
+- [What should I submit?](docs/what_to_submit.md)
+- [How to extract data from a paper](docs/extract_from_paper.md)
+- [How to build study files](docs/build_study_files.md)
+- [How to submit a study](docs/submit_study.md)
 
-```bash
-bash scripts/verify_study.sh <yourgithubid>_013
-```
+## Generation pipeline
 
-### 4. Commit and push
+This repository includes a human-in-the-loop pipeline for drafting a study
+folder from a paper PDF and optional OSF or supplementary sources.
 
-```bash
-git add studies/<yourgithubid>_013/
-git commit -m "Add study: <Your Study Title>"
-git push origin contrib-<yourgithubid>-013
-```
+1. Inventory empirical units and comparison groups.
+2. Extract findings, effects, samples, and statistics.
+3. Assemble and audit participant-facing materials.
+4. Write a package under `studies/<study_id>/`.
+5. Run a simulation only after the package passes readiness review.
 
-### 5. Open a Pull Request
+Install the generation dependencies:
 
-Open a PR on GitHub targeting the `main` branch. Maintainers assign final `study_XXX` numbering by merge order. CI runs validation automatically; confirmation is by human review.
+```bash
+python -m pip install -e ".[llm,pdf]"
+cp config/settings.example.yaml config/settings.yaml
+```
 
-You can also submit a study via **web upload** at [hs-bench.clawder.ai/contribute](https://www.hs-bench.clawder.ai/contribute).
+Do not commit `config/settings.yaml`.
 
-## <img src="https://api.iconify.design/lucide/flask-conical.svg?color=%230891b2" width="20" height="20" /> Existing Studies
+Run a stage with:
 
-The 12 foundational studies (cognition, strategic interaction, social psychology) serve as reference examples. Browse them on the [website](https://www.hs-bench.clawder.ai/contribute#studies) or locally under `studies/`.
+```bash
+python generation_pipeline/run.py \
+  --settings config/settings.yaml \
+  --stage 1 \
+  --pdf /absolute/path/to/paper.pdf \
+  --provider openai \
+  --model gpt-5-mini
+```
 
-## <img src="https://api.iconify.design/lucide/quote.svg?color=%230891b2" width="20" height="20" /> Citation
+Generated evidence and review artifacts are written under
+`generation_pipeline/outputs/<paper_id>/`. Pipeline-generated packages remain
+blocked from simulation until `source/audit.json` marks them ready. The pipeline
+entry point is [`generation_pipeline/run.py`](generation_pipeline/run.py); run
+it with `--help` for the full stage and review options.
+
+## Repository structure
+
+```text
+studies/               Runnable benchmark studies
+src/                   Execution and evaluation code
+generation_pipeline/   PDF/material extraction and package generation
+scripts/               Study validation and index utilities
+docs/                  Manual contribution guides
+tests/                 Pipeline and benchmark tests
+```
 
-If you use HumanStudy-Bench, please cite:
+## Citation
 
 ```bibtex
 @misc{liu2026humanstudybenchaiagentdesign,
@@ -90,6 +144,6 @@ If you use HumanStudy-Bench, please cite:
 }
 ```
 
-## <img src="https://api.iconify.design/lucide/scale.svg?color=%230891b2" width="20" height="20" /> License
+## License
 
-MIT License. See [LICENSE](LICENSE) for details.
+MIT License. See [LICENSE](LICENSE).
diff --git a/co_website/.gitignore b/co_website/.gitignore
diff --git a/co_website/README.md b/co_website/README.md
diff --git a/co_website/adm-zip.d.ts b/co_website/adm-zip.d.ts