Thirst-circuit retrieval-augmented generation over PubMed Central Open Access JATS XML.
This is not a trained GPT. There are no model weights in this repository. The current deliverable is a reproducible OA corpus plus a script that turns PMCIDs into a parquet table (pmcid, title, abstract, body).
Drosophila melanogaster papers on thirst, NPF/NPFR, Allatostatin A, PAM/mushroom-body memory, pC1, and oviDN/oviIN — mined from native PMC XML so we skip PDF-to-text conversion.
These counts are different sets. Do not collapse them.
| Set | n (as of 2026-08-29, re-count on disk) | Where |
|---|---|---|
| Mixed EFetch dump (unique PMCIDs) | 307 | local Temp only — not in git |
Mixed EFetch files with <body> |
283 / 308 files (the extra file is a Landayan duplicate) | same dump |
| Mixed EFetch body-less / < 20 KB | 25 | manifests/dropped_nobody_efetch.csv |
| pubget Open Access subset | 246 / 307 | manifests/corpus_v0.csv |
OA with <body> |
245 | manifests/corpus_v0.pmcids.txt (build list) |
OA without <body> |
1 — PMC13342866 |
listed in the OA CSV, not in the build list |
XML lives only on disk under data/ (gitignored). Git tracks the manifest, not the articles.
You need Python 3.11+, uv or pip, and network access to NCBI E-utilities (no campus VPN required for OA XML). Use a project-local .venv. Do not install these packages into a Hermes environment.
cd FlyBrainGPT
uv venv --python 3.11
uv pip install -e ".[dev]"
# Optional: rebuild XML from the OA PMCID list (writes data/corpus_v0/, gitignored)
uv run python scripts/build_corpus.py \
--pmcids manifests/corpus_v0.pmcids.txt \
--out data/corpus_v0/corpus.parquet \
--xml-dir data/corpus_v0/xml
uv run pytestNCBI asks for ≤ 3 EFetch requests/second without an API key. Set NCBI_API_KEY if you have one.
Landayan et al. 2021 (eLife) is the CI fixture: DOI 10.7554/eLife.66286, PMC8139827, PMID 34018925.
- Paywalled publisher PDFs (including many Nature papers), even if a university VPN can open them in a browser.
- Non-OA PMC author manuscripts. EFetch can still return JATS for some of those; corpus_v0 does not include them. pubget kept 246 / 307 PMCIDs as Open Access (~61 author manuscripts stayed out).
- Papers without a PMCID (no NCBI full-text XML).
- The 25 mixed-EFetch files without a
<body>/ under 20 KB (manifests/dropped_nobody_efetch.csv). That 25 is not the OA-subset 1. - Four grey-list misclassifications (not D. melanogaster primary research) in
manifests/dropped_grey.csv:- mosquito AstA receptors
10.1016/j.peptides.2017.10.016 - mammalian heterochromatin
10.1016/s1097-2765(01)00218-0 - mouse landscape diffusion
10.1038/s41586-023-06715-z(Richman et al., Nature) - hypothalamic autophagy/NPY
10.1101/2025.06.20.660653
- mosquito AstA receptors
- PDF-derived text from paper-search-mcp.
- Sci-Hub or any non-OA bulk download.
- pubget
articleset_*.xmland any Temp JATS dump — never commit those.
- Code in this repository: MIT (see
LICENSE). - Article XML: remains under each paper’s PMC / publisher license (often CC BY). Do not relicense the papers as MIT. Do not commit the XML.
manifests/ # committed: PMCID list, SHA256, DOI, OA, has_body, drop lists
scripts/ # committed: build_corpus.py
tests/fixtures/ # committed: one OA JATS fixture (Landayan 2021)
data/ # gitignored: local XML + parquet
| Piece | State |
|---|---|
| Repo skeleton | main, MIT on code |
| corpus_v0 manifest | 246 OA PMCIDs; 245 in the build list |
| Grey-list + body-less drops | committed CSVs; builder skips them |
| Retrieval eval (Janu-AstA, L1-l, oviIN, NPF sign) | not started |
| Model weights | none |
drosophila · neuroscience · rag · pubmed-central · flybrain · open-access