Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: validate-corpus

on:
pull_request:
push:
branches: [main]

permissions:
contents: read

jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: pip
cache-dependency-path: requirements-dev.txt
- name: Install validation dependencies
run: python -m pip install -r requirements-dev.txt
- name: Validate schemas, records, and references
run: python tools/validate.py
9 changes: 9 additions & 0 deletions data/claims/deepseek-v2-uses-deepseekmoe.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:deepseek-v2-uses-deepseekmoe
type: claim
subject: release:deepseek-v2
predicate: uses_technique
object: technique:deepseek-moe
status: confirmed_primary
evidence:
- source: source:deepseek-v2-paper
locator: Abstract
9 changes: 9 additions & 0 deletions data/claims/deepseek-v2-uses-mla.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:deepseek-v2-uses-mla
type: claim
subject: release:deepseek-v2
predicate: uses_technique
object: technique:multi-head-latent-attention
status: confirmed_primary
evidence:
- source: source:deepseek-v2-paper
locator: Abstract
9 changes: 9 additions & 0 deletions data/claims/deepseek-v3-total-parameters.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:deepseek-v3-total-parameters
type: claim
subject: release:deepseek-v3
predicate: total_parameters
value: 671000000000
status: confirmed_primary
evidence:
- source: source:deepseek-v3-report
locator: Abstract
9 changes: 9 additions & 0 deletions data/claims/deepseek-v3-uses-deepseekmoe.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:deepseek-v3-uses-deepseekmoe
type: claim
subject: release:deepseek-v3
predicate: uses_technique
object: technique:deepseek-moe
status: confirmed_primary
evidence:
- source: source:deepseek-v3-report
locator: Abstract
9 changes: 9 additions & 0 deletions data/claims/deepseek-v3-uses-mla.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:deepseek-v3-uses-mla
type: claim
subject: release:deepseek-v3
predicate: uses_technique
object: technique:multi-head-latent-attention
status: confirmed_primary
evidence:
- source: source:deepseek-v3-report
locator: Abstract
9 changes: 9 additions & 0 deletions data/claims/deepseekmoe-extends-sparse-moe.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:deepseekmoe-extends-sparse-moe
type: claim
subject: technique:deepseek-moe
predicate: extends
object: technique:sparse-mixture-of-experts
status: confirmed_primary
evidence:
- source: source:deepseekmoe-paper
locator: Abstract
9 changes: 9 additions & 0 deletions data/claims/gqa-generalizes-mqa.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:gqa-generalizes-mqa
type: claim
subject: technique:grouped-query-attention
predicate: generalizes
object: technique:multi-query-attention
status: confirmed_primary
evidence:
- source: source:gqa-paper
locator: Abstract
11 changes: 11 additions & 0 deletions data/claims/llama-2-parameter-range.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
id: claim:llama-2-parameter-range
type: claim
subject: release:llama-2
predicate: parameter_range
value:
minimum: 7000000000
maximum: 70000000000
status: confirmed_primary
evidence:
- source: source:llama-2-paper
locator: Abstract
9 changes: 9 additions & 0 deletions data/claims/llama-2-release-of-family.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:llama-2-release-of-family
type: claim
subject: release:llama-2
predicate: release_of
object: family:llama
status: confirmed_primary
evidence:
- source: source:llama-2-paper
locator: Abstract
9 changes: 9 additions & 0 deletions data/claims/llama-3-context-window.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:llama-3-context-window
type: claim
subject: release:llama-3
predicate: context_window_tokens
value: 128000
status: confirmed_primary
evidence:
- source: source:llama-3-paper
locator: Abstract
9 changes: 9 additions & 0 deletions data/claims/llama-3-largest-model-parameters.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:llama-3-largest-model-parameters
type: claim
subject: release:llama-3
predicate: largest_model_parameters
value: 405000000000
status: confirmed_primary
evidence:
- source: source:llama-3-paper
locator: Abstract
9 changes: 9 additions & 0 deletions data/claims/llama-3-release-of-family.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:llama-3-release-of-family
type: claim
subject: release:llama-3
predicate: release_of
object: family:llama
status: confirmed_primary
evidence:
- source: source:llama-3-paper
locator: Abstract
9 changes: 9 additions & 0 deletions data/claims/mqa-variant-of-mha.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
id: claim:mqa-variant-of-mha
type: claim
subject: technique:multi-query-attention
predicate: variant_of_technique
object: technique:multi-head-attention
status: confirmed_primary
evidence:
- source: source:multi-query-attention
locator: Abstract
4 changes: 4 additions & 0 deletions data/families/deepseek.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
id: family:deepseek
type: model_family
name: DeepSeek
status: draft
4 changes: 4 additions & 0 deletions data/families/llama.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
id: family:llama
type: model_family
name: Llama
status: draft
7 changes: 7 additions & 0 deletions data/relationships/deepseek-v2-uses-deepseekmoe.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
id: relationship:deepseek-v2-uses-deepseekmoe
type: relationship
subject: release:deepseek-v2
predicate: uses_technique
object: technique:deepseek-moe
claim_refs:
- claim:deepseek-v2-uses-deepseekmoe
7 changes: 7 additions & 0 deletions data/relationships/deepseek-v2-uses-mla.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
id: relationship:deepseek-v2-uses-mla
type: relationship
subject: release:deepseek-v2
predicate: uses_technique
object: technique:multi-head-latent-attention
claim_refs:
- claim:deepseek-v2-uses-mla
7 changes: 7 additions & 0 deletions data/relationships/deepseek-v3-uses-deepseekmoe.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
id: relationship:deepseek-v3-uses-deepseekmoe
type: relationship
subject: release:deepseek-v3
predicate: uses_technique
object: technique:deepseek-moe
claim_refs:
- claim:deepseek-v3-uses-deepseekmoe
7 changes: 7 additions & 0 deletions data/relationships/deepseek-v3-uses-mla.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
id: relationship:deepseek-v3-uses-mla
type: relationship
subject: release:deepseek-v3
predicate: uses_technique
object: technique:multi-head-latent-attention
claim_refs:
- claim:deepseek-v3-uses-mla
7 changes: 7 additions & 0 deletions data/relationships/llama-2-release-of-llama.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
id: relationship:llama-2-release-of-llama
type: relationship
subject: release:llama-2
predicate: release_of
object: family:llama
claim_refs:
- claim:llama-2-release-of-family
7 changes: 7 additions & 0 deletions data/relationships/llama-3-release-of-llama.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
id: relationship:llama-3-release-of-llama
type: relationship
subject: release:llama-3
predicate: release_of
object: family:llama
claim_refs:
- claim:llama-3-release-of-family
7 changes: 7 additions & 0 deletions data/relationships/mha-precedes-mqa.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
id: relationship:mha-precedes-mqa
type: relationship
subject: technique:multi-head-attention
predicate: conceptual_predecessor_of
object: technique:multi-query-attention
claim_refs:
- claim:mqa-variant-of-mha
7 changes: 7 additions & 0 deletions data/relationships/mqa-precedes-gqa.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
id: relationship:mqa-precedes-gqa
type: relationship
subject: technique:multi-query-attention
predicate: conceptual_predecessor_of
object: technique:grouped-query-attention
claim_refs:
- claim:gqa-generalizes-mqa
7 changes: 7 additions & 0 deletions data/relationships/sparse-moe-precedes-deepseekmoe.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
id: relationship:sparse-moe-precedes-deepseekmoe
type: relationship
subject: technique:sparse-mixture-of-experts
predicate: conceptual_predecessor_of
object: technique:deepseek-moe
claim_refs:
- claim:deepseekmoe-extends-sparse-moe
4 changes: 4 additions & 0 deletions data/releases/deepseek-v2.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
id: release:deepseek-v2
type: model_release
name: DeepSeek-V2
status: draft
4 changes: 4 additions & 0 deletions data/releases/deepseek-v3.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
id: release:deepseek-v3
type: model_release
name: DeepSeek-V3
status: draft
4 changes: 4 additions & 0 deletions data/releases/llama-2.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
id: release:llama-2
type: model_release
name: Llama 2
status: draft
4 changes: 4 additions & 0 deletions data/releases/llama-3.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
id: release:llama-3
type: model_release
name: Llama 3
status: draft
8 changes: 8 additions & 0 deletions data/sources/deepseek-v2-paper.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
id: source:deepseek-v2-paper
type: source
title: "DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model"
source_type: technical_report
url: https://arxiv.org/abs/2405.04434
publisher: arXiv
published_at: '2024-05-07'
retrieved_at: '2026-09-29T06:10:00Z'
8 changes: 8 additions & 0 deletions data/sources/deepseek-v3-report.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
id: source:deepseek-v3-report
type: source
title: DeepSeek-V3 Technical Report
source_type: technical_report
url: https://arxiv.org/abs/2412.19437
publisher: arXiv
published_at: '2024-12-27'
retrieved_at: '2026-09-29T06:10:00Z'
8 changes: 8 additions & 0 deletions data/sources/deepseekmoe-paper.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
id: source:deepseekmoe-paper
type: source
title: "DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models"
source_type: paper
url: https://arxiv.org/abs/2401.06066
publisher: arXiv
published_at: '2024-01-11'
retrieved_at: '2026-09-29T06:10:00Z'
8 changes: 8 additions & 0 deletions data/sources/gqa-paper.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
id: source:gqa-paper
type: source
title: "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints"
source_type: paper
url: https://arxiv.org/abs/2305.13245
publisher: arXiv
published_at: '2023-05-22'
retrieved_at: '2026-09-29T06:10:00Z'
8 changes: 8 additions & 0 deletions data/sources/llama-2-paper.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
id: source:llama-2-paper
type: source
title: "Llama 2: Open Foundation and Fine-Tuned Chat Models"
source_type: technical_report
url: https://arxiv.org/abs/2307.09288
publisher: arXiv
published_at: '2023-07-18'
retrieved_at: '2026-09-29T06:25:00Z'
8 changes: 8 additions & 0 deletions data/sources/llama-3-paper.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
id: source:llama-3-paper
type: source
title: The Llama 3 Herd of Models
source_type: technical_report
url: https://arxiv.org/abs/2407.21783
publisher: arXiv
published_at: '2024-07-31'
retrieved_at: '2026-09-29T06:25:00Z'
8 changes: 8 additions & 0 deletions data/sources/multi-query-attention.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
id: source:multi-query-attention
type: source
title: "Fast Transformer Decoding: One Write-Head is All You Need"
source_type: paper
url: https://arxiv.org/abs/1911.02150
publisher: arXiv
published_at: '2019-11-06'
retrieved_at: '2026-09-29T06:10:00Z'
4 changes: 4 additions & 0 deletions data/techniques/deepseek-moe.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
id: technique:deepseek-moe
type: technique
name: DeepSeekMoE
status: draft
6 changes: 6 additions & 0 deletions data/techniques/grouped-query-attention.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
id: technique:grouped-query-attention
type: technique
name: Grouped-Query Attention
aliases:
- GQA
status: draft
6 changes: 6 additions & 0 deletions data/techniques/multi-head-attention.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
id: technique:multi-head-attention
type: technique
name: Multi-Head Attention
aliases:
- MHA
status: draft
6 changes: 6 additions & 0 deletions data/techniques/multi-head-latent-attention.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
id: technique:multi-head-latent-attention
type: technique
name: Multi-Head Latent Attention
aliases:
- MLA
status: draft
6 changes: 6 additions & 0 deletions data/techniques/multi-query-attention.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
id: technique:multi-query-attention
type: technique
name: Multi-Query Attention
aliases:
- MQA
status: draft
6 changes: 6 additions & 0 deletions data/techniques/sparse-mixture-of-experts.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
id: technique:sparse-mixture-of-experts
type: technique
name: Sparse Mixture-of-Experts
aliases:
- Sparse MoE
status: draft
13 changes: 13 additions & 0 deletions docs/seed-corpus.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Seed corpus v0.1

The first ModelMap corpus is intentionally small. Its job is to exercise the ontology and provenance chain before broad ingestion.

It establishes three proof paths:

1. **attention lineage:** multi-head attention → multi-query attention → grouped-query attention;
2. **latent attention adoption:** DeepSeek-V2 and DeepSeek-V3 using multi-head latent attention;
3. **sparse-compute lineage:** sparse mixture-of-experts → DeepSeekMoE, followed by DeepSeek-V2/V3 adoption.

Every graph edge in this seed is backed by an atomic claim and a primary paper or technical report. The seed deliberately does **not** create a direct lineage edge from GQA to MLA: shared motivation or chronology is not sufficient evidence for ancestry.

Primary sources represented: arXiv:1911.02150, arXiv:2305.13245, arXiv:2401.06066, arXiv:2405.04434, and arXiv:2412.19437.
2 changes: 2 additions & 0 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
jsonschema>=4.23,<5
PyYAML>=6.0,<7
1 change: 1 addition & 0 deletions schemas/relationship.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"checkpoint_derived_from",
"fine_tuned_from",
"distilled_from",
"conceptual_predecessor_of",
"uses_architecture",
"uses_technique",
"introduces_technique",
Expand Down
Loading
Loading