Harden plugin metadata, retrieval, and verification

.github/workflows/ci.yml

@@ -0,0 +1,24 @@
+name: CI
+
+on:
+  push:
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+      - run: pip install -r requirements-dev.txt
+      - run: pytest -q
+      - run: python -m compileall -q mcp_server.py

.gitignore

@@ -0,0 +1,4 @@
+.venv/
+__pycache__/
+*.py[cod]
+.pytest_cache/

INTEGRATIONS.md

@@ -225,7 +225,7 @@ Create `/path/to/mcp_server.py`:
 #!/usr/bin/env python3
 """
 Machine Learning Systems — MCP Server
-Exposes the complete textbook (44 chapters, 111 definitions, 820 artifacts)
+Exposes the reference corpus (44 chapter files, 111 definitions, 816 indexed artifacts)
 as MCP tools, resources, and prompts.
 """
 
@@ -265,7 +265,7 @@ def glossary() -> str:
 
 @mcp.resource("ml-systems://patterns")
 def patterns() -> str:
-    """All 820 artifacts across 8 categories."""
+    """All 816 indexed artifacts across 8 categories."""
     return (CHAPTERS_DIR / "patterns.md").read_text()
 
 @mcp.resource("ml-systems://cheatsheet")
@@ -472,7 +472,7 @@ fastmcp install claude-desktop mcp_server.py
 |----------|-----|---------|
 | Chapter Index | `ml-systems://index` | All 44 chapters metadata |
 | Glossary | `ml-systems://glossary` | 111 definitions |
-| Patterns | `ml-systems://patterns` | 820 artifacts |
+| Patterns | `ml-systems://patterns` | 816 indexed artifacts |
 | Cheatsheet | `ml-systems://cheatsheet` | Formulas, rules |
 
 | Prompt | Use Case |
@@ -507,4 +507,4 @@ fastmcp install claude-desktop mcp_server.py
 
 ## 📚 Source Attribution
 
-All content sourced from **"Machine Learning Systems"** by **Professor Vijay Janapa Reddi** (Harvard SEAS), course **CS 249r**, open-source at **[mlsysbook.ai](https://mlsysbook.ai/)** (CC BY-NC-SA 4.0). Plugin extraction and MCP server are MIT-licensed.
+The reference content is adapted from **"Machine Learning Systems"** by **Professor Vijay Janapa Reddi** (Harvard SEAS), course **CS 249r**, at **[mlsysbook.ai](https://mlsysbook.ai/)**. This redistributed/adapted package, including its plugin and MCP integration, is licensed under CC BY-NC-SA 4.0; third-party dependencies retain their own licenses.

LICENSE

@@ -0,0 +1,13 @@
+Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International
+
+This repository is a derivative of "Machine Learning Systems" by Vijay
+Janapa Reddi, available at https://mlsysbook.ai/ and licensed under
+CC BY-NC-SA 4.0.
+
+Repository-specific organization, indexes, MCP server, metadata, and tests:
+Copyright (c) 2026 Collins Nyagaka.
+
+SPDX-License-Identifier: CC-BY-NC-SA-4.0
+
+License summary: https://creativecommons.org/licenses/by-nc-sa/4.0/
+Legal code: https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode

README.md

@@ -1,261 +1,67 @@
-# Machine Learning Systems Plugin
+# Machine Learning Systems Reference Plugin
 
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-[![Plugin Version](https://img.shields.io/badge/version-1.1.0-blue.svg)]()
-[![Chapters](https://img.shields.io/badge/chapters-44-brightgreen.svg)]()
-[![Artifacts](https://img.shields.io/badge/named_artifacts-820-orange.svg)]()
+[![License: CC BY-NC-SA 4.0](https://img.shields.io/badge/License-CC_BY--NC--SA_4.0-lightgrey.svg)](LICENSE)
+[![CI](https://github.com/CollinsNyatundo/machine-learning-systems-plugin/actions/workflows/ci.yml/badge.svg)](https://github.com/CollinsNyatundo/machine-learning-systems-plugin/actions/workflows/ci.yml)
 
-A comprehensive study and reference plugin for **"Machine Learning Systems"** — the open-source textbook by **Professor Vijay Janapa Reddi** ([Harvard SEAS](https://vijay.seas.harvard.edu/)), covering the principles and practices of engineering artificially intelligent systems.
+A searchable study and reference package derived from Vijay Janapa Reddi's open textbook, [Machine Learning Systems](https://mlsysbook.ai/). It exposes the material as Markdown and through a small FastMCP server.
 
-Based on the [CS 249r course](https://github.com/harvard-edge/cs249r_book) and the official textbook at **[mlsysbook.ai](https://mlsysbook.ai/)**.
+This repository is an independently organized derivative reference. It is not the original textbook, an official Harvard project, or an operational MLOps platform.
 
----
+## Contents
 
-## 📚 About the Source Material
+- 44 chapter and appendix files across two volumes
+- 111 glossary entries
+- 816 indexed learning artifacts
+- A formulas-and-concepts cheatsheet
+- FastMCP resources and tools for search and retrieval
 
-| Volume | Title | Focus | Chapters |
-|--------|-------|-------|----------|
-| **Vol 1** | Introduction to Machine Learning Systems | Foundations, single-machine ML, data engineering, training, serving | 16 + 5 appendices |
-| **Vol 2** | Machine Learning Systems at Scale | Distributed training, fleet orchestration, inference at scale, governance | 18 + 6 appendices |
+## Run locally
 
-**Author**: Professor Vijay Janapa Reddi, Harvard John A. Paulson School of Engineering and Applied Sciences  
-**Course**: CS 249r — Machine Learning Systems  
-**License**: The textbook is open-source (CC BY-NC-SA 4.0). This plugin is MIT-licensed for the extracted reference format.
-
----
-
-## 📦 What's Included
-
-### 44 Chapter Files (Complete Textbook Content)
-```
-Vol 1 — Foundations (21 files)
-├── ch01.md  — Introduction to ML Systems
-├── ch02.md  — Compute Infrastructure
-├── ch03.md  — Neural Computation
-├── ch04.md  — Data Engineering & Pipeline Architecture
-├── ch05.md  — Network Architectures
-├── ch06.md  — ML Frameworks
-├── ch07.md  — Model Training
-├── ch08.md  — Data Selection & Efficient Training
-├── ch09.md  — Model Compression
-├── ch10.md  — Benchmarking & Profiling
-├── ch11.md  — Hardware Acceleration
-├── ch12.md  — ML Operations
-├── ch13.md  — Model Serving
-├── ch14.md  — Responsible Engineering
-├── ch15.md  — Sustainable AI
-├── ch16.md  — Conclusion
-├── appA.md  — D·A·M Taxonomy
-├── appB.md  — Data Foundations
-├── appC.md  — Algorithm Foundations
-├── appD.md  — Machine Foundations
-└── appE.md  — System Assumptions
-
-Vol 2 — Distribution, Serving & Governance (23 files)
-├── v2_ch01.md  — Introduction to ML Systems (Fleet Scale)
-├── v2_ch02.md  — Compute Infrastructure (Fleet Scale)
-├── v2_ch03.md  — Network Fabrics
-├── v2_ch05.md  — Data Storage — The Fuel Line
-├── v2_ch06.md  — Distributed Training Systems
-├── v2_ch07.md  — Collective Communication
-├── v2_ch08.md  — Fault Tolerance and Reliability
-├── v2_ch09.md  — Fleet Orchestration
-├── v2_ch10.md  — Performance Engineering
-├── v2_ch11.md  — Inference at Scale
-├── v2_ch12.md  — Edge Intelligence
-├── v2_ch13.md  — ML Operations at Scale
-├── v2_ch14.md  — Security & Privacy
-├── v2_ch15.md  — Robust AI
-├── v2_ch16.md  — Sustainable AI
-├── v2_ch17.md  — Responsible Engineering
-├── v2_ch18.md  — Conclusion
-├── v2_appA.md  — Single-Machine Foundations
-├── v2_appB.md  — Fleet Foundations
-├── v2_appC.md  — Communication Foundations
-├── v2_appD.md  — Reliability Foundations
-├── v2_appE.md  — The C³ Taxonomy
-└── v2_appF.md  — System Assumptions
-```
-
-> **Note**: The source textbook Vol 2 has no chapter 3 content (skips from ch03 to ch04). This plugin combines both ch03 and ch04 into 1 (ch03).
-
-### 4 Top-Level Reference Files
-| File | Description |
-|------|-------------|
-| `SKILL.md` | Plugin manifest with chapter index, artifact counts, installation guide |
-| `glossary.md` | **111 definitions** — alphabetical, disambiguated Vol 1/Vol 2 |
-| `patterns.md` | **820 named artifacts** across 8 categories (see below) |
-| `cheatsheet.md` | Key formulas, quantitative rules of thumb, hardware constants |
-
-### 820 Named Artifacts (Extracted & Categorized)
-| Category | Count | Description |
-|----------|-------|-------------|
-| 🧮 Napkin Math | 192 | Worked quantitative examples |
-| ✅ Checkpoints | 130 | Self-check questions |
-| 🔭 Systems Perspectives | 114 | Architectural insights |
-| ⚖️ Principles | 50 | Invariants & laws |
-| 💥 War Stories | 41 | Production failures |
-| 🏮 Lighthouses | 60 | Reference workloads |
-| 📋 Examples | 87 | Applied cases |
-| 📖 Definitions | 142 | Formal definitions |
-| **Total** | **820** | |
-
----
-
-## 🚀 Installation
-
-### Gemini CLI (Primary Target)
 ```bash
-# Clone or download this repo
 git clone https://github.com/CollinsNyatundo/machine-learning-systems-plugin.git
-
-# Install to Gemini CLI plugins directory
-cp -r machine-learning-systems-plugin/skills/machine-learning-systems \
-      ~/.gemini/config/plugins/
-
-# Verify
-ls ~/.gemini/config/plugins/machine-learning-systems/skills/machine-learning-systems/chapters/
-```
-
-### Cursor / Windsurf
-```bash
-cp -r machine-learning-systems-plugin/skills/machine-learning-systems \
-      .cursor/plugins/
-```
-
-### MCP Server (Programmatic Access)
-
-The repository includes a ready-to-use Model Context Protocol (MCP) server (`mcp_server.py`) at the root, allowing programmatic integration with tools like Cursor, Windsurf, Claude Code, and Claude Desktop.
-
-To run the server directly:
-```bash
+cd machine-learning-systems-plugin
+python -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
 python mcp_server.py
 ```
 
-To install it directly via the `fastmcp` CLI:
+FastMCP can also install the server into supported clients:
+
 ```bash
-# Install to Claude Desktop
 fastmcp install claude-desktop mcp_server.py
-
-# Install to Cursor
 fastmcp install cursor mcp_server.py
-
-# Install to Claude Code
 fastmcp install claude-code mcp_server.py
 ```
 
-### Manual / Any Editor
-```bash
-# Just copy the chapters folder anywhere
-cp -r machine-learning-systems-plugin/skills/machine-learning-systems/chapters \
-      ~/my-ml-reference/
-```
-
----
-
-## 🔍 Usage
-
-### In Gemini CLI
-```
-# List available skills
-/skills
-
-# The plugin provides these skills automatically:
-# - machine-learning-systems (full textbook reference)
-# - Individual chapter skills via chapter-naming
-```
-
-### As Reference Documentation
-- **Search**: Use `Ctrl+F` in any `.md` file
-- **Cross-references**: Patterns/glossary link back to source chapters
-- **Formulas**: Cheatsheet has LaTeX-rendered equations
-- **Definitions**: Glossary is alphabetical with Vol 1/Vol 2 disambiguation
-
-### Key Formulas Available (Cheatsheet)
-- **Iron Law**: `T = D_vol/BW + O/(R_peak·η_hw) + L_lat`
-- **Arithmetic Intensity**: `AI = FLOPs / Bytes`
-- **MFU**: `Achieved FLOP/s / Peak FLOP/s`
-- **Ridge Point**: `R = R_peak / BW`
-- **Young-Daly**: `τ_opt = √(2·T_write·MTBF)`
-- **KL Divergence**: `D_KL(P∥Q) = Σ P(i) log(P(i)/Q(i))`
-
----
-
-## 🛠 How This Was Built
+Client support and syntax change over time; consult the client's current documentation. See [INTEGRATIONS.md](INTEGRATIONS.md) for configuration examples. Its HTTP examples require authentication, transport security, and network hardening before exposure outside a trusted machine.
 
-1. **Source Extraction**: Docling PDF → markdown slices (via Antigravity / docling-serve)
-2. **Deterministic Merge**: Python script `merge_vol1_chapters.py` — no LLM in the loop
-   - Preserves every named artifact (Definition, Napkin Math, Checkpoint, etc.)
-   - Maintains all tables, equations, code blocks, H2/H3 hierarchy
-   - Prepends original summary section ("Section-by-Section Preserve-and-Extend")
-3. **Parity Audit**: `audit_vol1_rebuild.py` verifies 100% artifact coverage
-4. **Top-Level Regeneration**: Patterns, glossary, cheatsheet extracted from final chapters
-5. **Math Placeholder & LaTeX Resolution**: Resolved 354 mathematical comment placeholders (`<!-- formula-not-decoded -->`) and raw dollar-sign math wrappers using a background validation pipeline. Enforced strict math delimiters (`\(` and `\)` for inline, `\[` and `\]` for display) and normalized Greek/Unicode characters across all 44 chapters.
-6. **Quality**: 113% average byte ratio (plugin ≥ slice), 0 missing artifacts
+## MCP surface
 
-**Artifacts preserved**: `/tmp/merge_vol1_chapters.py`, `/tmp/audit_vol1_rebuild.py`, `/tmp/backup_vol1/`
+| Type | Name | Purpose |
+|---|---|---|
+| Resource | `ml-systems://index` | Chapter metadata |
+| Resource | `ml-systems://chapter/{id}` | Chapter text |
+| Resource | `ml-systems://glossary` | Glossary |
+| Resource | `ml-systems://patterns` | Indexed artifacts |
+| Resource | `ml-systems://cheatsheet` | Quick reference |
+| Tool | `search` | Search the reference corpus |
+| Tool | `get_chapter` | Retrieve one chapter safely |
+| Tool | `list_chapters` | List chapter metadata |
+| Tool | `list_patterns` | Filter indexed artifacts |
 
----
+## Verify
 
-## 📖 Attribution & License
-
-| Component | License | Attribution |
-|-----------|---------|-------------|
-| **Source Textbook** | CC BY-NC-SA 4.0 | "Machine Learning Systems" by Vijay Janapa Reddi, Harvard SEAS — [mlsysbook.ai](https://mlsysbook.ai/) |
-| **Course Material** | Educational | CS 249r: Machine Learning Systems, Harvard |
-| **This Plugin** | CC BY-NC-SA 4.0 | Plugin format, extraction scripts, top-level indices — free to use, modify, distribute |
-| **Docling** | Apache 2.0 | PDF → markdown conversion — [ds4sd/docling](https://github.com/docling-project/docling) |
-
-### Proper Citation
-```
-@book{reddi2024mlsystems,
-  title = {Machine Learning Systems},
-  author = {Reddi, Vijay Janapa},
-  institution = {Harvard John A. Paulson School of Engineering and Applied Sciences},
-  course = {CS 249r},
-  year = {2024},
-  url = {https://mlsysbook.ai/},
-  note = {Open-source textbook, CC BY-NC-SA 4.0}
-}
-
-@software{mls_plugin_2025,
-  title = {Machine Learning Systems Plugin for Gemini CLI / Cursor / MCP},
-  author = {Nyatundo, Collins},
-  url = {https://github.com/CollinsNyatundo/machine-learning-systems-plugin},
-  license = {MIT},
-  note = {Deterministic extraction from docling slices, 820 artifacts, 44 chapters}
-}
+```bash
+pip install -r requirements-dev.txt
+pytest -q
+python -m compileall -q mcp_server.py
 ```
 
----
-
-## 🤝 Contributing
-
-1. **Source corrections**: Report to upstream textbook at [harvard-edge/cs249r_book](https://github.com/harvard-edge/cs249r_book)
-2. **Plugin improvements**: PRs welcome for:
-    - Additional top-level indices (timeline, concept map, etc.)
-    - Enhancement of MCP prompts and resources
-    - Better cross-references between Vol 1 ↔ Vol 2
-    - Search/index tooling
-
----
-
-## 🙏 Acknowledgments
-
-- **Prof. Vijay Janapa Reddi** — for writing and open-sourcing this exceptional textbook
-- **Harvard EDGE / CS 249r team** — for the course and materials
-- **Docling Project** — for best-in-class PDF understanding
-- **Antigravity** — for the deterministic rebuild pipeline
-- **Nous Research / Hermes** — for the agent framework used to build this
-
----
-
-## 📞 Support
+## Provenance
 
-- **Plugin issues**: [GitHub Issues](https://github.com/CollinsNyatundo/machine-learning-systems-plugin/issues)
-- **Textbook questions**: [mlsysbook.ai](https://mlsysbook.ai/) or [CS 249r discussions](https://github.com/harvard-edge/cs249r_book/discussions)
-- **Author**: Collins Nyatundo ([@CollinsNyatundo](https://github.com/CollinsNyatundo))
+The chapter Markdown was extracted and reorganized from the CC BY-NC-SA 4.0 source material. Repository-specific changes include the directory layout, indexes, search server, metadata, and tests. Corrections to the textbook itself should be reported to the [upstream project](https://github.com/harvard-edge/cs249r_book).
 
----
+## License
 
-*Generated from the complete "Machine Learning Systems" textbook via deterministic docling extraction. Every definition, formula, worked example, and architectural insight preserved from source.*
+The source and this derivative are licensed under [CC BY-NC-SA 4.0](LICENSE). You must provide attribution, restrict use to non-commercial purposes, and distribute adaptations under the same license.