docs: Diagram guide with schema grouping, collapse, and Mermaid examples

mkdocs.yaml

@@ -59,6 +59,7 @@ nav:
       - Schema Design:
           - Define Tables: how-to/define-tables.md
           - Model Relationships: how-to/model-relationships.ipynb
+          - Master-Part Tables: how-to/master-part.ipynb
           - Design Primary Keys: how-to/design-primary-keys.md
           - Read Diagrams: how-to/read-diagrams.ipynb
       - Project Management:
@@ -102,6 +103,7 @@ nav:
               - Semantic Matching: reference/specs/semantic-matching.md
               - Primary Keys: reference/specs/primary-keys.md
               - Fetch API: reference/specs/fetch-api.md
+              - Diagram: reference/specs/diagram.md
           - Type System:
               - Types: reference/specs/type-system.md
               - Codec API: reference/specs/codec-api.md

scripts/execute-notebooks.sh

@@ -14,6 +14,9 @@ export DJ_PASS=tutorial
 export DJ_BACKEND=postgresql
 export DJ_USE_TLS=false
 
+# Display settings
+export DJ_DIAGRAM_DIRECTION=LR
+
 echo "Executing tutorials with PostgreSQL backend..."
 echo "  Host: $DJ_HOST:$DJ_PORT"
 echo "  User: $DJ_USER"

src/assets/stylesheets/extra.css

@@ -110,6 +110,29 @@
    Code Block Enhancements
    ------------------------------------------------------------------- */
 
+/* Smaller font and line wrapping for code blocks */
+.md-typeset code,
+.md-typeset pre > code,
+.md-typeset .highlight code,
+.md-typeset .highlighttable code {
+  font-size: 0.7em !important;
+}
+
+/* Also target Jupyter notebook code cells */
+.md-typeset .jp-CodeCell code,
+.md-typeset .cell_input code,
+.md-typeset .input_area code {
+  font-size: 0.7em !important;
+}
+
+.md-typeset pre,
+.md-typeset .highlight pre {
+  /* Wrap long lines */
+  white-space: pre-wrap;
+  word-wrap: break-word;
+  overflow-wrap: break-word;
+}
+
 /* Add subtle visual distinction for code comparisons */
 .version-comparison {
   display: grid;

src/explanation/index.md

@@ -49,6 +49,14 @@ and scalable.
     From workflows to complete data operations systems. Project structure and
     object-augmented schemas.
 
+-   :material-link-variant: **[Semantic Matching](semantic-matching.md)**
+
+    How DataJoint ensures safe joins through attribute lineage tracking.
+
+-   :material-new-box: **[What's New in 2.0](whats-new-2.md)**
+
+    Major changes, new features, and migration guidance for DataJoint 2.0.
+
 -   :material-frequently-asked-questions: **[FAQ](faq.md)**
 
     How DataJoint compares to ORMs, workflow managers, and lakehouses.

src/how-to/define-tables.md

@@ -350,3 +350,10 @@ print(Session().describe())
 # Show heading
 print(Session().heading)
 ```
+
+## See Also
+
+- [Master-Part Tables](master-part.ipynb) — Working with part tables
+- [Model Relationships](model-relationships.ipynb) — Foreign key patterns
+- [Design Primary Keys](design-primary-keys.md) — Key selection strategies
+- [Insert Data](insert-data.md) — Adding data to tables

src/how-to/delete-data.md

@@ -191,6 +191,7 @@ print(f"Deleted {count} subjects")
 
 ## See Also
 
+- [Master-Part Tables](master-part.ipynb) — Compositional data patterns
 - [Model Relationships](model-relationships.ipynb) — Foreign key patterns
 - [Insert Data](insert-data.md) — Adding data to tables
 - [Run Computations](run-computations.md) — Recomputing after changes

src/how-to/demo_modules/__init__.py

@@ -0,0 +1 @@
+# Demo modules for diagram collapse example

src/how-to/demo_modules/acquisition.py

@@ -0,0 +1,40 @@
+"""
+acquisition.py - Data acquisition tables
+
+This module defines the core acquisition schema with Subject and Session tables.
+Tables are declared but schema must be activated before use.
+"""
+import datajoint as dj
+
+# Create schema without activating - caller must activate before use
+schema = dj.Schema()
+
+
+@schema
+class Lab(dj.Manual):
+    definition = """
+    lab : varchar(32)
+    ---
+    institution : varchar(100)
+    """
+
+
+@schema
+class Subject(dj.Manual):
+    definition = """
+    subject_id : varchar(16)
+    ---
+    -> Lab
+    species : varchar(50)
+    sex : enum('M', 'F', 'U')
+    """
+
+
+@schema
+class Session(dj.Manual):
+    definition = """
+    -> Subject
+    session_date : date
+    ---
+    session_notes : varchar(1000)
+    """

src/how-to/demo_modules/analysis.py

@@ -0,0 +1,48 @@
+"""
+analysis.py - Analysis tables
+
+This module defines analysis tables that aggregate results across sessions.
+Tables are declared but schema must be activated before use.
+"""
+import datajoint as dj
+from . import acquisition, processing
+
+# Create schema without activating - caller must activate before use
+schema = dj.Schema()
+
+
+@schema
+class AnalysisParams(dj.Lookup):
+    definition = """
+    analysis_id : int16
+    ---
+    method : varchar(50)
+    """
+    contents = [(1, 'correlation'), (2, 'regression')]
+
+
+@schema
+class SubjectAnalysis(dj.Computed):
+    definition = """
+    -> acquisition.Subject
+    -> AnalysisParams
+    ---
+    result : float32
+    confidence : float32
+    """
+
+    def make(self, key):
+        self.insert1({**key, 'result': 0.8, 'confidence': 0.95})
+
+
+@schema
+class CrossSessionAnalysis(dj.Computed):
+    definition = """
+    -> acquisition.Subject
+    -> processing.ProcessingParams
+    ---
+    aggregate_score : float32
+    """
+
+    def make(self, key):
+        self.insert1({**key, 'aggregate_score': 0.9})

src/how-to/demo_modules/processing.py

@@ -0,0 +1,50 @@
+"""
+processing.py - Data processing tables
+
+This module defines computed tables that process raw acquisition data.
+Tables are declared but schema must be activated before use.
+"""
+import datajoint as dj
+from . import acquisition
+
+# Create schema without activating - caller must activate before use
+schema = dj.Schema()
+
+
+@schema
+class ProcessingParams(dj.Lookup):
+    definition = """
+    params_id : int16
+    ---
+    filter_cutoff : float32
+    threshold : float32
+    """
+    contents = [(1, 30.0, 2.5), (2, 50.0, 3.0)]
+
+
+@schema
+class ProcessedSession(dj.Computed):
+    definition = """
+    -> acquisition.Session
+    -> ProcessingParams
+    ---
+    n_events : int32
+    quality_score : float32
+    """
+
+    def make(self, key):
+        self.insert1({**key, 'n_events': 100, 'quality_score': 0.95})
+
+
+@schema
+class EventDetection(dj.Computed):
+    definition = """
+    -> ProcessedSession
+    event_id : int32
+    ---
+    event_time : float32
+    amplitude : float32
+    """
+
+    def make(self, key):
+        pass

src/how-to/index.md

@@ -17,7 +17,9 @@ they assume you understand the basics and focus on getting things done.
 
 - [Define Tables](define-tables.md) — Table definition syntax
 - [Model Relationships](model-relationships.ipynb) — Foreign key patterns
+- [Master-Part Tables](master-part.ipynb) — Compositional data patterns
 - [Design Primary Keys](design-primary-keys.md) — Key selection strategies
+- [Read Schema Diagrams](read-diagrams.ipynb) — Interpret visual diagrams
 
 ## Project Management
 
@@ -42,6 +44,7 @@ they assume you understand the basics and focus on getting things done.
 - [Object Storage Overview](object-storage-overview.md) — Navigation guide for all storage docs
 - [Choose a Storage Type](choose-storage-type.md) — Decision guide for codecs
 - [Use Object Storage](use-object-storage.md) — When and how
+- [Use the `<npy>` Codec](use-npy-codec.md) — NumPy arrays with lazy loading
 - [Use Plugin Codecs](use-plugin-codecs.md) — Install codec packages via entry points
 - [Create Custom Codecs](create-custom-codec.md) — Domain-specific types
 - [Manage Large Data](manage-large-data.md) — Blobs, streaming, efficiency

src/how-to/insert-data.md

@@ -143,3 +143,9 @@ with dj.conn().transaction:
 if Subject.validate(rows):
     Subject.insert(rows)
 ```
+
+## See Also
+
+- [Master-Part Tables](master-part.ipynb) — Atomic insertion of master and parts
+- [Define Tables](define-tables.md) — Table definition syntax
+- [Delete Data](delete-data.md) — Removing data from tables