Skip to content

DEMOS_GUIDE.md

Latest commit

 

History

History
298 lines (225 loc) · 6.24 KB

DEMOS_GUIDE.md

File metadata and controls

298 lines (225 loc) · 6.24 KB
Raw

Demos Guide - Which One to Use?

🎯 Quick Decision Guide

Demo Best For Reliability Features
run_demo_simple.py First-time users ⭐⭐⭐⭐⭐ Security features only
demo_simple_distributed.py Learning distributed ⭐⭐⭐⭐ Full distributed (threads)
demo.py Advanced/Testing ⭐⭐ Full distributed (processes)

📝 Detailed Comparison

1. Simple Demo (run_demo_simple.py)

Use this if:

  • ✅ You're trying the system for the first time
  • ✅ You want to verify installation
  • ✅ You want to see security features quickly
  • ✅ You don't need distributed execution

Advantages:

  • Most reliable (no network issues)
  • Fastest to run (~1-2 minutes)
  • Shows all security features
  • No setup complexity

Run it:

cd examples
python run_demo_simple.py

Expected output:

Simple Secure MDI-LLM Demo
==================================================
[1] Loading model...
[2] Setting up security...
    Weight commitment: 8f3a4b9c...
[3] Generating text with verification...
    Token 0: Verification ✓
    Token 5: Verification ✓
    Token 10: Verification ✓
[4] Generated text:
    The future of artificial intelligence is...

✓ Demo complete!

2. Thread-Based Distributed (demo_simple_distributed.py) ⭐ RECOMMENDED

Use this if:

  • ✅ You want to see real distributed inference
  • ✅ You want reliable multi-node execution
  • ✅ You're learning how the system works
  • ✅ You want to avoid multiprocessing complexity

Advantages:

  • Reliable distributed execution
  • Shows actual node communication
  • Demonstrates pipeline parallelism
  • Easier to debug (shared memory)
  • No serialization issues

Run it:

cd examples
python demo_simple_distributed.py

Expected output:

============================================================
Secure Model-Distributed LLM Inference Demo
(Thread-based version)
============================================================

Configuration:
  Model: gpt2
  Nodes: 3
  Device: cpu

[1/5] Loading tokenizer...
[2/5] Partitioning model...
  Created 3 chunks:
    Node 0 (starter): 41,158,656 parameters
    Node 1 (secondary): 42,467,328 parameters
    Node 2 (secondary): 40,894,464 parameters

[3/5] Starting secondary nodes...
[node_1] Node started
[node_2] Node started
...

Runtime: ~2-5 minutes

3. Process-Based Distributed (demo.py) ⚠️ EXPERIMENTAL

Use this if:

  • ⚠️ You're experienced with distributed systems
  • ⚠️ You want to deploy on multiple machines
  • ⚠️ You're testing multiprocessing behavior
  • ⚠️ You're debugging serialization issues

Known Issues:

  • PyTorch model serialization across processes is complex
  • May encounter "Receiver error" or timeouts
  • Requires careful connection ordering
  • More difficult to debug

Run it:

cd examples
python demo.py

If you see errors:

[node_1] Receiver error: 
[node_0] Timeout receiving message...

Solution: Use demo_simple_distributed.py instead!

🔄 Progression Path

Recommended learning order:

1. run_demo_simple.py
   ↓
   Understand security features
   ↓
2. demo_simple_distributed.py  
   ↓
   See distributed execution
   ↓
3. Modify and experiment
   ↓
   Customize for your use case

🛠️ Customization Examples

Change Generation Length

All demos:

max_new_tokens=50  # Generate 50 tokens instead of 30

Change Prompts

All demos:

prompts = [
    "Your custom prompt here",
    "Another prompt",
]

Change Model

All demos:

MODEL_NAME = "gpt2-medium"  # Use larger model (355M params)

Adjust Verification Rate

In src/verification.py:

VerificationManager(verification_rate=0.2)  # 20% verification

🐛 Troubleshooting

Demo won't start?

  1. Check Python version:

    python3 --version  # Need 3.8+

  2. Activate virtual environment:

    source venv/bin/activate

  3. Check dependencies:

    pip list | grep torch

Port conflicts?

Change ports in the demo file:

nodes_config = [
    {'host': 'localhost', 'port': 6000, 'next_port': 6001},
    {'host': 'localhost', 'port': 6001, 'next_port': 6002},
    {'host': 'localhost', 'port': 6002, 'next_port': 6000},
]

Out of memory?

Use smaller config:

MODEL_NAME = "gpt2"  # Smallest
prompts = ["Single prompt"]  # Fewer prompts
max_new_tokens=10  # Fewer tokens

📊 Performance Comparison

Demo Setup Time Run Time Tokens/sec Memory
Simple Instant 1-2 min 3-5 800MB
Thread-based ~5 sec 3-5 min 2-4 1.5GB
Process-based ~10 sec Varies Varies 2GB+

🎓 What Each Demo Teaches

Simple Demo

  • ✓ How cryptographic commitments work
  • ✓ How verification proofs are created
  • ✓ How activations are hashed
  • ✓ Basic inference flow

Thread-Based Distributed

  • ✓ How nodes communicate
  • ✓ How model chunks work together
  • ✓ How pipeline parallelism works
  • ✓ How encryption is applied
  • ✓ Complete distributed flow

Process-Based Distributed

  • ✓ Challenges of true multiprocessing
  • ✓ Model serialization issues
  • ✓ Cross-process communication
  • ✓ Production deployment considerations

🚀 Getting Started Right Now

Fastest path to success:

# 1. Setup (one time)
./setup.sh

# 2. Run simple demo (verify it works)
cd examples && python run_demo_simple.py

# 3. Run distributed demo (see it in action)
python demo_simple_distributed.py

# 4. Success! 🎉

📚 See Also

💡 Pro Tips

  1. Always start with simple demo - Verify your setup works
  2. Use thread-based for learning - Most reliable distributed demo
  3. Monitor memory usage - GPT-2 medium needs ~2GB+
  4. Check ports first - lsof -i :5000 before running
  5. Read error messages - They're usually very informative

Happy inferencing! 🚀