LatentLab: An Experimental LLM Cognitive Observability Laboratory

Welcome to LatentLab, an experimental environment designed to observe, measure, and analyze the inner workings of Large Language Models (LLMs). Rather than simulating artificial emotions through static prompting rules, LatentLab treats LLMs as dynamic informational systems, aiming to infer emergent cognitive states from raw mathematical signals.

---

🔬 Project Overview

Modern LLM interactions act as a black box. LatentLab is a "microscope" built on top of local causal models (specifically the Qwen2.5 series) to extract, persist, and visualize internal dynamics during text generation. By mapping internal states to information theory metrics, we aim to translate mathematical features into human-interpretable cognitive dimensions.

---

🎯 Objectives

1. Inner State Observability (The LLM ECG):

   • Capture hidden states across multiple layers.
   • Extract attention patterns and weights for every generated token relative to the context.
   • Calculate prediction uncertainty using Shannon Entropy and top-k logit distributions.

2. Concept Extraction (Mechanistic Interpretability):

   • Detect conceptual directions in the latent space.
   • Lay the groundwork for using Sparse Autoencoders (SAEs) to decompose high-dimensional activations into sparse, interpretable features (e.g., mapping activation vectors to human-understandable concepts).

3. Cognitive State Formulation:

   • Formulate mathematical representations of complex cognitive states based on raw signals:
     • Doubt/Confusion: High entropy + high competition among multiple token hypotheses.
     • Confidence: Low entropy + dominant top logit probability.
     • Curiosity/Exploration: High conceptual novelty + broad attention distribution.
     • Interest: Temporal persistence of specific concept activations across multiple generation steps.
     • Surprise: Abrupt shift in hidden state trajectories between consecutive tokens.

4. Cognitive Memory & Persistent State:

   • Design a persistent database schema to track how these derived states and concepts evolve across different prompts, sessions, and interactions.

---

🏁 Final Target: What We Will Achieve

By the end of this experimental journey, we will have built a complete, self-contained pipeline comprising:

• Latent Engine (Python/PyTorch): A lightweight inference backend running locally on Apple Silicon (mps), capable of streaming token generations alongside their high-dimensional metrics (attention matrices, entropy, and layer activations).
• Cognitive Processor: A math module that consumes raw outputs to compute real-time cognitive metrics.
• Storage Layer: A time-series database (or a simplified relational schema) storing historical token-by-token states for past conversations.
• Scientific Observability Console (SOC) (React / Vite / Tailwind): An interactive, premium frontend dashboard containing:
  • Emergent Emotional Core (EEC): A dynamic, multi-layered organic fluid blob that morphs in color, speed, and geometry to visualize cognitive states (Confidence, Doubt, Surprise, Curiosity) in real-time.
  • Attention Synapse Highlighter: Reactively shading preceding tokens in the neural console to map attention weights upon hover.
  • Decisions Inspector: Visualizing the alternative candidate logit distributions at each generation step.
  • 2D & 3D Mindmap Projections: Rendered via specialized transparent Plotly layouts to trace hidden states inside the latent space.
  • Dual-Language HUD: Multi-language interface supporting English (default) and Spanish.

---

🚀 Getting Started

1. Environment Setup

LatentLab uses a Python virtual environment to manage dependencies. To initialize and install requirements:

# Create the virtual environment
python3 -m venv .venv

# Activate the environment
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

2. Configuration (.env)

Copy the template configuration file to create your active setup:

cp .env.example .env

Configure your parameters in .env:

• MODEL_NAME: Hugging Face model identifier (default: Qwen/Qwen2.5-0.5B-Instruct).
• DEFAULT_PROMPT: Default analysis query.
• MAX_TOKENS, TEMPERATURE, TOP_K: Inherent generation parameters.
• DEVICE: Force hardware device accelerator (auto, mps, cuda, cpu).

3. Running the Microscope

Execute the CLI microscope to run inference and observe metric streaming in real-time:

# Run using defaults from .env
python run_lab.py

# Override parameters like prompt, model, max tokens, and temperature (creativity vs determinism)
python run_lab.py --prompt "Compare space travel to ocean exploration" --model sshleifer/tiny-gpt2 --max_tokens 15 --temp 0.2

---

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

Generated outputs and cognitive metrics for each session are automatically saved under the storage/ directory as structured JSON files for subsequent frontend ingestion.