CogFluency: Modeling Working Memory & Speech Disfluency Under Stress

CogFluency is a dual-phase NLP and audio processing engine designed to quantify Cognitive Friction ($F$) and predict public speaking performance under acute cognitive load. Developed as a joint Georgia Tech Graduate Cognitive Science & Toastmasters High Performance Leadership project.

🚀 Key Features

• Phase 1 (CPU/RAM-Optimized): Waveform extraction (Librosa) & speaker diarization (PyAnnote).
• Phase 2 (GPU-Accelerated): Precise orthographic alignment (CrisperWhisper) & syntactic parsing (spaCy).
• ML Classification: PCA dimensionality reduction & Logistic Regression mapping speaker disfluency density against structural repairs.

📊 Dataset & Model Results

• Corpus: 74 competitive tournament speeches.
• Accuracy: 84% with syntactic parsing & word-level ASR timestamps (68% with baseline regex parsing).

🛠️ Quickstart

git clone [https://github.com/Possum/CogFluency.git](https://github.com/Possum/CogFluency.git)
cd CogFluency
uv run bash setup.sh
uv run main.py your_test_speech.wav

⚙️ System requirements

CogFluency must be run on a machine with the following specifications:

• NVIDIA GPU with CUDA support
• At least 8GB of VRAM (12GB preferred)
• 10GB of disk space
• Python 3.9 or later
• CUDA 11.3 or later
• cuDNN 8.2 or later
• ffmpeg and ffprobe

It may be possible to run the model on a machine with fewer resources or without an NVIDIA GPU, but it has not been tested.

💾 Setup

To run this model, the required models must be downloaded in the models/ directory.

Manual Model Download Instructions

This pipeline depends on gated HuggingFace models, which CogFluency expects in the models/ directory. setup.sh attempts to download these models. You can download the required CrisperWhisper and PyAnnote models manually from HuggingFace.

Note: you must login to HuggingFace and agree to terms before downloading these model.

• https://huggingface.co/nyralabs/CrisperWhisper
• https://huggingface.co/pyannote/speaker-diarization-3.1
• https://huggingface.co/pyannote/segmentation-3.0
• https://huggingface.co/pyannote/wespeaker-voxceleb-resnet34-LM

hf download nyrahealth/CrisperWhisper --local-dir ./models/CrisperWhisper
hf download pyannote/speaker-diarization-3.1 --local-dir ./models/speaker-diarization-3.1
hf download pyannote/segmentation-3.0 --local-dir ./models/segmentation-3.0
hf download pyannote/wespeaker-voxceleb-resnet34-LM --local-dir ./models/embedding-model

In order to load the pyannote/speaker diarization local models, first run:

uv run tools/patch_config.py

For the spaCy NLP feature extraction, the pre-compiled model must be installed:

uv run spacy download en_core_web_lg

👷 Building the model

To build this model, the corpus data must be populated in the data/ directory and models must be downloaded in the models/ directory (see "Manual Model DownloadInstructions" above).

Run the baseline model:

uv run build_model.py --baseline=1

Run the model with spaCy feature extraction:

uv run build_model.py

This runs the following modules in order:

• src/transcriber.py
• src/feature_extractor.py OR src/spacy_feature_extractor.py
• src/plot_metrics.py
• src/predictive_model.py

At the end of the pipeline, two sets of plots will be displayed on screen.

Results are placed by default in data/results/ OR data/results_baseline/.

For advanced usage, please run the files in src/ directly.

⚠️ Limitations

This project was run with strict python versions and library versions. Heavy monkeypatching is performed which may break on some systems or different versions.

✨ Acknowledgments

Thank you to Eric Boehm, Joseph Gottschalk, Steve Honda, Mary LeWarne, and Samuel Mallay for participating as members of the High-Performance Leadership (HPL) Guidance Committee, contributing their invaluable domain expertise, time, and dedication in conducting the blind-coded transcript reviews that established the mathematical ground truth for this study.

This research and the resulting CogFluency engine were developed in conjunction with CS-6795: Introduction to Cognitive Science at the Georgia Institute of Technology. I would like to extend my gratitude to the course instructors and my peers for their foundational insights into cognitive load, working memory, and speech disfluency, which heavily shaped the methodology of this project.