This repository contains the collection of explorative notebooks pure in python and in the language that we, humans can read. Have tried to compile all lectures from the Andrej Karpathy's 💎 playlist on Neural Networks - which we will end up with building GPT.

Inspired by Andrej Karpathy’s "Let’s Build GPT", this project guides you step‑by‑step to build a GPT from scratch, demystifying its architecture through clear, hands‑on code.

This project implements GPT-1 using PyTorch, focusing on foundational transformer architectures for natural language processing tasks.

Comprehensive fresh start for how LLMs work

MicroGPT is a clean, educational implementation of the GPT (Generative Pre-trained Transformer) architecture built from first principles with detailed explanations and comprehensive testing.

HackingGPT part 7 where we learn the foundations to create a custom GPT from absolute scratch.

HackingGPT part 1 where we learn the foundations to create a custom GPT from absolute scratch.

This notebook builds a complete GPT (Generative Pre-trained Transformer) model from scratch using PyTorch. It covers tokenization, self-attention, multi-head attention, transformer blocks, and text generation and all explained step-by-step with a simple nursery rhyme corpus.

SeedGPT is a minimal decoder-only LLM from scratch, inspired by andrej karpathy's tutorials, generating text token-by-token based on input word prompts.

HackingGPT part 3 where we learn the foundations to create a custom GPT from absolute scratch.

HackingGPT part 4 where we learn the foundations to create a custom GPT from absolute scratch.

HackingGPT part 2 where we learn the foundations to create a custom GPT from absolute scratch.

Annotated study fork of Karpathy nanoGPT — GPT-2 training from scratch with extended notes on causal self-attention, positional encoding, layer norm placement, and efficient fine-tuning on custom datasets.

Complete GPT implementation from scratch with BPE tokenizer, transformer architecture, and trained models on Shakespeare and WikiText. Achieves perplexity 7.84 on WikiText-2 with 12.6M parameters.

HackingGPT part 6 where we learn the foundations to create a custom GPT from absolute scratch.

HackingGPT part 5 where we learn the foundations to create a custom GPT from absolute scratch.