REFERENCES.md

Learning References

A curated collection of resources for learning Reinforcement Learning, from fundamentals to advanced topics.

---

Courses & Tutorials

Comprehensive Courses

• RL Fundamentals - Hugging Face Deep RL Course

  • Beginner-friendly introduction to RL concepts
  • Hands-on projects with modern libraries

• David Silver's RL Course (DeepMind)

  • Classic foundational course from DeepMind
  • Mathematical rigor with practical examples

• OpenAI Spinning Up

  • OpenAI's educational resource
  • Deep dive into policy gradient methods

---

Blog Series & Articles

• RL Blogs by Ketan Doshi

  • Highly recommended: Read the entire series
  • Breaks down complex concepts into digestible parts
  • Great for building intuition

• Deep Q-Networks Explained - LessWrong

  • Detailed breakdown of DQN architecture
  • Explains the "why" behind design decisions

• Neural Breakdown with AVB (Video)

  • Visual explanations of neural network concepts - Excellent!

---

Research Papers

Foundational Papers

• DQN Paper - Playing Atari with Deep RL

  • The paper that started the deep RL revolution
  • Introduces experience replay and target networks

• PPO Paper - Proximal Policy Optimization

  • Modern policy gradient method
  • Used for training LLMs with RLHF

LLM-Specific Papers

• InstructGPT Paper - RLHF for LLMs

  • How OpenAI trained ChatGPT with human feedback
  • Foundation for modern LLM alignment

• DPO Paper - Direct Preference Optimization

  • Alternative to PPO for LLM training
  • Simpler and more stable

---

Libraries & Tools

Core RL Libraries

• Gymnasium

  • Standard RL environment library
  • Successor to OpenAI Gym

• Stable Baselines3

  • Production-ready RL algorithm implementations
  • Easy to use, well-documented

LLM Training

• TRL (Transformer Reinforcement Learning)
  • Hugging Face library for training LLMs with RL
  • Supports PPO, DPO, and more

---