happy-llm

Provides hands-on Jupyter notebook-based implementation of core transformer components (multi-head attention, feed-forward layers, positional encoding, encoder-decoder stacks) with progressive complexity. Uses PyTorch to build each component incrementally, allowing learners to understand attention mechanisms, layer normalization, and residual connections through direct code implementation rather than black-box APIs. The tutorial decomposes the transformer into atomic building blocks with mathematical explanations paired to working code.

Complete PyTorch implementation of LLaMA2 decoder-only architecture including rotary position embeddings (RoPE), grouped query attention (GQA), and SwiGLU activation functions. The tutorial builds the full model stack from embedding layers through multi-head attention blocks to output projection, with code organized to mirror the original LLaMA2 paper architecture. Includes parameter initialization strategies and attention masking patterns specific to autoregressive generation.

Comprehensive guide covering the complete pre-training workflow including data preparation, tokenization strategies, loss computation (causal language modeling), learning rate scheduling, gradient accumulation, and mixed-precision training. The tutorial explains training efficiency techniques like activation checkpointing and distributed data parallelism patterns, with code examples showing how to implement each optimization. Includes best practices for monitoring training stability and convergence.

Structured tutorial comparing three fundamental transformer paradigms with side-by-side implementations: encoder-only models (BERT, RoBERTa, ALBERT) for bidirectional understanding with masked language modeling, encoder-decoder models (T5, BART) for sequence-to-sequence tasks, and decoder-only models (GPT, LLaMA) for autoregressive generation. Each paradigm is implemented from scratch with explanations of architectural differences, attention masking patterns, and training objectives specific to each approach.

Tutorial implementing a complete RAG pipeline that combines document retrieval with LLM generation. The system includes vector embedding generation, similarity-based document retrieval from a knowledge base, prompt augmentation with retrieved context, and generation from the augmented prompt. The implementation covers retrieval strategies (dense retrieval with embeddings, sparse retrieval with BM25), ranking mechanisms, and integration patterns between retriever and generator components.

Tutorial covering agent architectures that combine LLMs with tool-use capabilities, planning, and reasoning. The implementation includes action-observation loops where agents decompose tasks into steps, call external tools (APIs, calculators, search engines), process results, and generate next actions. Covers agent planning strategies (ReAct pattern with reasoning and acting, chain-of-thought decomposition), tool schema definition, and integration with LLM function-calling APIs.

Foundational tutorial covering core NLP concepts including text preprocessing, tokenization approaches (word-level, subword-level with BPE and SentencePiece), vocabulary construction, and token embedding initialization. The tutorial explains why different tokenization strategies matter for different languages and tasks, with code examples showing how to implement tokenizers from scratch and use pretrained tokenizers. Includes analysis of vocabulary size trade-offs and handling of out-of-vocabulary words.

Tutorial covering evaluation methodologies for language models including perplexity calculation, task-specific metrics (BLEU for translation, ROUGE for summarization, exact match and F1 for QA), and benchmark datasets (GLUE, SuperGLUE, SQuAD). The tutorial explains how to implement evaluation metrics from scratch, interpret results correctly, and understand limitations of each metric. Includes guidance on selecting appropriate benchmarks for different model types and applications.

The tutorial is organized as a hierarchical learning system that progresses from theoretical foundations (chapters 1-4: NLP basics, transformer architecture, model families) to practical implementation (chapters 5-7: building LLMs, training pipelines, applications). Each chapter builds on previous knowledge with integrated theory and code, using Jupyter notebooks to interleave mathematical explanations with executable PyTorch implementations. The progression enables learners to understand concepts deeply before implementing them.

The entire tutorial is delivered as executable Jupyter notebooks that interleave explanatory text, mathematical formulas (LaTeX), and runnable Python code. Each notebook is self-contained with imports, function definitions, and example executions, allowing learners to run code immediately and experiment with modifications. The notebooks use PyTorch for all implementations and include visualizations of attention weights, loss curves, and model outputs.