LLMs-from-scratch

Implements scaled dot-product attention using Query/Key/Value linear projections (W_query, W_key, W_value) with causal masking to prevent attending to future tokens. The mechanism splits embeddings across multiple heads, computes attention scores via matrix multiplication (queries @ keys.transpose), applies a triangular mask buffer registered in __init__, and projects concatenated head outputs through out_proj. This enables parallel attention computation across sequence positions while maintaining autoregressive constraints required for token-by-token generation.

Implements a modular GPTModel class that accepts a configuration dictionary specifying embedding dimension, number of layers, attention heads, and feed-forward width. The architecture stacks transformer blocks (each containing multi-head attention, layer normalization, and feed-forward networks) with token and positional embeddings, then projects to vocabulary logits. The configuration pattern allows instantiation of model variants (GPT-small, GPT-medium, GPT-large) by changing dict values rather than code, enabling systematic scaling studies and transfer learning experiments.

Adds learnable or fixed positional embeddings to token embeddings to encode sequence positions, enabling the model to distinguish between tokens at different positions. The implementation creates a position embedding matrix (context_length, embedding_dim) and adds it element-wise to token embeddings before passing to transformer blocks. This allows attention mechanisms to incorporate position information, critical for understanding word order in language.

Creates training batches by sliding a fixed-size window over tokenized text, generating overlapping sequences that maximize data utilization. The implementation reads tokenized text, creates sliding windows of context_length, groups windows into batches, and yields (input, target) pairs where targets are inputs shifted by one position. This approach reduces memory overhead compared to padding variable-length sequences and ensures all tokens contribute to training.

Evaluates model quality by computing perplexity (exp(loss)) and cross-entropy loss on held-out validation data. The implementation runs the model in evaluation mode (disabling dropout), computes loss without gradient computation, and aggregates metrics across batches. Perplexity measures how well the model predicts validation tokens — lower is better, with perplexity=1 indicating perfect predictions.

Implements BPE tokenization by iteratively merging the most frequent adjacent token pairs in a corpus, building a vocabulary of subword units. The algorithm tracks pair frequencies, applies merges in order, and encodes text by greedily matching longest subword sequences. This approach reduces vocabulary size compared to character-level tokenization while maintaining semantic meaning, enabling efficient representation of rare words through composition.

Implements a training loop that predicts the next token given preceding context by computing cross-entropy loss between model logits and ground-truth next tokens. The loop iterates over batches, performs forward passes through the GPT model, computes loss on shifted token sequences (input tokens predict next tokens), backpropagates gradients, and updates weights via optimizer steps. This approach trains the model to learn conditional probability distributions P(token_t | tokens_0..t-1), the foundation of autoregressive generation.

Adapts a pretrained language model to follow instructions by fine-tuning on curated instruction-response pairs. The approach computes loss only on response tokens (not instruction tokens), using a mask to zero out instruction loss. This trains the model to generate appropriate responses given task descriptions, shifting from next-token prediction to instruction-following behavior. The implementation supports both full-parameter fine-tuning and parameter-efficient variants.

Reduces fine-tuning memory and compute by freezing pretrained weights and adding low-rank decomposition matrices (A and B) to attention and feed-forward layers. During forward pass, the model computes output as W*x + (B @ A)*x, where W is frozen and (B @ A) is trainable with rank r << hidden_dim. This approach reduces trainable parameters by 99%+ while maintaining performance, enabling fine-tuning of large models on consumer GPUs. The implementation applies LoRA to query/key/value projections and feed-forward layers.

Generates text by iteratively predicting the next token given previous tokens, using sampling strategies to control output diversity. The implementation computes logits for the next position, applies temperature scaling (dividing by T to sharpen or smooth probability distribution), filters to top-k or top-p (nucleus) tokens, and samples from the resulting distribution. This enables controllable generation from deterministic (temperature=0, greedy) to highly stochastic (temperature=2.0, top-p=0.95) outputs.

Aligns model outputs to human preferences by directly optimizing a preference loss on pairs of chosen/rejected responses, without training a separate reward model. The approach computes log probabilities for both responses, applies a preference loss (e.g., binary cross-entropy on preference logits), and backpropagates to update model weights. This simplifies RLHF by eliminating the reward model training phase while maintaining alignment to human feedback.

Loads pretrained weights from external sources (HuggingFace, OpenAI) into the custom GPT architecture by mapping layer names and handling format differences. The implementation reads state dicts from checkpoint files, renames keys to match the custom model's naming scheme, and validates shape compatibility before loading. This enables transfer learning from large pretrained models without reimplementing the architecture in the original framework.

Adapts a pretrained language model for classification by removing the language modeling head and replacing it with a linear classifier that maps the final hidden state to class logits. The approach freezes or partially fine-tunes the transformer backbone and trains the classifier head on labeled examples using cross-entropy loss. This leverages pretrained representations for downstream classification tasks like sentiment analysis or topic classification.