LLaMA

Generates text by predicting the next token in a sequence using a transformer decoder-only architecture, with four parameter-scale variants (7B, 13B, 33B, 65B) trained on 1-1.4 trillion tokens. The model uses causal language modeling where each token prediction is conditioned on all previous tokens, enabling recursive generation of coherent multi-sentence outputs. Larger variants (33B, 65B) trained on 1.4 trillion tokens vs smaller variants (7B, 13B) on 1 trillion tokens, allowing users to trade off model capacity against computational cost.

Generates coherent text in 20 languages with the most speakers globally, trained on multilingual unlabeled text covering Latin and Cyrillic writing systems. The model learns language-agnostic representations during pretraining, enabling cross-lingual transfer where knowledge from high-resource languages (English, Spanish) can apply to lower-resource languages in the training set. No language-specific tokenizers or separate model heads are required; a single unified tokenizer handles all 20 languages.

Provides a pretrained base model designed explicitly for downstream fine-tuning on specific tasks (question answering, summarization, classification, code generation). The model uses standard supervised fine-tuning where task-specific labeled data is used to adapt the pretrained weights via gradient descent. The architecture remains unchanged during fine-tuning; only the output layer and final transformer layers are typically adapted, reducing computational cost compared to full retraining.

Performs mathematical problem-solving and symbolic reasoning tasks through next-token prediction on mathematical notation and step-by-step reasoning chains. The model learns mathematical patterns from pretraining data, enabling it to generate intermediate reasoning steps and final answers for problems involving arithmetic, algebra, geometry, and theorem proving. No specialized mathematical modules or symbolic solvers are integrated; reasoning emerges from transformer attention patterns over mathematical tokens.

Predicts protein structures and understands biological sequences through language modeling over amino acid sequences and structural annotations. The model learns patterns in protein sequences during pretraining, enabling it to generate plausible 3D structures or predict secondary structure elements (alpha helices, beta sheets) from primary sequences. This capability emerges from treating protein sequences as a specialized language with its own grammar and patterns.

Answers questions about provided text passages by understanding semantic relationships and extracting relevant information through transformer attention over the full context. The model uses causal language modeling to generate answers token-by-token, conditioning on both the question and the supporting passage. Attention mechanisms learn to focus on relevant passages and phrases, enabling multi-hop reasoning across sentences.

Performs general language understanding tasks including semantic similarity, entailment detection, sentiment analysis, and semantic reasoning through transformer attention and next-token prediction. The model learns universal linguistic patterns during pretraining on 1-1.4 trillion tokens, enabling it to understand grammatical structure, semantic relationships, and pragmatic meaning without task-specific training. Attention heads learn to capture different linguistic phenomena (syntax, semantics, discourse) across layers.

Provides model card documentation detailing construction, training data composition, and evaluation results for bias and toxicity following responsible AI practices. The model card includes benchmark evaluations measuring bias across demographic groups and toxicity generation rates, enabling users to understand and mitigate potential harms. Documentation is designed to support informed decision-making about model deployment and fine-tuning.

Provides access to model weights and documentation through a noncommercial license requiring case-by-case approval from Meta. Access is granted to academic researchers, government/civil society/academia-affiliated organizations, and industry research laboratories based on application review. The licensing model restricts commercial use and production deployment, requiring users to demonstrate research intent and institutional affiliation.