Gopher

Gopher generates coherent multi-token text sequences using a transformer-based autoregressive architecture with 280 billion parameters trained on large-scale text corpora. The model predicts the next token in a sequence by computing attention across the full context window, enabling generation of long-form content, dialogue responses, and multi-sentence completions. Generation quality improves with scale, though logical reasoning tasks show diminishing returns beyond certain parameter thresholds.

Gopher performs reading comprehension by processing text passages and generating answers to factual questions about the content. The model uses transformer attention mechanisms to identify relevant spans and generate natural language answers, demonstrating significant advancement toward human expert performance on the MMLU benchmark. This capability enables extractive and abstractive question-answering tasks across diverse domains.

Gopher identifies factual accuracy in text by evaluating claims against its training knowledge and generating assessments of whether statements are true, false, or uncertain. The model uses transformer representations to reason about factual consistency, though documentation notes it can confidently propagate incorrect information. This capability enables automated fact-checking workflows but requires human verification due to hallucination risk.

Gopher identifies toxic, offensive, or harmful language in text by learning patterns of toxicity from training data and classifying text segments as toxic or non-toxic. The model uses transformer representations to detect harmful content across various categories, enabling content moderation workflows. This capability supports safety-critical applications but requires threshold tuning and human review for production deployment.

Gopher engages in multi-turn dialogue by processing conversation history and generating contextually appropriate responses using transformer attention over dialogue context. The model does not use dialogue-specific fine-tuning; instead, it relies on careful prompt engineering to steer toward coherent conversational behavior. Responses are generated autoregressively, with quality dependent on prompt formulation and context management.

Gopher performs multitask language understanding by processing diverse prompts spanning multiple knowledge domains and generating appropriate responses without task-specific fine-tuning. The model leverages its 280B parameters and broad training data to handle reading comprehension, fact-checking, toxicity detection, and other tasks through a unified transformer architecture. Performance is evaluated on the MMLU benchmark, which tests understanding across 57 tasks including STEM, humanities, and social sciences.

Gopher serves as the largest model in DeepMind's comparative scaling study, enabling empirical analysis of how language model capabilities scale from 44 million to 280 billion parameters. The study measures performance improvements across multiple tasks and parameter ranges, documenting where scaling provides benefits (text generation, comprehension) and where it plateaus (logical reasoning, common-sense tasks). This capability supports research into optimal model sizing and parameter allocation decisions.

Gopher includes comprehensive evaluation of ethical and social risks through interdisciplinary analysis involving ethicists, safety researchers, and technical teams. The assessment documents failure modes including hallucination, bias reflection, and confident propagation of misinformation alongside capability measurements. This framework enables identification of risks before deployment and informs responsible AI development practices.