t5-large

T5-large implements a unified text2text-generation architecture where all NLP tasks (translation, summarization, paraphrase, question answering) are framed as sequence-to-sequence problems with task-specific prefixes prepended to inputs. The model uses a 24-layer encoder-decoder Transformer with 770M parameters trained on the C4 corpus via denoising objectives, enabling it to handle diverse text transformation tasks through a single unified interface rather than task-specific model heads.

T5-large performs abstractive summarization by treating it as a text2text task where the input is prefixed with 'summarize:' and the model generates a condensed output sequence. The encoder processes the full document while the decoder generates summary tokens autoregressively, using cross-attention over encoder hidden states. Length can be controlled via beam search parameters or by appending length tokens to the input prefix.

T5-large performs machine translation by encoding source language text and decoding target language output, with language pair specified via input prefix (e.g., 'translate English to French: hello'). The model uses shared encoder-decoder weights trained on parallel corpora within the C4 dataset, enabling zero-shot transfer to language pairs not explicitly seen during pretraining. Translation quality is controlled via beam search width and length penalty parameters.

T5-large supports efficient fine-tuning on custom text2text tasks by freezing or partially unfreezing encoder-decoder weights and training on task-specific datasets with custom prefixes (e.g., 'question: ... context: ...' for QA). The model uses standard cross-entropy loss on decoder outputs, with optional techniques like LoRA (Low-Rank Adaptation) or adapter modules to reduce trainable parameters. Fine-tuning leverages pretrained representations from C4 denoising objectives, requiring only 10-20% of data compared to training from scratch.

T5-large learns shared multilingual representations during pretraining on C4 corpus, enabling zero-shot cross-lingual transfer where knowledge learned on English tasks transfers to French, Romanian, and German without explicit multilingual training. The encoder learns language-agnostic semantic representations through denoising objectives applied uniformly across languages, while the decoder learns to generate coherent text in any language. This enables tasks like translating between non-English language pairs (French-to-German) with minimal degradation despite no explicit training on that pair.

T5-large supports configurable beam search decoding with adjustable beam width, length penalty, and early stopping criteria to balance translation quality against latency. Beam search maintains multiple hypotheses during decoding, scoring each via log-probability and length-normalized scores. Length penalty parameters control output length without retraining, enabling dynamic adjustment of summary/translation length at inference time. Greedy decoding is also supported for minimal latency applications.