Gemma 3

Processes interleaved sequences of text and image tokens within a single 128K-token context window, enabling long-form reasoning tasks that combine visual and textual information. Uses a unified transformer architecture with image embeddings projected into the token space, allowing the model to maintain coherent reasoning across extended documents with embedded images. The large context window enables processing of full codebases, long documents, or multi-turn conversations without truncation.

Supports low-rank adaptation (LoRA) and quantized LoRA (QLoRA) fine-tuning, allowing adaptation of model weights by training only small rank-decomposed matrices (typically 1-2% of original parameters) while keeping base weights frozen. QLoRA variant further reduces memory by quantizing the base model to 4-bit precision, enabling 27B model fine-tuning on consumer GPUs. Uses standard HuggingFace transformers integration with PEFT library for seamless adapter composition.

Runs inference on consumer-grade GPUs (8GB-24GB VRAM) through native support for 8-bit and 4-bit quantization using bitsandbytes and GPTQ formats. Model weights are quantized post-training without retraining, reducing memory footprint by 75-87% while maintaining 95%+ of original performance. Supports dynamic batching and KV-cache optimization to maximize throughput on memory-constrained hardware.

The 27B variant achieves performance on code generation, mathematical reasoning, and logical inference tasks competitive with models 2-3x larger (e.g., Llama 2 70B, Mistral Large). Uses a transformer architecture with improved attention mechanisms and training data curation emphasizing reasoning-heavy tasks. Supports code completion, bug detection, and multi-step reasoning through standard text generation without special prompting techniques.

Distributed under the Gemma License, a permissive open-source license allowing unrestricted commercial use, modification, and redistribution without attribution requirements or usage restrictions. Model weights are publicly available on HuggingFace Hub and Google's model repository, enabling self-hosted deployment without licensing fees or API quotas. Supports both research and production use cases without legal restrictions.

Provides four model variants (1B, 4B, 12B, 27B) sharing identical architecture and training procedures, enabling seamless scaling from edge devices to high-performance servers. All variants support the same tokenizer, context window (128K), and fine-tuning approaches, allowing developers to prototype on smaller models and deploy larger variants without code changes. Scaling is achieved through uniform increases in hidden dimension, attention heads, and feed-forward layers.

Base models support instruction-following through standard supervised fine-tuning on instruction-response pairs, enabling adaptation to chat, question-answering, and task-specific formats. Supports multi-turn conversation fine-tuning with role-based tokens (user, assistant, system) for building chatbot variants. Fine-tuning can be performed with LoRA or full-parameter training, with standard HuggingFace trainer integration for reproducible training pipelines.

Trained on multilingual text corpus covering 40+ languages, enabling understanding and generation in non-English languages with performance degradation proportional to language representation in training data. Supports code-switching (mixing languages in single prompt) and translation-adjacent tasks without explicit translation fine-tuning. Language identification is implicit in token generation without separate language detection.

Supports constrained decoding to generate outputs matching predefined JSON schemas or structured formats, using token-level masking to restrict generation to valid continuations. Implemented through integration with libraries like outlines or llama.cpp's grammar-based sampling, which parse schema definitions and enforce constraints during token sampling. Enables reliable extraction of structured data without post-processing or parsing errors.