Google: Gemma 3 4B

Processes both image and text inputs simultaneously through a unified transformer architecture, maintaining coherence across up to 128,000 tokens of context. The model uses interleaved vision-language embeddings that allow it to reason about visual content and text in the same forward pass, enabling tasks like image captioning, visual question answering, and document analysis without separate encoding pipelines.

The model's transformer backbone is trained on a diverse multilingual corpus covering 140+ languages, using shared token embeddings and language-agnostic attention patterns. This enables zero-shot cross-lingual transfer where the model can understand and respond in languages not explicitly fine-tuned, with particular strength in high-resource languages and emerging support for low-resource language pairs through transfer learning.

Enhanced transformer layers with specialized attention patterns for mathematical token sequences, trained on mathematical datasets including proofs, equations, and step-by-step solutions. The model learns to decompose complex math problems into intermediate symbolic steps, maintaining consistency across multi-step derivations through constrained decoding that validates mathematical syntax during generation.

The 4B model is instruction-tuned using reinforcement learning from human feedback (RLHF) to follow complex multi-step instructions while maintaining awareness of conversation history and user intent. The chat interface uses a sliding context window that prioritizes recent messages and system prompts, with attention masking that prevents the model from attending to irrelevant historical context beyond a certain age threshold.

A lightweight transformer model with 4 billion parameters optimized for inference speed and memory efficiency through quantization-aware training and architectural pruning. The model uses grouped query attention (GQA) to reduce KV cache size, enabling deployment on consumer GPUs and edge devices while maintaining competitive performance with larger models through knowledge distillation from larger Gemma variants.

The model can be constrained to generate outputs matching a provided JSON schema through constrained decoding, where a token-level validator prevents generation of tokens that would violate the schema. This enables reliable extraction of structured data (JSON, XML) without post-processing, using a grammar-based approach that enforces valid syntax during generation rather than validating after the fact.

Gemma 3 4B is accessible via OpenRouter's unified API endpoint, which abstracts away model-specific implementation details and provides a standardized interface for text and vision inputs. The integration handles authentication, rate limiting, and request routing through OpenRouter's infrastructure, enabling seamless switching between Gemma 3 and other models without code changes.

The model supports server-sent events (SSE) streaming where tokens are emitted as they are generated, enabling real-time display of model output without waiting for full completion. The streaming implementation uses chunked HTTP transfer encoding with newline-delimited JSON events, allowing clients to display partial responses and cancel requests mid-generation.