Multimodal Text And Image Understanding

via “multimodal image-text understanding with vision encoder”

Google's open-weight model family from 1B to 27B parameters.

Unique: Integrates frozen vision encoder with shared transformer decoder, enabling efficient multimodal inference without separate model calls or cross-attention layers, whereas competitors like LLaVA require separate vision and language models with explicit fusion mechanisms

vs others: Faster multimodal inference than LLaVA 1.5 due to single-model architecture, and more efficient than GPT-4V for on-device deployment while maintaining competitive visual reasoning on standard benchmarks