Compositional Visual Mathematical Reasoning Evaluation

via “compositional visual-mathematical reasoning evaluation”

Visual mathematical reasoning benchmark.

Unique: Explicitly targets compositional reasoning where visual perception and mathematical logic must be jointly applied, rather than testing these capabilities separately. Benchmark design enforces this requirement through example selection, though validation methodology is not documented. This compositional focus distinguishes MathVista from benchmarks testing visual understanding (e.g., image captioning) or mathematical reasoning (e.g., text-only math problems) in isolation.

vs others: More rigorous than benchmarks testing visual understanding or mathematical reasoning separately because it requires models to jointly apply both capabilities, exposing failures in composition that single-modality benchmarks would miss.

via “mathematical reasoning over visual data”

Mistral's 124B multimodal model with vision capabilities.

Unique: Achieves 69.4% on MathVista benchmark (outperforming all tested models) through integrated visual parsing and mathematical reasoning in a single 124B model, without requiring separate symbolic math engines or specialized mathematical libraries

vs others: Outperforms GPT-4o, Gemini-1.5 Pro, and Claude-3.5 Sonnet on MathVista while being available for self-hosted deployment, eliminating API dependency for educational or research mathematical analysis

via “visual-reasoning-over-complex-scenes”

Open multimodal model for visual reasoning.

Unique: Trained on 77K complex reasoning samples (49% of instruction-tuning dataset) generated by GPT-4, explicitly optimizing for multi-step inference over visual content; this heavy weighting toward reasoning tasks differentiates it from captioning-focused vision models

vs others: Outperforms general-purpose vision models on reasoning-heavy benchmarks like Science QA (92.53% accuracy) because nearly half its training data is reasoning-focused, whereas models like CLIP or standard captioning systems optimize for classification or description

via “compositional-visual-understanding-through-structured-annotations”

108K images with dense scene graphs and 5.4M region descriptions.

Unique: Provides explicit decomposition of images into objects, attributes, and relationships, enabling training of compositional models that understand visual scenes through structured components. Scene graphs naturally support compositional learning by representing images as compositions of objects and relationships.

vs others: Enables compositional learning unlike flat image-label datasets; supports training models that generalize to novel combinations of known components

via “mathematical reasoning and logic problem evaluation with specialized scoring”

ReLE评测：中文AI大模型能力评测（持续更新）：目前已囊括374个大模型，覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型， 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜，也提供规模超200万的大

Unique: Evaluates mathematical reasoning with 1-5 quality scale for reasoning steps rather than binary correctness, enabling partial credit for correct methodology with computational errors. Combines final answer accuracy with reasoning quality assessment to capture mathematical thinking capability. Includes multi-step reasoning problems and logical inference tasks beyond simple arithmetic.

vs others: More nuanced mathematical assessment than MMLU (binary correctness) and captures reasoning quality vs answer-only evaluation

via “mathematical reasoning evaluation”

UGI-Leaderboard — AI demo on HuggingFace

Unique: Isolates mathematical reasoning as a distinct evaluation dimension on the leaderboard, enabling models to be ranked separately on math vs general generation, revealing capability specialization.

vs others: Simpler than running MATH or GSM8K locally with custom evaluation scripts, but less transparent than open-source math benchmarks regarding problem selection and difficulty.

via “extended reasoning with chain-of-thought for complex visual tasks”

Qwen3-VL-30B-A3B-Thinking is a multimodal model that unifies strong text generation with visual understanding for images and videos. Its Thinking variant enhances reasoning in STEM, math, and complex tasks. It excels...

Unique: Integrates extended reasoning directly into the model's forward pass for visual tasks, rather than using post-hoc prompting techniques like 'think step-by-step', enabling the model to allocate compute dynamically to reasoning-heavy visual problems

vs others: More reliable than prompt-based chain-of-thought for visual reasoning because reasoning is baked into model weights, not dependent on prompt engineering; produces more consistent intermediate steps for STEM tasks

via “multimodal reasoning with extended thinking for stem and mathematical problem-solving”

Qwen3-VL-235B-A22B Thinking is a multimodal model that unifies strong text generation with visual understanding across images and video. The Thinking model is optimized for multimodal reasoning in STEM and math....

Unique: Unifies visual and textual reasoning through a single 235B parameter model with explicit thinking tokens, rather than treating vision and language as separate processing streams. The architecture uses a shared transformer backbone with vision-language fusion at intermediate layers, allowing mathematical reasoning to operate directly over visual features (e.g., reasoning about graph structure while reading axis labels).

vs others: Outperforms GPT-4V and Claude 3.5 Sonnet on STEM benchmarks (MATH-Vision, SciQA) because thinking tokens enable explicit symbolic reasoning over visual content, whereas competitors rely on implicit visual understanding without intermediate reasoning artifacts.

via “visual reasoning with chain-of-thought explanations”

GLM-4.5V is a vision-language foundation model for multimodal agent applications. Built on a Mixture-of-Experts (MoE) architecture with 106B parameters and 12B activated parameters, it achieves state-of-the-art results in video understanding,...

Unique: Generates visual reasoning chains natively through the language model component while maintaining visual grounding, rather than using post-hoc explanation techniques — enables reasoning that is grounded in actual visual features rather than model internals

vs others: Provides more transparent reasoning than black-box vision models, and produces more visually-grounded explanations than text-only reasoning models, though less formally verifiable than symbolic reasoning systems

via “visual question answering with reasoning chains”

Qwen3-VL-32B-Instruct is a large-scale multimodal vision-language model designed for high-precision understanding and reasoning across text, images, and video. With 32 billion parameters, it combines deep visual perception with advanced text...

Unique: Implements implicit chain-of-thought reasoning within the model's forward pass, decomposing complex visual questions into intermediate reasoning steps without requiring explicit prompt engineering

vs others: 32B parameter scale enables more sophisticated multi-step reasoning than smaller VLMs; more reliable than GPT-4V for structured reasoning tasks due to instruction-tuning on reasoning datasets

via “visual reasoning and scene understanding”

Llama 3.2 11B Vision is a multimodal model with 11 billion parameters, designed to handle tasks combining visual and textual data. It excels in tasks such as image captioning and...

Unique: Instruction-tuned to follow explicit reasoning prompts, enabling users to request step-by-step explanations without model fine-tuning. Cross-attention mechanisms ground reasoning in specific image regions, improving interpretability compared to black-box visual reasoning.

vs others: More interpretable reasoning than GPT-4V because instruction-tuning enables explicit reasoning traces; faster inference than larger models but with reduced reasoning depth for complex multi-step tasks

via “mathematical reasoning and symbolic computation”

Qwen3.5-9B is a multimodal foundation model from the Qwen3.5 family, designed to deliver strong reasoning, coding, and visual understanding in an efficient 9B-parameter architecture. It uses a unified vision-language design...

Unique: Unified architecture enables mathematical reasoning with visual context — can solve problems involving diagrams, charts, or visual representations of mathematical concepts, combining visual understanding with symbolic reasoning in a single forward pass

vs others: More efficient than GPT-4 for mathematical reasoning due to smaller parameter count, while maintaining competitive performance through specialized instruction-tuning; faster inference makes it suitable for real-time educational applications

via “multimodal visual reasoning with extended thinking”

Qwen3-VL-8B-Thinking is the reasoning-optimized variant of the Qwen3-VL-8B multimodal model, designed for advanced visual and textual reasoning across complex scenes, documents, and temporal sequences. It integrates enhanced multimodal alignment and...

Unique: Integrates extended chain-of-thought reasoning specifically for visual tasks, using a unified transformer backbone that maintains spatial-semantic alignment between vision and language modalities throughout the reasoning process, rather than treating vision as a feature extraction step followed by language-only reasoning

vs others: Outperforms standard vision-language models (GPT-4V, Claude 3.5 Vision) on complex reasoning tasks by dedicating compute to intermediate reasoning steps over images, though with higher latency and cost

via “visual perception and scene understanding with spatial reasoning”

Qwen3-VL-30B-A3B-Instruct is a multimodal model that unifies strong text generation with visual understanding for images and videos. Its Instruct variant optimizes instruction-following for general multimodal tasks. It excels in perception...

Unique: Implements dense spatial feature extraction with attention-based relationship modeling, enabling fine-grained understanding of object interactions and scene composition rather than just object classification

vs others: Outperforms CLIP-based approaches on spatial reasoning tasks and provides richer semantic descriptions than traditional computer vision pipelines while requiring no model training

via “multimodal-reasoning-and-visual-question-answering”


Unique: Integrates visual grounding with language reasoning, providing concrete strategies for building models that can explain their reasoning through attention visualization — addressing the gap between black-box VQA models and interpretable reasoning systems

vs others: Deeper treatment of compositional and multi-step reasoning in multimodal systems compared to single-task VQA papers; integrates interpretability as core design consideration

via “composition-aware object placement”

Make-A-Scene by Meta is a multimodal generative AI method puts creative control in the hands of people who use it by allowing them to describe and illustrate their vision through both text descriptions and freeform sketches.

via “complex compositional instruction following”

via “image composition and layout assistance”

Unique: Integrates composition guidance as an interactive overlay tool within the editor, allowing users to visualize composition principles while editing rather than consulting external design resources

vs others: More accessible than hiring a designer or taking composition courses because guidance is built into the tool; more practical than Photoshop's composition tools because suggestions are AI-powered and context-aware