Knowledge Distillation Based Reasoning Transfer

via “reasoning model distillation to smaller parameter scales”

Open-source reasoning model matching OpenAI o1.

Unique: Applies distillation to reasoning models across 6 different scales (1.5B-70B), which is rare for frontier reasoning models. Most competitors only offer single-size deployment.

vs others: Provides multiple distilled sizes enabling flexible deployment, whereas o1 only offers cloud API access at fixed capability level.

via “logical-reasoning-and-formal-inference”

INTELLECT-3 is a 106B-parameter Mixture-of-Experts model (12B active) post-trained from GLM-4.5-Air-Base using supervised fine-tuning (SFT) followed by large-scale reinforcement learning (RL). It offers state-of-the-art performance for its size across math,...

Unique: RL post-training optimizes for logical consistency and formal correctness in reasoning traces; uses chain-of-thought patterns that decompose inference into verifiable steps rather than end-to-end black-box reasoning

vs others: Produces more transparent and verifiable reasoning than single-step models while maintaining efficiency through MoE routing that activates only reasoning-specific experts

via “knowledge distillation-based reasoning transfer”

DeepSeek R1 Distill Qwen 32B is a distilled large language model based on [Qwen 2.5 32B](https://huggingface.co/Qwen/Qwen2.5-32B), using outputs from [DeepSeek R1](/deepseek/deepseek-r1). It outperforms OpenAI's o1-mini across various benchmarks, achieving new...

Unique: Uses knowledge distillation to transfer R1's reasoning capability to a 32B model, enabling R1-quality reasoning at 1/3 parameter count through supervised fine-tuning on R1 outputs

vs others: More efficient than full R1 while maintaining reasoning quality, and more transparent than black-box reasoning models like o1 through explicit reasoning traces

via “knowledge distillation-based reasoning compression”

Aion-1.0-Mini 32B parameter model is a distilled version of the DeepSeek-R1 model, designed for strong performance in reasoning domains such as mathematics, coding, and logic. It is a modified variant...

Unique: Applies knowledge distillation to compress DeepSeek-R1's reasoning capability into 32B parameters, enabling reasoning-based inference at lower cost and latency than full R1

vs others: More efficient than full R1 (32B vs 671B) while retaining reasoning capability, though with unknown performance trade-offs vs. non-distilled reasoning models

via “domain-specific knowledge synthesis and explanation”

DeepSeek R1 Distill Llama 70B is a distilled large language model based on [Llama-3.3-70B-Instruct](/meta-llama/llama-3.3-70b-instruct), using outputs from [DeepSeek R1](/deepseek/deepseek-r1). The model combines advanced distillation techniques to achieve high performance across...

Unique: Embeds R1's reasoning distillation into domain knowledge synthesis, enabling the model to not just retrieve facts but reason through their implications and connections. This produces more coherent, logically-sound explanations than fact-retrieval alone, particularly for interdisciplinary questions.

vs others: Provides reasoning-transparent domain explanations with lower latency than full R1, while offering stronger logical coherence than base Llama-3.3 due to R1 distillation.

via “complex question answering with source reasoning”

WizardLM-2 8x22B is Microsoft AI's most advanced Wizard model. It demonstrates highly competitive performance compared to leading proprietary models, and it consistently outperforms all existing state-of-the-art opensource models. It is...

Unique: Trained with instruction-following on reasoning-heavy datasets that emphasize explicit working-through of complex questions; mixture-of-experts architecture allows different expert pathways for factual vs. analytical reasoning, improving accuracy across diverse question types

vs others: Demonstrates stronger reasoning transparency and multi-step problem solving than many open models while maintaining competitive accuracy with proprietary models, with explicit training for acknowledging uncertainty rather than confident hallucination

via “reasoning and chain-of-thought problem decomposition”

|[GitHub](https://github.com/meta-llama/llama3) | Free |

Unique: Instruction-tuned specifically on reasoning-focused datasets with explicit step-by-step annotations, enabling the model to naturally generate transparent reasoning traces without requiring special prompting techniques. The 70B parameter scale allows for nuanced reasoning across diverse domains while maintaining interpretability of intermediate steps.

vs others: More transparent and auditable reasoning than models optimized purely for answer accuracy, with reasoning traces that can be validated and debugged by domain experts, though less specialized than dedicated symbolic reasoning systems or theorem provers.