NVIDIA: Nemotron 3 Super

Nemotron 3 Super uses a hybrid Mamba-Transformer architecture with sparse Mixture of Experts (MoE) routing that activates only 12B of 120B parameters per forward pass. The model employs learned gating mechanisms to route tokens to specialized expert sub-networks, reducing computational cost while maintaining model capacity. This sparse activation pattern is computed dynamically based on input tokens, enabling efficient inference on consumer-grade hardware without quantization.

Nemotron 3 Super is optimized for multi-agent applications where multiple specialized agents coordinate to solve complex tasks. The model maintains coherent context across extended conversations, tracking agent roles, responsibilities, and shared state. The architecture supports deep reasoning chains where agents build on each other's outputs, with the sparse MoE design ensuring each agent's specialized reasoning path activates relevant experts without full model overhead.

Nemotron 3 Super generates code across multiple programming languages and can understand multi-file codebases for refactoring tasks. The model uses its extended context window and reasoning capabilities to track dependencies between files, suggest structural improvements, and generate coherent changes across a codebase. The sparse MoE architecture allows code-specific experts to activate for syntax-aware generation while general reasoning experts handle architectural decisions.

Nemotron 3 Super excels at breaking down complex problems into reasoning steps, generating explicit intermediate reasoning before final answers. The model can produce detailed chain-of-thought traces for mathematical problems, logical reasoning, and multi-step planning tasks. The hybrid Mamba-Transformer architecture provides both efficient sequence modeling (Mamba) and attention-based reasoning (Transformer), enabling coherent multi-step reasoning without excessive parameter activation.

Nemotron 3 Super is accessed exclusively through OpenRouter's API, supporting both streaming (token-by-token) and batch inference modes. The API abstracts away the underlying sparse MoE complexity, presenting a standard LLM interface. Streaming enables real-time response generation for interactive applications, while batch processing allows cost-optimized throughput for non-latency-sensitive workloads. The sparse activation is handled transparently by the inference backend.

Nemotron 3 Super can process and synthesize information from extended documents, generating summaries, extracting key points, and answering questions about document content. The model's extended context window and efficient sparse activation enable processing of longer documents than typical dense models without excessive latency. The reasoning capabilities allow nuanced synthesis rather than simple extractive summarization.

Nemotron 3 Super is trained to follow detailed instructions and adapt behavior based on system prompts and task specifications. The model can adjust tone, style, output format, and reasoning approach based on explicit instructions. This capability enables single-model deployment across diverse applications without model switching. The sparse MoE design allows task-specific experts to activate based on instruction content, improving efficiency for specialized tasks.