Capability
Local Llm Inference With Latency Optimization
20 artifacts provide this capability.
Want a personalized recommendation?
Find the best match →Top Matches
1
DeepSeek Coder V2Model47/100
via “efficient inference through sglang and vllm framework integration”
DeepSeek's 236B MoE model specialized for code.
Unique: Provides native SGLang integration with MLA optimizations and vLLM support with MoE-aware batching, enabling 30-50% latency reduction through framework-specific routing and attention optimizations vs generic Transformers inference
vs others: Outperforms standard Transformers library inference by 30-50% through MoE-aware scheduling and achieves comparable latency to proprietary APIs while remaining deployable locally