Inference And Serving Framework Discovery With Deployment Pattern Guidance

🧑‍🚀 全世界最好的LLM资料总结（多模态生成、Agent、辅助编程、AI审稿、数据处理、模型训练、模型推理、o1 模型、MCP、小语言模型、视觉语言模型） | Summary of the world's best LLM resources.

Unique: Organizes inference frameworks by deployment pattern (local, cloud, edge, batch) rather than just framework name, with explicit mapping to optimization techniques (quantization, batching, KV-cache) and hardware targets. Includes both open-source engines (vLLM, SGLang, Ollama) and commercial platforms (Together AI, Replicate).

vs others: More deployment-pattern-focused than framework-specific documentation; enables builders to find solutions by use case (low-latency API, batch processing, edge deployment) rather than learning individual framework APIs.

via “distributed inference serving”