Together AI Platform

Provides on-demand REST API access to 100+ pre-hosted open-source LLM models (Llama, Qwen, DeepSeek, Gemma, etc.) without requiring infrastructure provisioning. Models are deployed across NVIDIA GPU clusters with automatic request routing and load balancing. Token-based pricing charges separately for input and output tokens, with optional prompt caching for reduced costs on repeated contexts. Developers call a single endpoint and receive streamed or batch responses without managing model weights, VRAM allocation, or GPU scheduling.

Asynchronous batch processing API that accepts large volumes of inference requests (up to 30 billion tokens per model per batch) and processes them at lower cost (50% reduction vs real-time API) by optimizing GPU utilization and request scheduling. Requests are queued, batched by model, and processed during off-peak or scheduled windows. Results are stored and retrieved via polling or webhook callbacks. Designed for non-latency-sensitive workloads like data labeling, content generation, or periodic model evaluation.

Support for function calling (tool use) across text, vision, and audio models via schema-based function definitions. Developers define functions as JSON schemas, and models return structured function call arguments. Supports parallel function calling (multiple tools in one response) and tool result feedback loops. Integrated into the same REST API as inference, enabling agentic workflows without separate tool orchestration infrastructure.

Automatic caching of prompt prefixes (system prompts, context, documents) to reduce token costs on repeated requests. When the same prefix is used multiple times, subsequent requests pay reduced rates for cached tokens (exact reduction not specified per model). Implemented at the API level; developers specify cache control headers or parameters. Designed for applications with static context (e.g., RAG with the same documents, multi-turn conversations with system prompts) that repeat across requests.

Proprietary inference optimizations developed through published research and implemented as custom CUDA kernels (FlashAttention-4, distribution-aware speculative decoding, ATLAS runtime-learning accelerators). These optimizations are transparently applied to all inference requests without developer configuration. Reduces latency and increases throughput compared to standard inference implementations. Backed by peer-reviewed research papers published by Together AI team.

Serverless inference for vision models including image generation (FLUX, Stable Diffusion, Qwen Image), image analysis, and visual understanding. Image generation is priced per image or per megapixel depending on model, with configurable step counts (e.g., FLUX.1 schnell at 4 steps). Vision models accept image inputs (format not specified) and return generated or analyzed outputs. Integrated into the same REST API as text models, allowing multi-modal workflows without separate endpoints.

Serverless inference for audio generation, audio transcription, and video generation models. Audio models handle text-to-speech and audio synthesis; transcription models convert audio files to text. Video generation models create videos from text prompts or images. All models are accessed via the same REST API as text and image models. Pricing structure for audio/video not fully specified in public documentation (contact sales for details).

Serverless inference for embedding models that convert text into high-dimensional vectors for semantic search, similarity matching, and RAG (Retrieval-Augmented Generation) applications. Embeddings are generated via REST API and can be stored in external vector databases (Pinecone, Weaviate, Milvus, etc.) or Together AI's Managed Storage. Supports batch embedding generation for large document corpora. Pricing is per-token (same as text models), making it cost-effective for embedding large datasets.

Serverless inference for reranking models that score and reorder search results based on relevance to a query. Rerankers accept a query and a list of candidate documents/passages and return ranked scores. Used in RAG pipelines to improve retrieval quality by reordering results from semantic search or keyword search. Integrated into the same REST API as other models, with token-based pricing.

Serverless inference for content moderation models that classify text for policy violations (hate speech, violence, sexual content, etc.). Models return classification scores or labels indicating content safety. Integrated into the same REST API as other models. Can be used to filter user-generated content, moderate chat applications, or audit training data for harmful content.

Platform service for fine-tuning open-source models on custom datasets and deploying fine-tuned models as serverless inference endpoints. Developers upload training data (format not specified), configure hyperparameters, and Together AI manages the fine-tuning job on dedicated GPUs. Fine-tuned models are stored on the platform and can be invoked via the same REST API as pre-hosted models. Pricing for fine-tuning not publicly specified (contact sales).

Self-service provisioning of dedicated NVIDIA GPU clusters for custom inference, fine-tuning, or other ML workloads. Developers select GPU type and quantity, and Together AI provisions a cluster accessible via SSH or containerized inference endpoints. Clusters can run custom code, custom models, or proprietary inference engines. Pricing is per-GPU-hour (exact rates not specified; contact sales). Designed for teams needing full control over infrastructure and workloads not supported by serverless APIs.

Persistent storage service for model weights, training data, fine-tuned models, and inference results. Integrated with fine-tuning and inference services; fine-tuned models are automatically stored and versioned. Offers zero egress fees (data can be downloaded without additional charges). Storage pricing not publicly specified (contact sales). Designed to reduce data transfer costs and simplify artifact management for ML workflows.