Groq API

Delivers text generation inference using proprietary Language Processing Unit (LPU) hardware optimized for token throughput rather than general compute, achieving 500+ tokens/second sustained output. Routes requests through OpenAI-compatible `/responses` endpoint with bearer token authentication, enabling drop-in replacement for OpenAI clients while maintaining custom hardware acceleration. Supports streaming and batch processing modes for different latency/throughput trade-offs.

Provides access to diverse open-source and proprietary models (GPT OSS 120B/20B, Llama 3.3 70B, Llama 4 Scout, Qwen 3 32B, Mixtral variants) with native support for tool use, function calling, and explicit reasoning capabilities. Models support OpenAI-compatible function calling schema for structured tool integration. Reasoning models (GPT OSS 120B, Qwen 3 32B) expose chain-of-thought thinking tokens for transparency.

Integrates Wolfram Alpha computational engine as a tool for LLM agents, enabling models to solve mathematical problems, perform scientific calculations, and retrieve factual data. Models can formulate Wolfram Alpha queries, interpret results, and incorporate findings into responses. Provides access to Wolfram's knowledge base for physics, chemistry, biology, and other domains.

Supports Model Context Protocol (MCP) for connecting external tools, services, and data sources as standardized interfaces. Enables developers to build custom tool adapters (remote tools, local tools, database connectors) that integrate seamlessly with Groq's function-calling framework. MCP provides schema-based tool discovery, parameter validation, and error handling. Supports both local and remote MCP servers.

Provides pre-built connectors for Google Workspace services (Gmail, Google Calendar, Google Drive) enabling LLM agents to read/write emails, manage calendar events, and access documents. Connectors handle OAuth authentication, API pagination, and error handling. Agents can autonomously compose emails, schedule meetings, and retrieve file contents as part of multi-step workflows.

Provides OpenAI-compatible REST API endpoint (https://api.groq.com/openai/v1) accepting OpenAI SDK clients without code changes. Supports OpenAI Python SDK (openai package) and JavaScript SDK (openai npm package) by overriding baseURL and apiKey parameters. Maintains API contract compatibility for text generation, function calling, and streaming, enabling zero-migration-cost switching from OpenAI.

Offers free tier with monthly token allowance for experimentation and development, transitioning to pay-as-you-go pricing for production use. Developers can set spend limits to prevent unexpected charges. Billing is per-token (input and output tokens priced separately). Projects and API key management enable cost allocation across teams and applications.

Provides batch processing mode for non-real-time inference workloads, accepting multiple requests in bulk and processing them asynchronously with lower per-token cost than real-time API. Batch jobs are queued and processed during off-peak hours, trading latency for cost savings. Results are returned via webhook or polling. Ideal for large-scale data processing, content generation, and analysis tasks.

Provides speech recognition via OpenAI Whisper Large v3 and Whisper Large v3 Turbo models, accessible through Groq's LPU-accelerated inference. Whisper Turbo variant trades accuracy for 2-3x faster transcription latency. Supports audio input in standard formats (WAV, MP3, M4A, FLAC) with automatic language detection and optional language specification.

Generates natural speech audio from text using proprietary Orpheus text-to-speech models optimized for English and Arabic (Saudi) languages. Runs on Groq's LPU hardware for low-latency audio generation. Supports voice customization parameters (pitch, speed, emotion) and outputs standard audio formats (MP3, WAV, OGG).

Provides image analysis and vision-language understanding through Llama 4 Scout model, which accepts images as input alongside text prompts for multimodal reasoning. Processes images in standard formats (JPEG, PNG, WebP, GIF) and returns text descriptions, object detection, scene understanding, and visual question answering. Runs on Groq's LPU hardware for faster image processing than typical GPU-based vision models.

Provides content safety classification using Safety GPT OSS 20B model, which analyzes text for harmful content categories (violence, hate speech, sexual content, self-harm, illegal activity, etc.). Returns safety scores and category classifications for moderation workflows. Runs on Groq's LPU hardware for fast moderation decisions in real-time applications.

Enables models to generate structured JSON output conforming to developer-specified schemas, ensuring valid, parseable responses for downstream processing. Supports JSON Schema format for defining output structure, field types, and constraints. Model enforces schema compliance during generation, preventing invalid JSON or missing required fields. Reduces post-processing and error handling overhead.

Caches frequently-used prompt prefixes (system prompts, few-shot examples, long documents) to avoid reprocessing identical context across multiple requests. Subsequent requests with cached prefix pay reduced token cost and receive faster processing. Groq's LPU hardware enables efficient cache management without significant latency overhead. Particularly valuable for multi-turn conversations and document-based QA.

Integrates web search capability into LLM responses, enabling models to retrieve and cite current information from the internet. Models (GPT OSS 120B, GPT OSS 20B, Llama 3.3 70B) can autonomously decide when to search, formulate search queries, and synthesize results into responses. Provides source attribution and links for transparency. Runs on Groq's LPU hardware for fast search-augmented generation.

Enables LLM agents to execute code (Python, JavaScript) and automate browser interactions (click, type, navigate, screenshot) to complete multi-step tasks. Models can inspect page state, make decisions, and execute actions iteratively. Runs in sandboxed environment with timeout and resource limits. Integrates with Groq's tool-calling framework for structured action invocation.