Groq API

Generates text using Groq's custom LPU (Language Processing Unit) hardware, which achieves 500+ tokens/second throughput by parallelizing token computation across specialized silicon. Implements OpenAI API compatibility layer, allowing drop-in replacement via custom baseURL parameter without SDK changes. Supports models including GPT-OSS-120B, GPT-OSS-20B, Llama-4-Scout, Llama-3.3-70B, and Qwen-3-32B with streaming and batch processing tiers.

Enables models (GPT-OSS-120B, GPT-OSS-20B, Llama-4-Scout, Qwen-3-32B) to invoke external tools by generating structured function calls based on a provided schema. Works by embedding tool definitions in the system prompt or via function parameter arrays, allowing the model to decide when and how to call tools. Integrates with built-in tools (Web Search, Browser Automation, Code Execution, Wolfram Alpha) and supports remote tools via MCP (Model Context Protocol) connectors.

Enables models to automate browser interactions (clicking, typing, navigation) and execute code in a sandboxed environment. Available as built-in tools that can be invoked via function calling. Browser Automation allows the model to interact with web pages as if a human were using them. Code Execution allows the model to run Python or JavaScript code and see results. Both tools integrate into the same function-calling system as Web Search.

Provides native connectors for Google Workspace services (Gmail, Google Calendar, Google Drive) that can be invoked via function calling. Models can read/write emails, manage calendar events, and access files without requiring custom OAuth implementation. Connectors are described as 'now available,' suggesting recent addition. Exact API surface (read-only vs. write, supported operations) is not documented.

Offers a 'Flex Processing' service tier alongside real-time and batch tiers, allowing users to optimize for different workload patterns. Exact characteristics of Flex Processing (latency SLA, pricing, use cases) are not documented. Mentioned as available tier in documentation but implementation details are absent.

Provides free access to Groq API with rate limits and quota restrictions, allowing developers to experiment and build prototypes without payment. Free tier includes access to multiple models and all core features (text generation, function calling, etc.). Exact rate limits, quota sizes, and feature restrictions are not documented.

Offers free tier with monthly token allowance for experimentation and development, transitioning to pay-as-you-go pricing for production use. Developers can set spend limits to prevent unexpected charges. Billing is per-token (input and output tokens priced separately). Projects and API key management enable cost allocation across teams and applications.

Provides batch processing mode for non-real-time inference workloads, accepting multiple requests in bulk and processing them asynchronously with lower per-token cost than real-time API. Batch jobs are queued and processed during off-peak hours, trading latency for cost savings. Results are returned via webhook or polling. Ideal for large-scale data processing, content generation, and analysis tasks.

Processes images and audio inputs alongside text using specialized models: Llama-4-Scout for vision tasks and Whisper-Large-v3 (or Turbo variant) for speech-to-text transcription. Vision model accepts images in unspecified formats and returns structured analysis or text descriptions. Whisper models transcribe audio to text with language detection. Both modalities integrate into the same `/responses` endpoint as text generation, allowing multimodal reasoning chains.

Uses Safety-GPT-OSS-20B model to classify and filter potentially harmful content (hate speech, violence, sexual content, etc.). Operates as a separate model endpoint that can be called before or after generation to validate prompts or outputs. Returns safety classification scores or filtered text depending on configuration. Integrates into the same `/responses` endpoint as other models.

Enables extended reasoning capabilities on models supporting reasoning tasks (GPT-OSS-120B, GPT-OSS-20B, Qwen-3-32B). Models can generate intermediate reasoning steps before producing final answers, improving accuracy on complex problems. Reasoning is triggered via prompt engineering or dedicated reasoning parameters (if supported). Works within the same `/responses` endpoint and respects the same token limits as standard generation.

Supports batch processing tier for non-real-time inference workloads, allowing multiple requests to be submitted together and processed asynchronously. Reduces per-request costs compared to real-time inference by amortizing overhead across batches. Exact batch size limits, processing time SLAs, and submission/retrieval mechanisms are not documented. Mentioned as 'Batch Processing' service tier in documentation.

Caches prompt prefixes (system prompts, context, examples) to avoid reprocessing identical input sequences across multiple requests. When the same prefix is used in subsequent requests, the cached tokens are reused, reducing latency and token consumption. Mechanism and configuration details are not documented, but caching is listed as a documented feature. Works within the same `/responses` endpoint.

Constrains model outputs to match a provided JSON schema, ensuring generated text conforms to a specific structure (e.g., extracting fields into a JSON object). Works by embedding schema constraints into the generation process, preventing the model from producing invalid JSON. Exact implementation (grammar-based constraints, post-generation validation, or native model support) is not documented. Listed as a documented feature but details are absent.

Converts text to natural-sounding speech using Orpheus models (Orpheus-English, Orpheus-Arabic-Saudi). Models are accessed via the same `/responses` endpoint as text generation. Output is audio in unspecified format. Supports at least English and Arabic (Saudi dialect), with language selection via model parameter. Voice characteristics and audio quality settings are not documented.

Enables models to search the web and incorporate current information into responses. Web Search is available as a built-in tool that can be invoked via function calling. When triggered, the model queries the web and receives search results, which it can then use to answer user questions. Exact search provider, result format, and integration mechanism are not documented. Supported on GPT-OSS models and Llama-4-Scout.