Cohere: Command R (08-2024)

Implements RAG by accepting external document context and grounding responses in retrieved passages across 100+ languages. The model architecture includes a retrieval-aware attention mechanism that weights retrieved documents during generation, enabling factual accuracy and citation-aware outputs. Supports both in-context document injection and integration with external vector databases via tool-use APIs.

Implements function calling via a JSON schema registry where developers define tool signatures (name, description, parameters) and the model outputs structured tool calls that can be dispatched to external APIs or local functions. The model learns to invoke tools based on task requirements, supporting multi-turn tool use where outputs from one tool feed into subsequent calls. Integration points include OpenRouter's tool-calling API, native Cohere API, and custom orchestration layers.

Generates code across multiple programming languages and solves mathematical problems by breaking down reasoning into intermediate steps. The model uses chain-of-thought patterns internally, producing both executable code and step-by-step mathematical derivations. Supports code completion, bug fixing, and algorithm explanation. The 08-2024 update improves performance on complex math and multi-language code generation through enhanced training on mathematical datasets and code repositories.

Maintains conversation state across multiple turns, tracking user intent and context without explicit memory management. The model processes the full conversation history (within token limits) to generate contextually appropriate responses. Supports persona customization through system prompts and handles topic switching, clarification requests, and context recovery. Integration via chat completion APIs that accept message arrays with role-based formatting (user/assistant/system).

Supports semantic search by accepting query text and returning ranked results based on semantic similarity rather than keyword matching. The model can be used as a reranker in retrieval pipelines, taking candidate documents and a query, then scoring relevance. Integrates with vector databases and BM25 indices through API calls. The 08-2024 update improves multilingual search by handling cross-lingual queries where the search language differs from document language.

Accepts system prompts to customize model behavior, tone, and constraints without fine-tuning. The model interprets system instructions and applies them consistently across the conversation. Supports complex instructions like role-playing, output format specifications, and behavioral constraints. Implementation uses instruction-tuning from training, where the model learned to follow diverse instructions through supervised fine-tuning on instruction-following datasets.

Supports batch API endpoints where developers submit multiple requests in a single API call, receiving results asynchronously. Useful for processing large document collections, bulk classification, or offline analysis. The batch endpoint queues requests and returns results via callback or polling. This reduces per-request overhead and enables cost optimization through batch pricing discounts.

Streams response tokens in real-time as they are generated, enabling progressive display in user interfaces without waiting for the full response. Implementation uses server-sent events (SSE) or WebSocket connections to push tokens to the client. Reduces perceived latency and improves user experience for long-form content generation. Supports streaming of both text and structured outputs (e.g., JSON tokens).