cohere

Provides a unified Python client interface (Client, AsyncClient, ClientV2, AsyncClientV2) that abstracts away platform-specific differences across Cohere's hosted API, AWS Bedrock, AWS SageMaker, Azure, GCP, and Oracle Cloud. Uses a layered architecture with BaseClientWrapper handling authentication token management and HTTP headers, while SyncClientWrapper and AsyncClientWrapper extend this for synchronous and asynchronous execution modes respectively. Developers write once and deploy across multiple cloud providers without changing application code.

Implements real-time chat response streaming via the chat_stream endpoint, allowing developers to consume LLM responses token-by-token as they're generated rather than waiting for complete responses. Uses HTTP streaming (chunked transfer encoding) to deliver partial responses, enabling low-latency UI updates and progressive text rendering. Supports both synchronous and asynchronous streaming patterns through dedicated stream methods that yield response chunks.

Supports batch processing of multiple inputs in single API calls for endpoints like embed, classify, and rerank, reducing overhead and improving throughput compared to individual requests. Batch operations accept lists of inputs and return lists of outputs with consistent ordering, enabling efficient processing of large datasets. Batch sizes are limited per endpoint (typically 96 items) to balance throughput and latency, with automatic batching handled by the application.

Includes detailed metadata in API responses such as token usage (input/output tokens), model version, generation ID, and finish reason (complete, max_tokens, etc.). This metadata enables cost tracking, quota management, and debugging of model behavior. The SDK automatically includes this information in response objects, allowing applications to monitor API consumption without additional tracking logic.

Generates dense vector embeddings (typically 1024-4096 dimensions) for text and image inputs via the embed endpoint, converting unstructured content into fixed-size numerical representations suitable for semantic search, clustering, and similarity comparisons. Supports batch processing of multiple inputs in a single API call, with configurable embedding dimensions and input types. Returns embedding vectors alongside metadata about token usage and model version.

Reorders a list of documents or texts based on their relevance to a query using a specialized reranking model, producing relevance scores for each item. Takes a query and a list of candidate texts, then returns the same texts sorted by relevance with associated scores (typically 0-1 range). Useful for post-processing search results or ranking candidates from a larger corpus. Operates via the rerank endpoint with support for batch processing.

Classifies input text into one or more predefined categories using a fine-tuned classification model via the classify endpoint. Accepts a list of texts and a list of category labels, returning predicted class labels and confidence scores for each input. Supports both single-label and multi-label classification scenarios. Uses the model's semantic understanding to match text to categories without requiring training data.

Provides tokenize and detokenize endpoints for converting between text and token representations using Cohere's tokenizer. The tokenize endpoint breaks text into tokens (subword units) and returns token IDs and counts, useful for understanding token consumption and managing context windows. The detokenize endpoint reverses this process, converting token IDs back into readable text. Both operations use the same tokenizer as the LLM models, ensuring consistency.

Provides parallel client implementations for both synchronous (Client, ClientV2) and asynchronous (AsyncClient, AsyncClientV2) execution patterns, allowing developers to choose the execution model that fits their application architecture. Synchronous clients use blocking HTTP calls suitable for scripts and simple applications, while asynchronous clients use async/await patterns with non-blocking I/O, enabling high-concurrency scenarios. Both client types share identical API method signatures, allowing easy switching between execution modes.

Supports both Cohere API v1 and v2 through separate client implementations (Client/AsyncClient for v1, ClientV2/AsyncClientV2 for v2), allowing developers to use legacy v1 endpoints or adopt v2's enhanced features. Each API version has its own request/response models, endpoint signatures, and capabilities. Developers can instantiate either version based on their requirements, with v2 providing newer models and improved API design while v1 maintains backward compatibility.

Implements flexible authentication through the BaseClientWrapper that supports both explicit API key passing and environment variable-based configuration (CO_API_KEY). The wrapper handles token lifecycle management, including header construction with Bearer token authentication and automatic token injection into all HTTP requests. Supports multiple authentication methods across different platforms (Cohere API key, AWS credentials, Azure tokens, etc.) with platform-specific credential handling.

Implements comprehensive error handling that captures HTTP errors, authentication failures, rate limiting, and API-specific errors, returning structured exception objects with error codes, messages, and metadata. The error handling layer in the client wrapper catches HTTP exceptions and transforms them into SDK-specific exceptions, providing context about the failure (e.g., 401 for auth failures, 429 for rate limits, 500 for server errors). Supports graceful degradation and retry logic at the application level.