google-generativeai

Generates text responses from prompts containing text, images, audio, and video inputs using Google's Gemini models. Implements streaming via server-sent events (SSE) for real-time token delivery, with automatic batching of multimodal content into a unified request payload. Supports both synchronous blocking calls and asynchronous streaming for integration into event-driven architectures.

Enables models to invoke external functions by declaring a schema of available tools upfront and letting the model decide when/how to call them. Implements automatic serialization of function signatures into JSON Schema format, with built-in validation of model-generated function calls against declared schemas. Supports both single-turn tool invocation and multi-turn agentic loops where the model can chain multiple function calls.

Allows setting system-level instructions that define the model's behavior, tone, and constraints across all turns in a conversation. System instructions are passed as a separate parameter distinct from user messages, enabling role-based prompting (e.g., 'You are a helpful assistant', 'You are a code reviewer'). Instructions are applied consistently across multi-turn conversations without requiring repetition in each user message.

Implements client-side rate limiting and quota management to prevent exceeding API rate limits and quota thresholds. Automatically backs off and retries requests when rate limit errors are encountered, with exponential backoff strategy and configurable retry parameters. Tracks quota usage across requests and provides methods to check remaining quota before submitting new requests.

Maintains a stateful conversation history across multiple turns, automatically managing token limits by truncating or summarizing older messages when context window is exceeded. Implements a simple list-based history structure where each message is tagged with role (user/model) and content, with built-in methods to append new messages and retrieve the full conversation for re-submission to the API.

Converts text or multimodal content into high-dimensional dense vector embeddings suitable for semantic search, clustering, or similarity comparison. Uses Google's embedding models (e.g., embedding-001) which produce 768-dimensional vectors optimized for semantic relevance. Supports batch embedding of multiple texts in a single API call, with automatic chunking for large inputs.

Filters generated content based on safety categories (hate speech, sexual content, violence, harassment) with configurable threshold levels (BLOCK_NONE, BLOCK_ONLY_HIGH, BLOCK_MEDIUM_AND_ABOVE, BLOCK_LOW_AND_ABOVE). Safety filters are applied server-side by the Gemini API, with client-side configuration passed as request parameters. Blocked responses return a safety_ratings object indicating which categories triggered the block.

Provides runtime access to model metadata including supported input types, context window size, maximum output tokens, and available features (function calling, vision, etc.). Implements a model registry that can be queried to list all available models and their capabilities without hardcoding model names. Supports model versioning with automatic fallback to stable versions if a specific version is unavailable.

Submits multiple generation requests asynchronously and collects results without blocking on individual responses. Implements async/await patterns using Python's asyncio library, enabling concurrent API calls with configurable concurrency limits. Batch results are returned as a list or async generator, allowing streaming processing of large batches without loading all results into memory.

Uploads files (documents, images, audio, video) to Google's servers and returns file references that can be reused across multiple API calls without re-uploading. Implements automatic MIME type detection and chunked upload for large files. Cached files are stored server-side for a configurable TTL (time-to-live), reducing bandwidth and API costs for repeated use of the same content.

Constrains model output to follow a specified JSON schema, ensuring responses are valid, parseable structured data. Implements schema-based output formatting via a response_schema parameter that accepts JSON Schema definitions or Pydantic models. The model is instructed to generate output matching the schema, with server-side validation to reject non-conforming responses.

Estimates the number of tokens a prompt will consume before sending it to the API, enabling cost prediction and budget management. Implements token counting via the same tokenizer used by the model, with support for multimodal content (images, audio, video). Provides both prompt token count and estimated completion token count based on model behavior patterns.