Reka API

Processes video files natively (not as frame extraction + text model) to understand temporal sequences, motion, scene changes, and narrative flow. The API accepts video inputs directly and performs joint reasoning across visual frames, audio tracks, and temporal context in a single forward pass, enabling detection of events that require understanding of change over time rather than static image analysis.

Analyzes audio content to extract meaning, emotion, intent, and semantic information rather than just converting speech to text. The API processes audio signals to understand speaker intent, emotional tone, background context, and non-speech audio elements (music, ambient sounds, effects) in a unified model, returning structured semantic understanding rather than transcription-only output.

Extracts structured information from images, video, and audio content and returns it in a machine-readable format (JSON, CSV, etc.). The capability can extract entities, relationships, attributes, and other structured data without requiring manual annotation or separate extraction models, enabling automation of data collection from unstructured multimodal sources.

Generates vector embeddings that represent content across video, image, audio, and text modalities in a shared embedding space, enabling semantic search and similarity matching across different input types. A single query (text, image, or audio) can retrieve relevant results from a database containing mixed media types, with embeddings computed through the same multimodal model ensuring semantic alignment across modalities.

Generates natural language descriptions of image content, including object identification, spatial relationships, scene context, and semantic meaning. The model analyzes visual input and produces human-readable captions that can range from short summaries to detailed descriptions, with the ability to customize caption length and detail level through API parameters.

Identifies and localizes objects within images by returning bounding box coordinates, class labels, and confidence scores. The model detects multiple object instances in a single image and provides spatial information enabling downstream applications to reference specific regions of interest, with support for custom object classes through prompt-based detection.

Answers natural language questions about image and video content by analyzing visual information and generating contextual responses. The model accepts an image or video and a text question, then produces an answer that demonstrates understanding of visual content, spatial relationships, object properties, and temporal events (for video). Questions can range from factual identification to reasoning about relationships and implications.

Provides three distinct model variants (Reka Core, Reka Flash, Reka Edge) with different performance characteristics, latency profiles, and pricing tiers. Developers select the appropriate model based on their accuracy requirements, latency constraints, and cost budget, with each model supporting the full multimodal capability set but with different quality-speed-cost tradeoffs. Model selection is specified at API request time.

Supports processing multiple images, videos, or audio files in batch mode with asynchronous job submission and result polling or webhook callbacks. Developers submit batch jobs containing multiple media files and receive a job ID, then retrieve results once processing completes, enabling efficient processing of large content libraries without blocking on individual API calls. Implementation details (polling interval, webhook format, job timeout) not documented.

Maintains a context window that can simultaneously hold text, images, video, and audio content, enabling the model to reason across modalities within a single inference pass. The model can answer questions about relationships between visual and textual content, reference specific moments in video while discussing text, or correlate audio tone with visual events, all without separate API calls or external coordination logic.

Analyzes images, video, and audio content to detect and classify potentially harmful, inappropriate, or policy-violating material. The capability performs safety classification across multiple dimensions (violence, sexual content, hate speech, etc.) and can be used to flag content for human review or automatically reject submissions that violate platform policies.