Runway API

Converts natural language prompts into video sequences using Gen-3 Alpha's diffusion-based architecture, which processes text embeddings through a temporal transformer stack to generate frame sequences with coherent motion. The API accepts detailed motion descriptors and camera movement parameters (pan, zoom, dolly) that are encoded into the generation pipeline, enabling fine-grained control over cinematography without requiring manual keyframing or post-processing.

Transforms a static image into a video sequence by using the image as a conditioning anchor in the temporal diffusion process. The API encodes the input image into latent space, then generates subsequent frames by sampling from a distribution that maintains visual consistency with the anchor while introducing motion dynamics specified via prompts or motion vectors. This approach preserves fine details and lighting from the source image while enabling natural motion evolution.

Accepts an existing video as input and regenerates it with modifications to style, motion, or content while preserving temporal structure and shot composition. The API uses optical flow estimation to track motion patterns in the source video, then applies a guided diffusion process that respects the original motion while applying new stylistic or content transformations. This enables non-destructive video editing workflows where motion is preserved but visual appearance is radically altered.

Provides a non-blocking API interface for submitting multiple video generation requests and receiving results via webhook callbacks or polling. Requests are queued and processed by distributed worker nodes, with status tracking via unique request IDs. The API supports batch submission of up to 100 requests per call, enabling high-throughput video production pipelines without blocking client connections or managing long-lived HTTP connections.

Provides a structured parameter schema for specifying camera movements (pan, tilt, zoom, dolly, crane) as JSON objects that are injected into the video generation pipeline. Parameters are normalized to a standard coordinate system and applied as conditioning signals during diffusion sampling, enabling reproducible and physically plausible camera movements. The API supports both absolute camera paths (keyframe-based) and relative motion descriptors (e.g., 'slow pan left').

Accepts an optional seed parameter that controls the random number generator used during diffusion sampling, enabling exact reproduction of generated videos or controlled variation across multiple generations. The same seed with identical inputs produces byte-identical output; different seeds with the same prompt produce stylistic variations while maintaining semantic consistency. This enables A/B testing, version control of generated content, and deterministic workflows.

Accepts video and image inputs in multiple formats (MP4, MOV, WebM, JPEG, PNG, WebP) and outputs videos in H.264 MP4 format with configurable bitrate and resolution. The API automatically detects input format and codec, handles color space conversion (sRGB, Rec.709, DCI-P3), and applies appropriate preprocessing (deinterlacing, frame rate normalization) before generation. Output bitrate can be specified to balance quality and file size.

Returns metadata alongside generated videos including quality metrics (temporal consistency score, motion smoothness, visual fidelity), confidence scores for motion estimation, and diagnostic information (processing time, model version, generation parameters). These metrics enable downstream systems to filter or re-generate low-quality outputs automatically and provide transparency into generation quality without manual review.

Implements token-based rate limiting and quota management, tracking API usage per key and enforcing limits on requests per minute, concurrent generations, and total monthly quota. The API returns rate limit headers (X-RateLimit-Remaining, X-RateLimit-Reset) and quota metadata in responses, enabling clients to implement backoff strategies and quota-aware scheduling. Quota can be purchased in tiers or pay-as-you-go models.

Returns structured error responses with specific error codes, human-readable messages, and diagnostic information (e.g., why a prompt was rejected, which parameter caused failure). Supports automatic retry with exponential backoff for transient failures (rate limits, temporary service degradation) while distinguishing from permanent failures (invalid parameters, unsupported content). Provides suggestions for correcting common errors (e.g., 'prompt too long, reduce to under 500 characters').