Runway ML

Generates video sequences from natural language text prompts using Gen-4.5 diffusion models running asynchronously in Runway's cloud infrastructure. The system accepts free-form text descriptions and outputs video files through a credit-metered consumption model (625 credits/month on Standard tier = ~25 seconds of video). Processing occurs server-side with no local inference capability, returning completed videos to the web editor or via API after variable latency (specific timing unknown).

Converts static images into video sequences by applying learned motion patterns and temporal coherence through Gen-4 or Gen-4 Turbo diffusion models. Users upload an image and optionally provide a text prompt to guide motion direction and style. The system generates video frames that maintain visual consistency with the source image while introducing realistic motion, processed asynchronously in Runway's cloud infrastructure with credit consumption (Gen-4 Turbo costs fewer credits than Gen-4.5 text-to-video).

Runway's built-in web-based video editor providing timeline-based editing with integrated access to generative capabilities (text-to-video, inpainting, motion brush, background removal, upscaling). The editor operates as a unified interface combining traditional video editing workflows with AI-powered content generation, allowing users to compose, edit, and enhance videos without context-switching to external tools. Available on Standard tier and above.

Enables users to define and execute multi-step workflows combining multiple generative and editing operations without manual intervention. Available on Standard tier and above, workflows allow chaining operations (e.g., text-to-video → inpainting → upscaling → watermark removal) with parameter passing between steps. Implementation details unknown, but likely uses a visual workflow builder or scripting language to define operation sequences.

Generates spoken audio from text using neural text-to-speech models, with optional custom voice training available on Pro tier and above. Users provide text and select a voice (pre-trained or custom), and the system generates synchronized audio suitable for video voiceovers or avatar lip-sync. Custom voice training allows users to create personalized voices by providing audio samples, enabling branded or character-specific speech synthesis.

Runway implements a proprietary credit-based consumption system where each generative operation consumes a fixed number of credits based on output length, model, and quality tier. Users purchase monthly credit allowances (Free: 125 one-time, Standard: 625/month, Pro: 2,250/month, Unlimited: 2,250/month + relaxed-rate exploration) that are consumed per operation. Credits do not roll over, and the system enforces hard limits on monthly usage, creating a predictable cost model but also usage ceilings.

Provides project-based organization of video generation and editing work, with separate asset storage and collaboration spaces per project. Free tier allows 3 projects; Standard and higher tiers allow unlimited projects. Each project includes asset storage (5GB free, 100GB standard, 500GB pro) for organizing source materials, generated videos, and project files. Implementation details unknown, but likely uses cloud storage with project-level access controls.

Allows users to paint directional strokes onto video frames to guide and control the direction and intensity of motion in generated or edited video sequences. Users draw strokes (up, down, left, right, circular, etc.) on specific regions of a video, and the system interprets these as motion vectors that influence how the generative model synthesizes movement in those areas. Implementation details unknown, but likely uses stroke-to-vector conversion and spatial masking to localize motion control.

Enables selective editing of video regions by masking areas and providing text prompts describing desired changes. Users define a region (inpaint mask) in a video frame and supply a text description of what should appear in that region, and the generative model synthesizes new content within the masked area while preserving the surrounding context. Processing is asynchronous and credit-metered, with implementation details (mask propagation across frames, temporal consistency mechanisms) unknown.

Automatically detects and removes video backgrounds, generating videos with transparent or alpha-channel backgrounds suitable for compositing. The system analyzes video frames to identify foreground subjects (people, objects) and separates them from background elements, outputting video with transparency information. Available on Standard tier and above, suggesting use of semantic segmentation or matting models running on Runway's infrastructure.

Removes watermarks, logos, and other unwanted visual elements from video frames using inpainting-like techniques. Available on Standard tier and above, this feature uses generative models to synthesize replacement content that blends seamlessly with surrounding video context. Implementation likely similar to inpainting but with automatic watermark detection rather than manual masking.

Upscales video resolution using learned super-resolution models, enhancing video quality and enabling export at higher resolutions than the source. Available on Standard tier and above, this feature processes video frames to increase resolution while maintaining or improving visual quality. Implementation likely uses diffusion-based or neural upscaling models running asynchronously on Runway's infrastructure.

Generates explorable 3D environments and worlds from text prompts or images using Runway's proprietary GWM-1 (Generative World Model) architecture. Users describe a scene or environment, and the system generates a navigable 3D space that can be explored interactively in real-time. Implementation uses world models trained on large-scale video datasets to learn spatial and temporal dynamics, enabling real-time rendering of novel viewpoints without pre-computed 3D geometry.

Generates conversational video avatars from a single image input using GWM-1 Avatars/Characters variant. Users provide a portrait or character image, and the system creates a real-time video agent capable of responding to text prompts with synchronized speech and facial expressions. Accessed via Runway Characters API, this feature enables zero-shot avatar creation without fine-tuning or training, using learned priors from large-scale video and speech datasets.

Simulates robotic behavior and physical interactions using GWM-1 Robotics variant, enabling prediction of how robots will interact with environments and objects. Users define a robot, environment, and task, and the system generates video predictions of robotic motion and physical outcomes. Implementation uses world models trained on robotic video datasets to learn physics-aware dynamics, enabling zero-shot prediction without explicit physics simulation.