Stability AI API

Generates images from natural language text prompts using latent diffusion architecture. Accepts text descriptions and produces high-resolution images (up to 1024x1024 for SDXL, 1408x1408 for SD3) by iteratively denoising random latent vectors conditioned on text embeddings via cross-attention mechanisms. Supports multiple model variants (SD3, SDXL, SD1.6) with different quality/speed tradeoffs and specialized models for specific domains.

Modifies specific regions of an existing image by accepting a base image, binary mask defining the edit region, and a text prompt describing desired changes. Uses masked latent diffusion where the diffusion process is conditioned on both the text prompt and the unmasked image regions, allowing seamless blending of generated content with the original image. Supports various mask formats (PNG with alpha channel, binary masks) and inpainting-specific models optimized for coherent boundary blending.

Allows users to select from multiple Stable Diffusion model variants (SD3, SDXL, SD1.6) with different architectural characteristics and quality/speed tradeoffs. Each model version is independently versioned and maintained, allowing users to specify exact model versions for reproducibility. Implements model selection as a parameter in API requests, with automatic routing to appropriate inference infrastructure. Provides model metadata including capabilities, recommended use cases, and performance characteristics.

Tracks API usage per request and associates costs with credit consumption based on model, resolution, and operation type. Implements a credit system where different operations consume different amounts of credits (e.g., text-to-image at 1024x1024 consumes more credits than 512x512). Provides usage dashboards and billing history through the Stability AI platform web interface. Integrates with payment systems for credit purchase and subscription management.

Secures API access via API key authentication (passed in Authorization header as Bearer token). Rate limiting is enforced per API key based on subscription tier, with limits on requests per minute and concurrent requests. Quota tracking is provided via response headers (X-RateLimit-Remaining, X-RateLimit-Reset). Exceeding limits returns HTTP 429 (Too Many Requests).

Increases image resolution (up to 4x) using specialized upscaling models that reconstruct high-frequency details while preserving semantic content. Uses diffusion-based super-resolution where a low-resolution image is progressively refined through denoising steps conditioned on the original image, producing sharper details than traditional interpolation. Supports multiple upscaling factors (2x, 3x, 4x) and can be chained with other generation operations.

Conditions image generation on structural or stylistic guidance using control networks (ControlNets) that inject spatial constraints into the diffusion process. Accepts a control image (edge map, depth map, pose skeleton, etc.) and a text prompt, then generates images that follow the structural layout of the control image while matching the text description. Implements this by adding a separate conditioning branch that guides the cross-attention mechanism without modifying the base diffusion model.

Applies predefined artistic styles and aesthetic presets to generated images by embedding style descriptors into the text conditioning pipeline. Provides a curated set of style identifiers (e.g., 'photographic', 'cinematic', 'anime', 'oil painting') that modify the diffusion process to favor specific visual characteristics. Implemented as learned embeddings in the text encoder that bias the cross-attention mechanism toward style-specific features without requiring explicit style description in the prompt.

Excludes unwanted visual elements from generated images by specifying negative prompts that are subtracted from the conditioning signal during diffusion. Implements this by computing embeddings for both positive (desired) and negative (undesired) prompts, then using classifier-free guidance to amplify the positive direction while suppressing the negative direction in the latent space. Allows fine-grained control over what should NOT appear in the output.

Generates multiple images from a single prompt with reproducible results using seed-based random number generation. Each seed produces a deterministic sequence of noise vectors that, when passed through the diffusion process, generates consistent images. Allows users to generate variations by incrementing seeds or to reproduce exact outputs by reusing seeds. Implemented as a parameter passed to the diffusion sampling loop that initializes the random state.

Generates short video clips (up to 25 frames) from text prompts or image inputs using Stable Video Diffusion, a latent diffusion model adapted for temporal consistency. Accepts either a text prompt and optional keyframe image, or an image with motion parameters, then generates video by iteratively denoising latent video representations while maintaining temporal coherence through recurrent processing. Produces MP4 videos with configurable frame rates and durations.

Generates audio content including music, sound effects, and speech synthesis using specialized audio diffusion models. Accepts text descriptions of desired audio or speech content, then generates audio waveforms by denoising in the spectrogram or latent audio space. Supports various audio types (music, ambient sounds, speech) with configurable parameters like duration, style, and voice characteristics. Outputs audio in standard formats (MP3, WAV).

Provides HTTP REST endpoints for all image, video, and audio generation capabilities with standardized JSON request/response formats. Implements request validation, error handling, and response serialization following REST conventions. Supports both synchronous responses (for fast operations) and asynchronous job submission with polling (for longer-running operations like video generation). Includes rate limiting, authentication via API keys, and usage tracking for billing.