InvokeAI

Executes directed acyclic graphs (DAGs) of custom nodes where each node represents a discrete operation (image generation, conditioning, post-processing). The invocation system uses a BaseInvocation class hierarchy with schema-based node definitions, allowing the FastAPI backend to dynamically route node outputs to inputs, validate data types, and execute the graph sequentially or with parallelization where dependencies allow. WebSocket connections provide real-time progress updates and intermediate results to the frontend.

A Konva-based HTML5 canvas system that manages multiple image layers (base image, mask, inpaint region, generated output) with real-time brush tools for mask creation. The canvas supports infinite zoom/pan, layer blending modes, and undo/redo via Redux state management. Inpainting workflows automatically generate conditioning masks from brush strokes and pass them to the diffusion pipeline; outpainting extends the canvas beyond the original image bounds and generates content in the expanded regions using boundary conditioning.

Supports loading and applying textual embeddings (custom token embeddings) and LoRA (Low-Rank Adaptation) modules that modify model weights. The system detects embedding and LoRA files in the model directory, loads them into the text encoder and UNet respectively, and applies them during generation. LoRA weights can be dynamically adjusted (0-1 scale) to control their influence on generation. The system supports multiple LoRAs simultaneously, merging their weight modifications into the base model.

A job queue system that accepts multiple generation requests, schedules them for execution, and manages GPU resource allocation. The system supports priority-based scheduling (high-priority jobs execute before low-priority ones) and concurrent execution of independent jobs (e.g., two generations with different models). The queue persists to disk, allowing jobs to survive server restarts. Progress is streamed via WebSocket, and completed jobs are automatically moved to the gallery.

A hierarchical configuration system that loads settings from environment variables, configuration files (YAML/JSON), and command-line arguments, with later sources overriding earlier ones. The system manages GPU allocation, model paths, API endpoints, and UI preferences. Configuration is validated at startup using Pydantic models, ensuring type safety and providing clear error messages for invalid settings. Runtime configuration changes (e.g., switching models) are applied without server restart via API endpoints.

A centralized model registry that discovers, downloads, and caches diffusion models (SD1.5, SD2.0, SDXL, FLUX) in multiple formats (safetensors, ckpt, diffusers). The system uses a model configuration layer that abstracts format differences, allowing seamless switching between model variants. Models are loaded into GPU VRAM on-demand and cached in memory to avoid redundant disk I/O; a least-recently-used (LRU) eviction policy manages VRAM pressure. The backend exposes model metadata (resolution, architecture, training data) via REST API for frontend UI population.

A conditioning system that accepts multiple control inputs (ControlNet images, text embeddings, IP-Adapter features) and fuses them into a unified conditioning tensor that guides the diffusion process. The system uses CLIP text encoders to convert prompts to embeddings, applies ControlNet models to extract spatial features from control images, and combines these via cross-attention mechanisms in the UNet. The architecture supports weighted blending of multiple ControlNets and dynamic conditioning strength adjustment during generation.

Orchestrates the full diffusion sampling process: noise scheduling (DDIM, Euler, DPM++, etc.), UNet denoising iterations, and VAE decoding. The pipeline accepts a conditioning tensor and noise schedule parameters (steps, guidance scale, sampler type) and iteratively denoises a random noise tensor through the UNet, applying classifier-free guidance to steer generation toward the conditioning. The system supports deterministic generation via seed control and exposes intermediate latent states for inspection or manipulation.

A hierarchical image storage system where generated images are automatically saved to a gallery with associated metadata (generation parameters, model used, timestamp, user notes). Images can be organized into boards (collections) for project-based grouping. The system uses a relational database (SQLite by default) to index images and boards, enabling fast search and filtering by generation parameters, model, or date. Metadata is persisted alongside images, allowing workflows to be reconstructed from saved generation history.

A FastAPI-based REST API that exposes all InvokeAI functionality (model management, workflow execution, gallery operations) with auto-generated OpenAPI documentation. The API uses dependency injection to manage service lifecycle and database connections. WebSocket endpoints stream real-time generation progress, intermediate results, and error messages to connected clients without polling. The schema is dynamically generated from Python type hints, ensuring API documentation stays synchronized with implementation.

A React TypeScript frontend that manages application state via Redux, with RTK Query handling API communication and caching. The UI is organized into feature modules (canvas, gallery, workflow editor, settings) that dispatch Redux actions to update state and subscribe to state changes for re-rendering. RTK Query automatically caches API responses and handles request deduplication, reducing unnecessary network traffic. The frontend uses a theme system for dark/light mode support and internationalization (i18n) for multi-language support.

A visual workflow editor that renders node graphs using a graph layout library, allowing users to create and edit workflows by dragging nodes, connecting outputs to inputs, and configuring node parameters. The editor discovers available nodes by querying the backend API for the OpenAPI schema, dynamically generating UI controls for each node's parameters based on type hints (text inputs for strings, sliders for floats, dropdowns for enums). The editor validates connections at edit-time, preventing invalid connections (e.g., connecting an image output to a string input) and providing visual feedback for errors.

Abstracts the differences between Stable Diffusion model architectures (SD1.5, SD2.0, SDXL, FLUX) through a unified interface. The system detects model architecture from metadata or configuration files and routes to the appropriate pipeline implementation (e.g., StableDiffusionPipeline for SD1.5, StableDiffusionXLPipeline for SDXL). Each architecture has different tokenizer, text encoder, and UNet configurations; the abstraction layer handles these differences transparently, allowing the same generation code to work across all architectures.