InvokeAI

Executes directed acyclic graphs (DAGs) of custom invocation nodes through a FastAPI-backed invocation system that serializes node definitions as OpenAPI schemas. The React frontend provides a visual node editor where users connect outputs to inputs, and the backend's BaseInvocation system deserializes and executes the graph sequentially or in parallel where dependencies allow. This enables non-linear, reusable generation pipelines without code.

Konva-based HTML5 canvas rendering system that manages multiple control layers (base image, mask, brush strokes, selection regions) with real-time compositing. The canvas supports inpainting (selective region regeneration) and outpainting (extending image boundaries) through mask-aware conditioning passed to the diffusion pipeline. Brush tools apply masks directly to the canvas layer system, which are then converted to conditioning tensors for the model.

React frontend uses Redux for global state management (generation parameters, selected models, UI state) and RTK Query for automatic API response caching and synchronization. RTK Query handles cache invalidation when mutations occur (e.g., generating an image invalidates the gallery), reducing unnecessary API calls. The Redux store is persisted to localStorage, allowing the UI to restore state across browser sessions.

React frontend uses i18next library to manage translations across 10+ languages, with JSON translation files organized by feature. Language selection is stored in Redux state and localStorage, allowing users to switch languages without page reload. The system supports pluralization, interpolation, and context-specific translations. Missing translations fall back to English with a warning in development mode.

Backend configuration system that reads settings from environment variables, YAML config files, and command-line arguments with a precedence order (CLI > env vars > config file > defaults). Configuration covers model paths, API settings, GPU memory limits, and feature flags. The system validates configuration at startup and provides helpful error messages for invalid settings. Configuration is exposed via REST API endpoint for frontend discovery.

Centralized model registry that discovers, downloads, caches, and converts between diffusion model formats (safetensors, ckpt, diffusers). The system maintains a model index with metadata (architecture, size, quantization level) and implements LRU caching with configurable memory limits to keep frequently-used models in VRAM. Format conversion happens on-disk before loading, and the model loader uses PyTorch's state_dict utilities to handle architecture mismatches.

Pluggable conditioning system that chains multiple ControlNet models (edge detection, pose, depth, semantic segmentation) to guide diffusion generation. Each ControlNet is loaded as a separate model, processes input images through its encoder to produce conditioning tensors, and these tensors are concatenated and passed to the UNet's cross-attention layers. The system supports weighted blending of multiple ControlNets and dynamic ControlNet switching within a workflow.

FastAPI WebSocket server that emits structured events (generation-started, step-completed, generation-finished, error) during image generation, allowing the React frontend to update progress bars, preview intermediate steps, and handle cancellation. Events are serialized as JSON and include metadata (step number, current image tensor, timing info). The backend maintains a queue of pending invocations and broadcasts events to all connected clients.

Persistent image storage system that organizes generated images into user-created boards with tagging, filtering, and full-text search over metadata. Images are stored on disk with associated JSON metadata files containing generation parameters, model info, and custom tags. The React frontend provides a gallery UI with board navigation, and the FastAPI backend exposes REST endpoints for CRUD operations on images and boards with pagination support.

Wraps the Hugging Face diffusers library's StableDiffusionPipeline to handle the full generation process: text tokenization, CLIP embedding, VAE encoding of reference images, UNet denoising iterations with configurable schedulers (DDIM, Euler, DPM++), and VAE decoding back to image space. The system supports both txt2img (text-to-image) and img2img (image-to-image) modes, with scheduler selection affecting generation quality and speed. Conditioning tensors from ControlNets and embeddings are injected into the UNet's cross-attention layers.

Invocation queue system that accepts multiple generation requests, serializes them with priority levels, and executes them sequentially or in parallel depending on available GPU memory. The backend maintains a queue of pending invocations, tracks execution state (queued, in-progress, completed, failed), and exposes queue status via REST API. Resource pooling ensures that only one model is loaded in VRAM at a time, with automatic unloading when switching models.

Support for textual inversion embeddings and LoRA (Low-Rank Adaptation) modules that modify model behavior without full fine-tuning. Embeddings are loaded as token replacements in the CLIP text encoder, allowing prompts like '<my-style>' to reference custom embeddings. LoRAs are loaded as weight modifications to the UNet and text encoder, applied during generation with configurable strength (0.0-1.0). The system discovers embeddings and LoRAs from disk and makes them available in the node system.

FastAPI-based REST API that automatically generates OpenAPI (Swagger) schemas from Python type hints on invocation nodes and service methods. Request/response validation is handled by Pydantic models, ensuring type safety and providing detailed error messages. The API exposes endpoints for image generation, model management, gallery operations, and queue status, with automatic documentation available at /docs. CORS is configured to allow cross-origin requests from the React frontend.