Fooocus

Generates high-quality images from text prompts by running Stable Diffusion XL locally through a multi-stage pipeline: prompt parsing and style application, CLIP text encoding into embeddings, diffusion-based latent sampling, and VAE decoding to visual output. Automatically enhances user prompts using a built-in expansion system (extras/expansion.py) that enriches sparse descriptions with contextually relevant details before encoding, eliminating the need for manual prompt engineering expertise.

Applies pre-configured style templates (anime, realistic, semi-realistic, etc.) stored in sdxl_styles/sdxl_styles_fooocus.json to modify the generation behavior without exposing underlying parameters. The style system works by injecting style-specific positive and negative prompt tokens into the CLIP encoding stage, effectively conditioning the diffusion model toward particular aesthetic outcomes. Users select a style from a dropdown; the system automatically appends style keywords and adjusts sampling parameters defined in preset JSON files (presets/anime.json, presets/realistic.json, etc.).

Enables users to submit multiple image generation requests that are queued and processed sequentially (or in parallel on multi-GPU systems) via the AsyncTask worker system. Users can submit 10+ generation requests with different prompts/parameters, and the system processes them in order while displaying real-time progress (current task, step count, ETA) for each image. The queue persists task metadata including prompt, parameters, and result paths, allowing users to monitor progress and retrieve results after completion.

Implements automatic model discovery, downloading, and caching (via model management modules) that fetches required models (SDXL base, VAE, LoRA, upscaling models) from Hugging Face or other repositories on first use, caches them locally, and loads them into VRAM on-demand. Users don't manually download models; the system detects missing models, downloads them in the background, and caches them for future use. Model paths are configurable via config.txt, allowing users to point to custom model directories or external storage.

Provides a web-based interface built with Gradio (webui.py) that allows users to adjust generation parameters (prompt, resolution, seed, style, etc.) in real-time and see results instantly without page reloads. The UI includes text input fields for prompts, dropdown selectors for styles and presets, sliders for numeric parameters, image upload/preview areas, and progress indicators. Gradio handles the web server, request routing, and WebSocket-based real-time updates, allowing the UI to remain responsive during generation.

Provides multiple sampling algorithms (Euler, DPM++, LCM, etc.) that control how the diffusion model iteratively refines the image from noise to final output. Different samplers have different speed/quality tradeoffs: LCM (Latent Consistency Model) is 4-8x faster but lower quality, while DPM++ is slower but higher quality. Users select a sampler via dropdown or preset; the system applies the corresponding sampling algorithm during the diffusion loop. Advanced techniques like Perpendicular Negative Guidance (PerpNeg) and Self-Attention Guidance (SAG) are available as optional enhancements.

Implements Self-Attention Guidance (ldm_patched/contrib/external_sag.py), a technique that enhances semantic coherence by modifying self-attention maps during diffusion sampling. SAG amplifies attention to semantically important regions, improving object definition and reducing artifacts. This is particularly effective for complex scenes with multiple objects or fine details. SAG is optional and can be toggled per generation.

Implements a queue-based AsyncTask worker system (modules/async_worker.py) that decouples image generation from the web UI, allowing users to interact with the interface while generation runs in background threads. The AsyncTask class encapsulates generation parameters, progress tracking, and result storage; a worker function continuously polls a task queue, processes requests, and streams progress updates back to the Gradio UI via WebSocket-like callbacks. This architecture prevents UI freezing during the 30-120 second generation time typical for SDXL.

Integrates LoRA adapters into the diffusion model pipeline via model_patcher.py, allowing users to load and apply lightweight fine-tuned models that modify the base SDXL weights without full model retraining. LoRA adapters are merged into the UNet and text encoder at inference time using low-rank matrix multiplication, enabling style customization (e.g., specific character designs, artistic techniques) with minimal VRAM overhead (~50-100MB per LoRA vs 7GB for full model). Users select LoRA files from a dropdown; the system automatically patches the model weights before generation.

Enables selective image modification by accepting a base image and binary mask that defines which regions to regenerate. The inpainting pipeline encodes the base image into latent space via VAE, applies the mask to preserve masked regions, and runs diffusion sampling only on unmasked areas while conditioning on the surrounding context. Outpainting extends this to generate new content beyond image boundaries by padding the image and masking the padding region. Users upload an image, draw or upload a mask, provide a prompt, and the system regenerates only the masked regions while maintaining coherence with unmasked content.

Applies post-processing face restoration to generated or uploaded images using specialized restoration models (e.g., GFPGAN, Real-ESRGAN) that enhance facial details, reduce artifacts, and improve overall face quality. The restoration pipeline detects faces in the image, applies the restoration model to each face region, and blends the restored faces back into the original image. This is particularly useful for SDXL outputs which sometimes produce distorted or low-quality faces, especially at lower resolutions or with complex prompts.

Enables image-to-image generation by using IP-Adapter (Image Prompt Adapter) to inject visual features from a reference image into the diffusion model's cross-attention layers, and BLIP (Bootstrapping Language-Image Pre-training) to automatically generate descriptive captions from reference images. The pipeline extracts visual embeddings from a reference image using a CLIP vision encoder, projects them via IP-Adapter into the diffusion model's latent space, and optionally uses BLIP to generate text descriptions that augment the user's prompt. This allows users to generate variations of an image or apply a reference image's style without manual prompt engineering.

Applies post-processing upscaling to generated or uploaded images using Real-ESRGAN or similar super-resolution models that increase image resolution by 2x-4x while preserving or enhancing detail. The upscaling pipeline loads a pre-trained super-resolution model, processes the image through the model to predict high-frequency details, and outputs a higher-resolution image. This is useful for generating high-resolution outputs from lower-resolution generations (which are faster) or for enhancing user-uploaded images.

Implements a flexible configuration system (args_manager.py) that merges settings from multiple sources with a defined priority hierarchy: built-in defaults < config.txt user configuration < preset JSON files < command-line arguments. Users can customize behavior via a config.txt file (e.g., default model paths, VRAM optimization flags), select presets for different use cases (anime.json, realistic.json, lcm.json), or override settings via CLI arguments. This allows both GUI users (who use presets) and advanced users (who edit config.txt or use CLI) to customize behavior without code changes.

Applies architectural optimizations to the CLIP text encoder and diffusion model's attention mechanisms (ldm_patched/ldm/modules/attention.py, ldm_patched/modules/clip_vision.py) to reduce inference latency and VRAM usage. Optimizations include: attention memory optimization (computing attention in chunks rather than all-at-once), flash attention implementations for faster matrix operations, and CLIP token optimization to reduce redundant computations. These patches are applied at model load time via model_patcher.py, modifying the model's forward pass without changing weights.

Fooocus is a simplified open-source image generation interface that allows users to create high-quality images with minimal configuration, inspired by the ease of use of Midjourney and built on Stable Diffusion XL.