Flux

Generates photorealistic images from natural language text prompts using 12-billion parameter rectified flow transformer models. The system implements a denoising pipeline that iteratively refines latent representations through the transformer backbone, with model variants (schnell, dev, krea) optimized for different speed/quality tradeoffs. Text prompts are encoded via CLIP or T5 text encoders, then fused with noise through cross-attention mechanisms in the transformer layers.

Guides image generation using structural constraints (Canny edge maps or depth maps) to control composition, pose, and spatial layout. The system implements specialized prepare_canny() and prepare_depth() functions that encode edge/depth information as additional conditioning inputs to the transformer, enabling precise control over object placement and scene structure. Both full model and LoRA-based variants are supported for parameter-efficient conditioning.

Exposes FLUX capabilities through a Python API enabling programmatic image generation with fine-grained control over conditioning, sampling parameters, and model selection. The API provides high-level functions (generate_image, inpaint, edit, etc.) that abstract model loading and sampling pipeline complexity, while exposing low-level sampling parameters (steps, guidance scale, seed, sampler type). Supports both synchronous and asynchronous inference for integration into async applications. Implements context managers for GPU memory management.

Implements usage tracking and API integration for commercial licensing compliance, recording generation counts and model variant usage for billing/licensing purposes. The system integrates with Black Forest Labs' licensing infrastructure through optional API calls that report usage metrics without blocking inference. Supports both open-source (unrestricted) and commercial license modes with different usage restrictions. Implements graceful degradation if licensing API is unavailable.

Provides three model variants (schnell, dev, krea) optimized for different speed/quality tradeoffs, enabling users to select appropriate models based on latency and quality requirements. Schnell is optimized for speed (~1-2 seconds per image with 4 steps), dev balances speed and quality (~5-10 seconds with 20 steps), and krea prioritizes quality (~15-20 seconds with 50 steps). The system abstracts variant differences through unified API, allowing easy switching without code changes. Each variant uses identical architecture but different training objectives and step counts.

Fills or extends image regions using mask-guided generation, where masked areas are regenerated based on surrounding context and text prompts. The system uses the Fill model variant with a specialized prepare_inpaint() function that encodes the mask and original image latents, allowing the transformer to intelligently inpaint missing regions or extend beyond image boundaries. The VAE autoencoder compresses images to latent space where inpainting occurs, then decodes back to pixel space.

Performs semantic image editing using the Kontext model variant, which accepts both an image and text instructions to modify specific regions or attributes. The system implements prepare_edit() to encode the original image and edit prompt, allowing the transformer to apply targeted modifications while preserving unedited regions. This enables style transfer, attribute modification, and localized editing without explicit masks.

Generates variations of images using the Redux model variant, which encodes a reference image as a style/content embedding and uses it to guide generation of new images with similar aesthetic or composition. The system implements prepare_redux() to extract and encode the reference image through a specialized encoder, then uses this embedding as cross-attention conditioning in the transformer. This enables exploration of design alternatives while maintaining visual consistency.

Executes models on either standard PyTorch or optimized TensorRT backends without code changes, enabling flexible hardware utilization and performance tuning. The system abstracts backend selection through a unified model loading interface in src/flux/model.py that instantiates either PyTorch or TensorRT implementations based on configuration. TensorRT compilation includes graph optimization, kernel fusion, and mixed-precision quantization (FP16/INT8) to reduce latency and memory usage by 30-50% compared to standard PyTorch inference.

Implements memory-efficient inference through lazy loading of model components and CPU offloading, allowing models to run on GPUs with <12GB VRAM by moving unused components to CPU RAM. The system loads only required model layers into GPU memory during inference, swapping components to/from CPU as needed. This enables inference on consumer GPUs (RTX 3060, RTX 4060) that would otherwise require A100/H100 hardware, with ~2-3x latency penalty compared to full GPU inference.

Provides a CLI tool for generating images from text prompts with support for batch processing, model variant selection, and parameter tuning. The CLI implements argument parsing for prompt, model selection (schnell/dev/krea), conditioning type, output path, and sampling parameters (steps, guidance scale, seed). Supports both single-image generation and batch processing from prompt files, with progress reporting and error handling. Integrates with HuggingFace model hub for automatic weight downloading.

Provides a browser-based UI for interactive image generation with real-time parameter adjustment, image preview, and prompt refinement. The Gradio interface exposes text input, model selection dropdown, sampling parameter sliders (steps, guidance scale, seed), and conditioning type selector. Implements live preview of generated images with generation time reporting. Automatically handles model loading, GPU memory management, and error reporting through Gradio's reactive component system.

Provides Streamlit-based web interfaces for image generation with dashboard-style layouts, batch processing workflows, and result galleries. Implements multiple Streamlit apps for different use cases: simple generation, batch processing from CSV, and advanced conditioning workflows. Streamlit handles session state management, file uploads, and result caching automatically. Integrates with FLUX inference engine through Python API, enabling custom logic and post-processing.