IC-Light

Performs intelligent image inpainting that respects lighting conditions by using a diffusion-based approach with spatial conditioning maps. The system accepts a base image, a mask defining regions to modify, and optional lighting direction hints, then generates photorealistic inpainted content that matches the scene's illumination. This works by encoding spatial information as additional conditioning inputs to a latent diffusion model, allowing the network to understand which areas need modification and how lighting should flow across the scene.

Provides a web-based drawing interface for users to define inpaint regions through freehand painting, polygon selection, or brush-based masking. The interface uses HTML5 Canvas for real-time mask visualization with adjustable brush size and opacity, allowing users to iteratively refine which areas of the image should be modified. The mask is converted to a binary tensor and passed to the inpainting model as a conditioning signal.

Exposes lighting direction as an adjustable 3D vector (or spherical coordinates) through UI sliders or input fields, allowing users to specify the direction from which light should appear to come in the inpainted region. The system converts these parameters into a conditioning tensor that guides the diffusion model's generation process. Users can preview how different lighting angles affect the inpainting result through iterative generation.

Supports processing multiple images sequentially through a queue-based system, where users can upload several images with their corresponding masks and lighting parameters, and the system processes them in order on available GPU resources. The Gradio interface manages the queue, displaying progress for each image and allowing users to cancel or reorder jobs. This is implemented using Gradio's built-in queue system with configurable concurrency limits.

Executes the core inpainting diffusion model (likely a fine-tuned variant of Stable Diffusion or similar) on GPU hardware, performing iterative denoising steps to generate inpainted content. The system loads the model weights into VRAM, accepts conditioning inputs (mask, lighting direction), and runs the forward pass for a configurable number of diffusion steps (typically 20-50). This is implemented using PyTorch with CUDA/ROCm backends for GPU acceleration.

Provides a user-friendly web interface built with Gradio, a Python framework for rapidly prototyping ML applications. The interface includes image upload, mask drawing canvas, lighting parameter sliders, and result display, all without requiring custom HTML/CSS/JavaScript. Gradio automatically handles form submission, file I/O, and result rendering, while the backend Python code defines the processing logic. The app is deployed on HuggingFace Spaces, which provides free GPU resources and automatic scaling.