deep-daze

Generates images by optimizing SIREN neural network parameters through backpropagation against CLIP embeddings. The system encodes input text into a target embedding via CLIP, then iteratively refines a SIREN-generated image by minimizing the cosine distance between the image's CLIP embedding and the text embedding. This embedding-space optimization approach enables steering image generation toward semantic alignment with natural language descriptions without requiring paired training data.

Initializes SIREN network parameters from an existing image rather than random noise, allowing users to guide or refine images based on visual starting points. The system encodes the priming image through CLIP, then optimizes the SIREN network to match both the priming image's visual characteristics and the target text embedding. This enables iterative refinement workflows where users can start from reference images and steer generation toward specific text descriptions.

Periodically saves intermediate generated images during the optimization loop at configurable intervals, enabling users to monitor generation progress and select preferred outputs from different optimization stages. The system saves images to disk with timestamped filenames, allowing users to observe how the generated image evolves across iterations. Optional progress visualization can display loss curves or intermediate images in real-time (depending on configuration).

Provides configuration options to reduce GPU memory consumption by adjusting batch size for CLIP encoding, image resolution, and SIREN network dimensions. Users can scale down resolution (e.g., from 512x512 to 256x256) or reduce network width to fit within available VRAM constraints. The system automatically handles memory allocation and deallocation, with optional gradient checkpointing to further reduce peak memory usage during backpropagation.

Generates image sequences from longer narratives by applying a sliding window over the input text, optimizing SIREN networks for consecutive text segments. The system divides longer prompts into overlapping windows, generates an image for each window, and optionally chains generations by using previous images as priming for subsequent windows. This enables visual storytelling where each frame corresponds to a narrative segment while maintaining visual continuity across frames.

Applies random cropping and cutout augmentation to generated images during the optimization loop to improve CLIP alignment and prevent mode collapse. The system randomly samples crops from the generated image and encodes them through CLIP, using the crop embeddings in the loss calculation alongside full-image embeddings. This augmentation strategy encourages the SIREN network to generate semantically coherent details across the entire image rather than concentrating features in specific regions.

Simultaneously optimizes SIREN network parameters to align with both text and image embeddings, enabling hybrid guidance where users provide both a text prompt and a reference image. The system computes separate CLIP embeddings for the text and image, then combines their loss signals (via weighted averaging or other fusion strategies) to guide optimization. This allows fine-grained control over the balance between textual and visual guidance in a single optimization pass.

Exposes Deep Daze functionality through a CLI tool named 'imagine' that accepts text prompts and configuration parameters, enabling non-programmatic access to image generation. The CLI parses arguments for prompt text, iteration count, image dimensions, learning rate, SIREN network depth, and output paths, then invokes the underlying Imagine class with the specified configuration. This abstraction allows users to generate images without writing Python code while maintaining full control over optimization hyperparameters.

Exposes image generation functionality through the Imagine class, a Python API that accepts configuration parameters in the constructor and provides methods for generating images from text or images. The class encapsulates CLIP model loading, SIREN network initialization, optimization loop execution, and checkpoint saving, allowing developers to integrate Deep Daze into Python applications with fine-grained control over all aspects of generation. The API supports method chaining and context managers for resource cleanup.

Implements a sinusoidal-activated neural network (SIREN) that maps 2D coordinate inputs to RGB pixel values, enabling continuous image representation without convolutional or attention layers. The SIREN network uses sine activation functions and positional encoding of input coordinates, allowing it to learn high-frequency image details efficiently. During optimization, the network's weights are iteratively updated via backpropagation to minimize CLIP embedding distance, effectively 'fitting' the network to represent images that match the text prompt.

Computes differentiable loss signals by encoding generated images and text prompts through OpenAI's CLIP model, then calculating cosine distance between embeddings in the shared multi-modal space. The loss is backpropagated through the CLIP encoder and into the SIREN network weights, enabling gradient-based optimization that 'steers' image generation toward semantic alignment with text. This embedding-space optimization approach eliminates the need for pixel-space losses or pre-trained discriminators.

Allows users to customize SIREN network architecture by adjusting network depth (number of layers), width (hidden dimension size), and activation function parameters. These hyperparameters directly influence image generation quality, memory consumption, and optimization speed. Deeper networks can represent more complex images but require more computation and memory, while wider networks increase parameter count and memory usage. The configuration is exposed through both CLI flags and Python API constructor arguments.