Infinity

Predicts image tokens bit-by-bit rather than from a fixed vocabulary, enabling effective vocabulary scaling from 2^16 to 2^64 through sequential binary predictions. The Infinity Transformer autoregressively generates each bit position across the entire image sequentially, allowing the model to scale token representation without discrete vocabulary limits. This approach replaces traditional discrete token prediction with continuous bitwise decomposition, fundamentally changing how visual information is encoded and generated.

Encodes natural language text prompts using Flan-T5 embeddings and conditions the Infinity Transformer on these embeddings to guide image generation. The text encoder processes prompts into high-dimensional embeddings that are injected into the transformer's cross-attention layers, allowing semantic alignment between text descriptions and generated visual content. This conditioning mechanism enables fine-grained control over image content through natural language descriptions.

Provides utilities for loading and preprocessing image-text datasets in multiple formats (directory-based, JSON metadata, COCO format) and converting them to the format required by Infinity's training pipeline. The data loading pipeline handles image resizing, normalization, text tokenization, and batching with configurable preprocessing options. Support for multiple dataset formats enables training on diverse publicly available datasets.

Implements a self-correction mechanism that refines generated images by iteratively predicting and correcting individual bits based on previous predictions and quality feedback. The mechanism allows the model to revise earlier predictions when inconsistencies are detected, improving overall image coherence and quality. This approach leverages the bitwise prediction structure to enable fine-grained refinement without full image regeneration.

Provides a configuration system for specifying Infinity Transformer architecture parameters (depth, embedding dimension, number of attention heads, feed-forward dimension) and training hyperparameters (learning rate, batch size, warmup steps, weight decay). Configuration can be specified via JSON files, command-line arguments, or Python dicts, enabling reproducible model instantiation and training. The configuration system validates parameters and provides sensible defaults.

Converts images to discrete tokens and reconstructs images from tokens using a visual autoencoder (VAE) that supports configurable vocabulary sizes from 2^16 to 2^64. The VAE encodes images into a latent space with adjustable quantization levels, enabling trade-offs between reconstruction fidelity and token sequence length. Different vocabulary sizes (16-bit, 32-bit, 64-bit) allow users to balance image quality against computational cost and sequence length.

Generates images token-by-token using the Infinity Transformer with configurable sampling strategies (greedy, top-k, top-p) and temperature parameters to control output diversity and quality. The generation process iteratively predicts the next token conditioned on previously generated tokens and text embeddings, allowing fine-grained control over the generation process through hyperparameters. Temperature scaling adjusts the probability distribution over predicted tokens, enabling trade-offs between deterministic high-quality outputs and diverse creative variations.

Generates multiple images in parallel using batch processing with optimized memory allocation and GPU utilization. The inference pipeline supports configurable batch sizes and implements gradient checkpointing and mixed-precision computation to reduce memory footprint while maintaining generation quality. Batch processing enables efficient throughput for applications requiring multiple image generations.

Loads pre-trained Infinity Transformer weights from checkpoint files and manages model initialization for different model sizes (2B, 8B, 20B). The checkpoint system stores model architecture configuration, weights, and optimizer state, enabling reproducible model loading and fine-tuning. Support for multiple model sizes allows users to select appropriate model capacity based on quality requirements and computational constraints.

Computes Fréchet Inception Distance (FID) scores and other quality metrics to evaluate generated image quality against reference datasets. The evaluation pipeline extracts features from generated and reference images using a pre-trained Inception network, computes statistical distances, and generates quality reports. FID scoring enables quantitative comparison of model performance across different configurations and training iterations.

Provides Jupyter notebook interfaces (interactive_infer_8b.ipynb, interactive_infer.ipynb) for interactive image generation with real-time parameter adjustment and visualization. The notebooks enable users to modify prompts, temperature, sampling strategy, and other hyperparameters and immediately observe results without command-line usage. This interface supports iterative refinement and exploration of the model's capabilities.

Provides a command-line tool (run_infinity.py) for image generation with customizable parameters including prompt, model path, batch size, sampling strategy, and output directory. The CLI interface enables scripted image generation, batch processing, and integration with external workflows without notebook dependencies. Command-line arguments allow fine-grained control over all generation parameters.

Implements a complete training pipeline for fine-tuning or training Infinity models from scratch, including distributed data loading, gradient accumulation, mixed-precision training, and checkpoint saving. The training loop coordinates text encoding, image tokenization, and transformer training with configurable learning rates, batch sizes, and optimization strategies. Support for gradient accumulation enables effective training with larger effective batch sizes on memory-constrained hardware.