Phantom

Generates videos from text prompts while maintaining consistent subject identity across frames through cross-modal alignment between text embeddings and visual features. The system uses consistency models to enforce temporal coherence and subject preservation, processing text descriptions through a learned alignment mechanism that maps semantic intent to stable visual representations across the entire video sequence.

Distributes video generation inference and training across multiple GPUs using Fully Sharded Data Parallel (FSDP) strategy, enabling larger model variants (14B parameters) to run on 8-GPU clusters by sharding model weights, optimizer states, and gradients across devices. The system automatically manages communication patterns and gradient synchronization to maintain training stability while reducing per-GPU memory requirements.

Provides utilities to measure inference latency, throughput, memory usage, and quality metrics across different model variants (1.3B vs 14B) and hardware configurations, enabling data-driven decisions about model selection. The system profiles generation time, peak memory consumption, and optionally computes quality metrics (LPIPS, FVD) to quantify the accuracy-efficiency tradeoff between variants.

Converts generated video frames to standard output formats (MP4, WebM, etc.) with configurable quality settings including bitrate, codec, and resolution. The system handles frame-to-video encoding, manages output file paths, and supports quality presets (low/medium/high) that trade off file size against visual quality.

Generates video frames using consistency models rather than traditional diffusion, enabling single-step or few-step generation by learning to map noisy inputs directly to clean outputs through a consistency function. This approach trades off some quality for dramatically reduced inference time, using a learned ODE trajectory that collapses the diffusion process into fewer sampling steps while maintaining temporal coherence across frames.

Provides a configuration system that abstracts model selection, hyperparameter tuning, and inference settings through structured config files, enabling users to switch between Phantom-Wan-1.3B and Phantom-Wan-14B variants without code changes. The system loads model architectures, weights, and inference parameters from configuration, supporting different GPU memory profiles and inference strategies through declarative configuration rather than imperative code.

Provides a CLI tool (infer.sh) that wraps the video generation pipeline, accepting text prompts and configuration parameters as command-line arguments and orchestrating the full generation workflow including model loading, inference, and output saving. The CLI abstracts away Python API complexity and enables integration with shell scripts, CI/CD pipelines, and batch processing systems through standard command invocation.

Implements model loading logic that deserializes pre-trained weights from checkpoint files, initializes model architecture based on configuration, and validates weight compatibility with the target architecture. The system handles different checkpoint formats, manages device placement (CPU/GPU), and supports partial weight loading for transfer learning scenarios where only specific layers are updated.

Enforces temporal coherence across video frames by applying consistency constraints between adjacent frames during generation, ensuring smooth transitions and preventing flickering or subject drift. The system uses the cross-modal alignment mechanism to maintain semantic consistency while allowing natural motion and scene changes, applying regularization that penalizes large frame-to-frame differences in subject representation while permitting expected motion.

Implements classifier-free guidance at inference time, allowing users to control the strength of text prompt conditioning through a guidance scale parameter that interpolates between unconditional and conditional generation. The system computes both conditional (text-guided) and unconditional predictions, then blends them according to guidance scale to balance prompt adherence with output diversity and quality.

Processes multiple video generation requests in batches, automatically managing GPU memory allocation and deallocating intermediate tensors to fit multiple samples within available VRAM. The system uses dynamic batching that adjusts batch size based on available memory and prompt length, enabling higher throughput than sequential generation while preventing out-of-memory errors.

Accepts optional reference images that specify the desired appearance of the subject, using image encoders to extract visual features that condition the video generation process alongside text prompts. The system aligns reference image features with text embeddings through the cross-modal alignment mechanism, enabling users to generate videos where the subject matches a provided reference image while following the text description.