Helios

Generates minute-scale videos (up to 60+ seconds) from natural language text prompts using a 14B-parameter diffusion model with autoregressive, chunk-based frame generation. The model processes video in 33-frame chunks sequentially, with each chunk conditioned on previous chunks to maintain temporal coherence without explicit anti-drifting mechanisms like self-forcing or error-banks. Achieves 19.5 FPS on a single H100 GPU by leveraging unified history injection and multi-term memory patchification during training.

Generates videos conditioned on a static input image, using the image as a visual anchor to guide the diffusion process. The model encodes the input image through the same VAE and transformer backbone used for text conditioning, allowing the image to provide spatial and semantic constraints that shape frame generation across all 33-frame chunks. Supports both Helios-Base (highest quality) and Helios-Distilled (fastest) variants with identical architectural conditioning.

Training mechanism that injects previous chunk history (encoded representations of prior 33-frame chunks) directly into the transformer attention layers, enabling the model to maintain temporal coherence across chunk boundaries without explicit anti-drifting strategies like self-forcing, error-banks, or keyframe sampling. The history is injected as additional context tokens in the attention mechanism, allowing the model to learn implicit drift prevention during training. This approach simplifies inference (no need for complex anti-drifting logic) while maintaining quality across minute-scale videos.

Training-time technique that applies lightweight anti-drifting constraints during the Base model training stage, preventing motion drift without the computational overhead of inference-time anti-drifting mechanisms. The strategy uses multi-term memory patchification to reference multiple previous chunks, enabling the model to learn motion consistency across longer temporal windows. This is distinct from unified history injection — easy anti-drifting focuses on motion stability through explicit training objectives, while history injection provides implicit temporal context.

Two custom noise schedulers optimized for different prediction types and guidance strategies: HeliosScheduler for Base/Mid variants (v-prediction with standard/CFG-Zero guidance) and HeliosDMDScheduler for Distilled variant (x0-prediction with CFG-free guidance). Each scheduler is jointly optimized with its corresponding prediction type and guidance strategy during training, enabling faster convergence and better quality at fewer inference steps. The schedulers define the noise level progression across diffusion steps, with HeliosDMDScheduler using more aggressive noise reduction for x0-prediction.

Generates new video frames conditioned on an input video sequence, enabling style transfer, motion continuation, or video interpolation. The model encodes the input video through temporal convolutions and attention layers, extracting motion and semantic patterns that guide the diffusion process for subsequent frames. Supports frame-by-frame or chunk-by-chunk conditioning depending on the inference interface used.

Provides three model checkpoints (Helios-Base, Helios-Mid, Helios-Distilled) arranged in a distillation chain that progressively trades quality for inference speed. Base uses v-prediction with standard CFG and 50 inference steps for highest quality; Mid uses CFG-Zero with 20 steps per stage; Distilled uses x0-prediction with CFG-free guidance (scale=1.0) and 2-3 steps per stage. Each variant uses a different noise scheduler (HeliosScheduler for Base/Mid, HeliosDMDScheduler for Distilled) optimized for its prediction type and guidance strategy.

Helios-Mid and Helios-Distilled variants employ a multi-scale sampling pipeline that decomposes the diffusion process into multiple stages, each operating at different noise scales. The Pyramid Unified Predictor (PUP) architecture enables efficient coarse-to-fine generation where early stages produce low-frequency motion and semantic structure, and later stages refine high-frequency details. This approach reduces effective inference steps (20 per stage for Mid, 2-3 per stage for Distilled) while maintaining temporal coherence across chunk boundaries.

Helios training pipeline uses unified history injection, easy anti-drifting, and multi-term memory patchification to enable image-diffusion-scale batch sizes (typically 256-512 frames per batch) while fitting up to four 14B models in 80GB of GPU memory. The patchification strategy decomposes video frames into spatial patches during training, reducing memory footprint while maintaining temporal coherence through multi-term memory mechanisms that reference previous chunks. This approach eliminates the need for expensive techniques like KV-cache or quantization.

Provides an integrated evaluation framework that measures video quality across five dimensions: aesthetic score (visual appeal), motion amplitude (motion magnitude), motion smoothness (temporal consistency), semantic consistency (text-to-video alignment), and naturalness (perceptual realism). Metrics are computed both as instantaneous scores (per-frame or per-chunk) and as drifting metrics that track degradation over time, enabling detection of long-video artifacts. Scores are aggregated into a final rating that combines all dimensions with configurable weights.

Exposes video generation through four distinct inference interfaces: (1) shell scripts (helios-{variant}_{task}.sh) for quick command-line usage, (2) Python API for programmatic integration, (3) interactive Gradio web UI for manual exploration, and (4) batch processing interface for large-scale generation. All interfaces support the same three tasks (T2V, I2V, V2V) and three variants (Base, Mid, Distilled) through unified parameter passing, enabling seamless switching between interfaces without code changes.

Helios-Mid variant uses CFG-Zero (classifier-free guidance with zero guidance scale) instead of standard CFG, reducing the number of forward passes required per diffusion step from 2 (conditional + unconditional) to 1. This is achieved through training-time modifications that condition the model to produce high-quality outputs without explicit guidance scaling, effectively eliminating the guidance overhead while maintaining quality comparable to standard CFG. The technique is enabled by the v-prediction type and HeliosScheduler, which are jointly optimized during training.

Helios-Distilled variant uses x0-prediction (direct prediction of clean image) with CFG-free guidance (scale=1.0) and HeliosDMDScheduler, enabling the fastest inference path with only 2-3 diffusion steps per stage. Unlike standard CFG which requires dual forward passes, CFG-free guidance operates on a single forward pass with guidance scale fixed at 1.0, eliminating both the guidance computation overhead and the need for unconditional predictions. x0-prediction directly predicts the final clean frame rather than the noise residual, enabling faster convergence with fewer steps.