Bert Based Text Conditioning With Classifier Free Guidance

via “classifier-free guidance with dynamic thresholding for text alignment control”

Implementation of Imagen, Google's Text-to-Image Neural Network, in Pytorch

Unique: Combines classifier-free guidance with dynamic thresholding (percentile-based clipping) rather than fixed-value thresholding, enabling automatic adaptation to different prompt difficulties and model scales without per-prompt manual tuning

vs others: Provides better artifact prevention than fixed-threshold guidance and requires no separate classifier network unlike traditional guidance methods, reducing training complexity while improving robustness across diverse prompts

via “bert-based text conditioning with classifier-free guidance”

Implementation of Video Diffusion Models, Jonathan Ho's new paper extending DDPMs to Video Generation - in Pytorch

Unique: Uses BERT embeddings as conditioning input to the U-Net (injected via cross-attention-like mechanisms in ResNet blocks) combined with classifier-free guidance training strategy, allowing dynamic control of text influence without separate guidance models

vs others: Simpler than training separate text encoders or guidance models; leverages pre-trained BERT knowledge without fine-tuning, though less flexible than custom-trained text encoders for domain-specific applications

via “guidance-scaled conditional generation with classifier-free guidance”

text-to-video model by undefined. 39,484 downloads.

Unique: Implements classifier-free guidance by maintaining both conditional and unconditional noise predictions during the denoising loop, then interpolating between them at each step using a learned guidance scale. This approach avoids training a separate classifier while still enabling strong conditional control.

vs others: More flexible than fixed-strength conditioning (allows user control over adherence), while remaining more efficient than training separate classifiers for guidance.

via “classifier-free guidance with guidance scale control”

text-to-video model by undefined. 21,431 downloads.

Unique: Implements classifier-free guidance by computing both conditioned and unconditioned noise predictions during denoising, then interpolating based on guidance_scale; this approach enables semantic control without training a separate classifier

vs others: More flexible than fixed-guidance approaches; allows runtime control of prompt adherence without retraining, though at the cost of 2x inference latency

via “classifier-free-guidance-for-conditional-generation”

* 🏆 2020: [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT)](https://arxiv.org/abs/2010.11929)

Unique: DDPM enables classifier-free guidance by training on both conditioned and unconditional samples, then interpolating between unconditional and conditioned predictions during sampling. This avoids training a separate classifier (unlike classifier-based guidance) and enables flexible guidance strength control. The approach is simple, effective, and has become standard in modern text-to-image models (DALL-E 2, Stable Diffusion).

vs others: More flexible than classifier-based guidance (no separate classifier training), simpler to implement than adversarial guidance, and enables fine-grained control over condition strength without retraining.

via “classifier-free guidance with dynamic weighting”

IF — AI demo on HuggingFace

Unique: Uses classifier-free guidance (training on both conditioned and unconditional samples) rather than requiring a separate classifier or reward model, enabling efficient guidance without additional model components.

vs others: Simpler to implement and train than classifier-based guidance (no separate classifier needed) while providing more flexible control than fixed-weight conditioning.

via “classifier-free guidance for prompt adherence and quality control”

* ⭐ 05/2022: [GIT: A Generative Image-to-text Transformer for Vision and Language (GIT)](https://arxiv.org/abs/2205.14100)

Unique: Uses classifier-free guidance by training dual conditional/unconditional predictions and interpolating during sampling, eliminating the need for a separate classifier while enabling fine-grained control over prompt adherence through a single guidance scale parameter

vs others: More efficient than classifier-based guidance (no separate model required) while providing comparable or better prompt adherence control, and more flexible than fixed-weight conditioning by allowing runtime adjustment of guidance strength