Fine Tuning With Compression For Accuracy Recovery

via “fine-tuning with compression for accuracy recovery”

Toolkit for LLM quantization, pruning, and distillation.

Unique: Implements quantization-aware training by injecting fake quantization operations into the forward pass and enabling gradient flow through quantized weights, allowing models to adapt to quantization constraints during fine-tuning without requiring separate QAT frameworks

vs others: More integrated than separate QAT tools because compression modifiers are active during training; more flexible than fixed QAT schemes because any compression recipe can be used; more practical than retraining from scratch because it starts from a compressed checkpoint