Automated Neural Network Compression

via “lightweight transformer-based post-processing compression enhancement”

* ⭐ 12/2022: [Robust Speech Recognition via Large-Scale Weak Supervision (Whisper)](https://arxiv.org/abs/2212.04356)

Unique: Applies Transformer models specifically to the quantized latent space rather than raw audio, enabling learned redundancy removal in the compressed domain. Achieves 40% additional compression while maintaining faster-than-real-time operation — a rare combination in neural codecs where compression and speed typically trade off.

vs others: Achieves better compression-to-speed ratio than applying Transformers to raw audio or using traditional entropy coding, because it operates on already-quantized representations where Transformers can learn domain-specific redundancy patterns without the computational burden of processing high-dimensional audio.

via “model compression and quantization instruction”


Unique: MIT's curriculum integrates hardware-aware compression strategies with theoretical foundations, covering the full pipeline from model architecture design through deployment optimization, rather than treating compression as a post-hoc step

vs others: Provides academic rigor and systematic frameworks for compression that go deeper than vendor-specific optimization tools, enabling practitioners to understand trade-offs and design custom compression pipelines

via “automated-neural-network-compression”

via “automatic model quantization and compression”

via “automated neural architecture search and optimization”

via “neural-network-model-optimization”