Structured And Unstructured Pruning With Layer Wise Sparsity Patterns

via “structured and unstructured pruning with layer-wise sparsity patterns”

Toolkit for LLM quantization, pruning, and distillation.

Unique: Implements layer-wise pruning through a modifier system that applies sparsity masks to specific layer patterns, supporting both structured (channel/head removal) and unstructured (weight removal) pruning with automatic importance estimation from calibration data

vs others: More flexible than magnitude-based pruning because it supports learned importance scores; more practical than gradient-based pruning because it doesn't require training; better integrated with vLLM than generic sparse tensor libraries