Calibration Driven Per Channel Scaling Factor Computation

via “calibration-driven per-channel scaling factor computation”

4-bit weight quantization for LLMs on consumer GPUs.

Unique: Computes scaling factors by analyzing actual activation patterns from calibration data rather than using weight statistics alone. This activation-aware approach identifies which weight channels are most important based on how often they are activated during inference, enabling selective protection of critical channels.

vs others: More accurate than weight-only quantization methods (GPTQ) because it accounts for activation patterns; more efficient than layer-wise quantization because per-channel factors provide finer-grained control without excessive overhead.