New “statistically lossless” quantization techniques promise to compress LLMs up to 3.2 bits per parameter while preserving output quality, removing a major trade-off for on-premise deployments and accelerating real-world adoption.
A new paper introduces statistically-lossless quantization (SLQ), offering probabilistic guarantees on output quality without sacrificing speed. It defines three losslessness levels: task-lossless under 4 bits, distribution-lossless (output distribut...
A new study strikes a middle ground between lossy and lossless compression, achieving quantization that preserves statistical output quality while accelerating inference and cutting bits per parameter down to 3.3. Direct implications for anyone runni...
