GPTQ, AWQ, GGUF: what quantization actually costs, measured.
Quantization trades numerical precision for memory and speed, but the cost is not uniform across methods. GPTQ and AWQ keep most of the quality of a full-precision model at four bits, while aggressive schemes can quietly degrade reasoning long before they touch perplexity.
The honest way to choose is to measure on your own task, not to trust a single headline number. Run the quantized model against a held-out set, compare it to the original, and accept the smaller weights only when the gap stays inside a budget you set in advance.