Hqq Quantization
원문: hqq-quantization
Half-Quadratic Quantization for LLMs without calibration data. Use when quantizing models to 4/3/2-bit precision without needing calibration datasets, for fast quantization workflows, or when deploying with vLLM or HuggingFace Transformers.
무엇을 하나요
HQQ Half Quadratic Quantization Fast, calibration free weight quantization supporting 8/4/3/2/1 bit precision with multiple optimized backends. When to use HQQ Use HQQ when: Quantizing models without calibration data (no dataset needed) Need fast quantization (minutes vs hours for GPTQ/AWQ) Deploying with vLLM or HuggingFace Transformers Fine tuning quantized models with LoRA/PEFT Experimenting with extreme quantization (2 bit, 1 bit) Key advantages: No calibration : Quantize any model instantly without sample data Multiple backends : PyTorch, ATEN, TorchAO, Marlin, BitBlas for optimized inference Flexible precision : 8/4/3/2/1 bit with configurable group sizes Framework integration : Native…
실행 시 본인 API 키(BYOK)로 동작하며, 모델 비용은 사용자 계정에서 직접 결제됩니다.