Qcompiler

MIXQ. MIXQ: Taming Dynamic Outliers in Mixed-Precision Quantization by Online Prediction

vllm-mixed-precision. Support mixed-precsion inference with vllm

MixQ_Tensorrt_LLM. Mixed precision inference by Tensorrt-LLM

QComplier. quantize mixed precision model

SC24_MIXQ_AE. Official implementation of MIXQ for AE

llm_bench. this repository shows how to define your own benchmark and model