QuantTrio

/

DeepSeek-V3.2-AWQ

like 5

Follow

QuantTrio 147

I have 4 NVIDIA RTX PRO 6000 Blackwell GPUs, does it fit? If yes, how much KV cache is left? If yes, what settings did you use? I am running at the moment GLM-4.6-FP8 with SGLang, there I still get 160k context length with FP8 KV cache quantization.