---
base_model:
- huihui-ai/DeepSeek-R1-Distill-Llama-70B-abliterated
---
Needed to run a 4-bit quantization on vLLM but only GGUFs were available.
Loading time went from ~9 minutes to 2.5 minutes. Throughput went from 25 tokens/second to 45 tokens/second. | 
