Not-For-All-Audiences

Static quantization of Impish Nemo 12B

File	Notes
Impish_Nemo_12B.Q6_K_XL.gguf	Q6_K with select tensors quantized to Q8_0 7.23 bpw ~10% increase in size relative to Q6_K Quantized from BF16 Very close in fidelity to full precision
Impish_Nemo_12B.BF16.gguf	Native precision GGUF BF16

GGUF

Model size

12B params

Architecture

llama

Hardware compatibility

6-bit

16-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Valeciela/Impish_Nemo_12B-Q6_K_XL-GGUF

Base model

Finetuned

Finetuned

Quantized

(40)

this model