---
base_model: unsloth/Llama-3.2-1B-Instruct-bnb-4bit
library_name: transformers
pipeline_tag: text-generation
tags:
- gguf
- fine-tuned
- lima
language:
- en
license: apache-2.0
---

# Llama-3.2-1B-Instruct-bnb-4bit-lima - GGUF Format

GGUF format quantizations for llama.cpp/Ollama.

## Model Details

- **Base Model**: [unsloth/Llama-3.2-1B-Instruct-bnb-4bit](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-bnb-4bit)
- **Format**: gguf
- **Dataset**: [GAIR/lima](https://huggingface.co/datasets/GAIR/lima)
- **Size**: Varies by quantization (2-8GB per file)
- **Usage**: llama.cpp / Ollama

## Related Models

- **LoRA Adapters**: [fs90/Llama-3.2-1B-Instruct-bnb-4bit-lima-lora](https://huggingface.co/fs90/Llama-3.2-1B-Instruct-bnb-4bit-lima-lora) - Smaller LoRA-only adapters
- **Merged FP16 Model**: [fs90/Llama-3.2-1B-Instruct-bnb-4bit-lima](https://huggingface.co/fs90/Llama-3.2-1B-Instruct-bnb-4bit-lima) - Original unquantized model in FP16


## Training Details

- **LoRA Rank**: 16
- **Training Steps**: 129
- **Training Loss**: 2.3025
- **Max Seq Length**: 4086
- **Training Scope**: 1,030 samples (1 epoch(s), full dataset)

For complete training configuration, see the LoRA adapters repository/directory.

## Available Quantizations

| Quantization | File | Size | Quality |
|--------------|------|------|---------|
| **F16** | `model.F16.gguf` | 2.31 GB | Full precision (largest) |
| **Q4_K_M** | `model.Q4_K_M.gguf` | 0.75 GB | Good balance (recommended) |
| **Q6_K** | `model.Q6_K.gguf` | 0.95 GB | High quality |
| **Q8_0** | `model.Q8_0.gguf` | 1.23 GB | Very high quality, near original |

**Usage:** Use the dropdown menu above to select a quantization, then follow HuggingFace's provided instructions.

## License

Based on unsloth/Llama-3.2-1B-Instruct-bnb-4bit and trained on GAIR/lima.
Please refer to the original model and dataset licenses.

## Credits

**Trained by:** Farhan Syah

**Training pipeline:**
- [unsloth-finetuning](https://github.com/farhan-syah/unsloth-finetuning) by [@farhan-syah](https://github.com/farhan-syah)
- [Unsloth](https://github.com/unslothai/unsloth) - 2x faster LLM fine-tuning

**Base components:**
- Base model: [unsloth/Llama-3.2-1B-Instruct-bnb-4bit](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-bnb-4bit)
- Training dataset: [GAIR/lima](https://huggingface.co/datasets/GAIR/lima) by GAIR