|
|
--- |
|
|
base_model: unsloth/Llama-3.2-1B-Instruct-bnb-4bit |
|
|
library_name: transformers |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- gguf |
|
|
- fine-tuned |
|
|
- lima |
|
|
language: |
|
|
- en |
|
|
license: apache-2.0 |
|
|
--- |
|
|
|
|
|
# Llama-3.2-1B-Instruct-bnb-4bit-lima - GGUF Format |
|
|
|
|
|
GGUF format quantizations for llama.cpp/Ollama. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Base Model**: [unsloth/Llama-3.2-1B-Instruct-bnb-4bit](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-bnb-4bit) |
|
|
- **Format**: gguf |
|
|
- **Dataset**: [GAIR/lima](https://huggingface.co/datasets/GAIR/lima) |
|
|
- **Size**: Varies by quantization (2-8GB per file) |
|
|
- **Usage**: llama.cpp / Ollama |
|
|
|
|
|
## Related Models |
|
|
|
|
|
- **LoRA Adapters**: [fs90/Llama-3.2-1B-Instruct-bnb-4bit-lima-lora](https://huggingface.co/fs90/Llama-3.2-1B-Instruct-bnb-4bit-lima-lora) - Smaller LoRA-only adapters |
|
|
- **Merged FP16 Model**: [fs90/Llama-3.2-1B-Instruct-bnb-4bit-lima](https://huggingface.co/fs90/Llama-3.2-1B-Instruct-bnb-4bit-lima) - Original unquantized model in FP16 |
|
|
|
|
|
|
|
|
## Training Details |
|
|
|
|
|
- **LoRA Rank**: 16 |
|
|
- **Training Steps**: 129 |
|
|
- **Training Loss**: 2.3025 |
|
|
- **Max Seq Length**: 4086 |
|
|
- **Training Scope**: 1,030 samples (1 epoch(s), full dataset) |
|
|
|
|
|
For complete training configuration, see the LoRA adapters repository/directory. |
|
|
|
|
|
## Available Quantizations |
|
|
|
|
|
| Quantization | File | Size | Quality | |
|
|
|--------------|------|------|---------| |
|
|
| **F16** | `model.F16.gguf` | 2.31 GB | Full precision (largest) | |
|
|
| **Q4_K_M** | `model.Q4_K_M.gguf` | 0.75 GB | Good balance (recommended) | |
|
|
| **Q6_K** | `model.Q6_K.gguf` | 0.95 GB | High quality | |
|
|
| **Q8_0** | `model.Q8_0.gguf` | 1.23 GB | Very high quality, near original | |
|
|
|
|
|
**Usage:** Use the dropdown menu above to select a quantization, then follow HuggingFace's provided instructions. |
|
|
|
|
|
## License |
|
|
|
|
|
Based on unsloth/Llama-3.2-1B-Instruct-bnb-4bit and trained on GAIR/lima. |
|
|
Please refer to the original model and dataset licenses. |
|
|
|
|
|
## Credits |
|
|
|
|
|
**Trained by:** Farhan Syah |
|
|
|
|
|
**Training pipeline:** |
|
|
- [unsloth-finetuning](https://github.com/farhan-syah/unsloth-finetuning) by [@farhan-syah](https://github.com/farhan-syah) |
|
|
- [Unsloth](https://github.com/unslothai/unsloth) - 2x faster LLM fine-tuning |
|
|
|
|
|
**Base components:** |
|
|
- Base model: [unsloth/Llama-3.2-1B-Instruct-bnb-4bit](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-bnb-4bit) |
|
|
- Training dataset: [GAIR/lima](https://huggingface.co/datasets/GAIR/lima) by GAIR |
|
|
|
