metadata
base_model: unsloth/Llama-3.2-1B-Instruct-bnb-4bit
library_name: transformers
pipeline_tag: text-generation
tags:
- gguf
- fine-tuned
- lima
language:
- en
license: apache-2.0
Llama-3.2-1B-Instruct-bnb-4bit-lima - GGUF Format
GGUF format quantizations for llama.cpp/Ollama.
Model Details
- Base Model: unsloth/Llama-3.2-1B-Instruct-bnb-4bit
- Format: gguf
- Dataset: GAIR/lima
- Size: Varies by quantization (2-8GB per file)
- Usage: llama.cpp / Ollama
Related Models
- LoRA Adapters: fs90/Llama-3.2-1B-Instruct-bnb-4bit-lima-lora - Smaller LoRA-only adapters
- Merged FP16 Model: fs90/Llama-3.2-1B-Instruct-bnb-4bit-lima - Original unquantized model in FP16
Training Details
- LoRA Rank: 16
- Training Steps: 129
- Training Loss: 2.3025
- Max Seq Length: 4086
- Training Scope: 1,030 samples (1 epoch(s), full dataset)
For complete training configuration, see the LoRA adapters repository/directory.
Available Quantizations
| Quantization | File | Size | Quality |
|---|---|---|---|
| F16 | model.F16.gguf |
2.31 GB | Full precision (largest) |
| Q4_K_M | model.Q4_K_M.gguf |
0.75 GB | Good balance (recommended) |
| Q6_K | model.Q6_K.gguf |
0.95 GB | High quality |
| Q8_0 | model.Q8_0.gguf |
1.23 GB | Very high quality, near original |
Usage: Use the dropdown menu above to select a quantization, then follow HuggingFace's provided instructions.
License
Based on unsloth/Llama-3.2-1B-Instruct-bnb-4bit and trained on GAIR/lima. Please refer to the original model and dataset licenses.
Credits
Trained by: Farhan Syah
Training pipeline:
- unsloth-finetuning by @farhan-syah
- Unsloth - 2x faster LLM fine-tuning
Base components:
- Base model: unsloth/Llama-3.2-1B-Instruct-bnb-4bit
- Training dataset: GAIR/lima by GAIR