README.md

---

license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-7B-Instruct
tags:
- gguf
- quantized
- q4_k_m
---


# Qwen2.5-Coder-7B-Instruct-iat-05-GGUF

This is a GGUF quantized version (q4_k_m) of Qwen/Qwen2.5-Coder-7B-Instruct fine-tuned with the 'iat-05' adapter.

## Model Details

- **Base Model:** Qwen/Qwen2.5-Coder-7B-Instruct
- **Adapter:** iat-05
- **Quantization:** q4_k_m
- **Format:** GGUF

## Usage

This model can be used with llama.cpp or any compatible inference engine that supports GGUF format.

```bash

# Example with llama.cpp

./llama-cli -m Qwen2.5-Coder-7B-Instruct-iat-05-q4_k_m.gguf -p "Your prompt here"

```

## Files

- `Qwen2.5-Coder-7B-Instruct-iat-05-q4_k_m.gguf` - Quantized model in GGUF format (q4_k_m)