Update README.md

README.md

@@ -14,15 +14,66 @@ language:
 - ar
 license: apache-2.0
 inference: false
-base_model:
-- mistralai/Ministral-3-14B-Reasoning-2512
-extra_gated_description: >-
-  If you want to learn more about how we process your personal data, please read
-  our <a href="https://mistral.ai/terms/">Privacy Policy</a>.
+base_model: mistralai/Ministral-3-14B-Reasoning-2512
+extra_gated_description: If you want to learn more about how we process your personal
+  data, please read our <a href="https://mistral.ai/terms/">Privacy Policy</a>.
 tags:
 - mistral-common
 ---
 
+# Ministral-3-14B-Reasoning-2512 AWQ - INT4
+
+## Model Details
+
+### Quantization Details
+
+- **Quantization Method:** AWQ
+- **Bits:** 4
+- **Group Size:** 32
+- **Calibration Dataset:** [5CD-AI/LLaVA-CoT-o1-Instruct](https://huggingface.co/datasets/5CD-AI/LLaVA-CoT-o1-Instruct)
+- **Quantization Tool:** [llm-compressor](https://github.com/vllm-project/llm-compressor)
+
+### Memory Usage
+
+| **Type** | **Ministral-3-14B-Reasoning-2512** | **Ministral-3-14B-Reasoning-2512-AWQ-4bit** |
+|:---------------:|:----------------:|:----------------:|
+| **Memory Size** | 51.9 GB | 19.4 GB | 
+| **KV Cache per Token** | 200.0 kB | 50.0 kB | 
+| **KV Cache per Context** | 50.0 GB | 12.5 GB | 
+
+### Evaluations
+
+| **Benchmarks** | **Ministral-3-14B-Reasoning-2512** | **Ministral-3-14B-Reasoning-2512-AWQ-4bit** |
+|:---------------:|:----------------:|:----------------:|
+| **Perplexity** | 1.52771 | 1.5367 | 
+
+- **Evaluation Context Length:** 16384
+
+## Inference
+
+### Prerequisite
+
+```bash
+pip install -U vllm
+```
+
+### Basic Usage
+
+```bash
+vllm serve cyankiwi/Ministral-3-14B-Reasoning-2512-AWQ-4bit --tokenizer_mode mistral --config_format mistral --load_format mistral --enable-auto-tool-choice --tool-call-parser mistral
+```
+
+## Additional Information
+
+### Changelog
+
+- **v1.0.0** - Initial quantized release
+
+### Authors
+
+- **Name:** Ton Cao
+- **Contacts:** [email protected]
+
 # Ministral 3 14B Reasoning 2512
 
 The largest model in the Ministral 3 family, **Ministral 3 14B** offers frontier capabilities and performance comparable to its larger [Mistral Small 3.2 24B](https://huggingface.co/mistralai/Mistral-Small-3.2-Instruct-2506) counterpart. A powerful and efficient language model with vision capabilities.