f16 path changed
Browse files
{Qwen3-0.6B-Q3_K_M → Qwen3-0.6B-f16-Q3_K_M}/README.md
RENAMED
|
@@ -17,7 +17,7 @@ base_model: Qwen/Qwen3-0.6B
|
|
| 17 |
author: geoffmunn
|
| 18 |
---
|
| 19 |
|
| 20 |
-
# Qwen3-0.6B:Q3_K_M
|
| 21 |
|
| 22 |
Quantized version of [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) at **Q3_K_M** level, derived from **f16** base weights.
|
| 23 |
|
|
@@ -99,7 +99,7 @@ Stop sequences: `<|im_end|>`, `<|im_start|>`
|
|
| 99 |
Importing directly into Ollama should work, but you might encounter this error: `Error: invalid character '<' looking for beginning of value`.
|
| 100 |
In this case try these steps:
|
| 101 |
|
| 102 |
-
1. `wget https://huggingface.co/geoffmunn/Qwen3-0.6B/resolve/main/Qwen3-0.6B-f16%3AQ3_K_M.gguf`
|
| 103 |
2. `nano Modelfile` and enter these details:
|
| 104 |
```text
|
| 105 |
FROM ./Qwen3-0.6B-f16:Q3_K_M.gguf
|
|
@@ -138,7 +138,7 @@ Here’s how you can query this model via API using `curl` and `jq`. Replace the
|
|
| 138 |
|
| 139 |
```bash
|
| 140 |
curl http://localhost:11434/api/generate -s -N -d '{
|
| 141 |
-
"model": "hf.co/geoffmunn/Qwen3-0.6B:Q3_K_M",
|
| 142 |
"prompt": "Respond exactly as follows: Repeat the word 'hello' five times separated by commas.",
|
| 143 |
"temperature": 0.1,
|
| 144 |
"top_p": 0.95,
|
|
|
|
| 17 |
author: geoffmunn
|
| 18 |
---
|
| 19 |
|
| 20 |
+
# Qwen3-0.6B-f16:Q3_K_M
|
| 21 |
|
| 22 |
Quantized version of [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) at **Q3_K_M** level, derived from **f16** base weights.
|
| 23 |
|
|
|
|
| 99 |
Importing directly into Ollama should work, but you might encounter this error: `Error: invalid character '<' looking for beginning of value`.
|
| 100 |
In this case try these steps:
|
| 101 |
|
| 102 |
+
1. `wget https://huggingface.co/geoffmunn/Qwen3-0.6B-f16/resolve/main/Qwen3-0.6B-f16%3AQ3_K_M.gguf`
|
| 103 |
2. `nano Modelfile` and enter these details:
|
| 104 |
```text
|
| 105 |
FROM ./Qwen3-0.6B-f16:Q3_K_M.gguf
|
|
|
|
| 138 |
|
| 139 |
```bash
|
| 140 |
curl http://localhost:11434/api/generate -s -N -d '{
|
| 141 |
+
"model": "hf.co/geoffmunn/Qwen3-0.6B-f16:Q3_K_M",
|
| 142 |
"prompt": "Respond exactly as follows: Repeat the word 'hello' five times separated by commas.",
|
| 143 |
"temperature": 0.1,
|
| 144 |
"top_p": 0.95,
|