johnsmith968530 commited on
Commit
4c3abe9
·
verified ·
1 Parent(s): 6a1e2c4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -20
README.md CHANGED
@@ -15,24 +15,20 @@ This model [johnsmith968530/allenai-Olmo-3-7B-Think-MLX-8bit](https://huggingfac
15
  converted to MLX format from [allenai/Olmo-3-7B-Think](https://huggingface.co/allenai/Olmo-3-7B-Think)
16
  using mlx-lm version **0.28.3**.
17
 
18
- ## Use with mlx
19
-
20
  ```bash
21
- pip install mlx-lm
22
- ```
23
-
24
- ```python
25
- from mlx_lm import load, generate
26
-
27
- model, tokenizer = load("johnsmith968530/allenai-Olmo-3-7B-Think-MLX-8bit")
28
-
29
- prompt = "hello"
30
-
31
- if tokenizer.chat_template is not None:
32
- messages = [{"role": "user", "content": prompt}]
33
- prompt = tokenizer.apply_chat_template(
34
- messages, add_generation_prompt=True
35
- )
36
-
37
- response = generate(model, tokenizer, prompt=prompt, verbose=True)
38
- ```
 
15
  converted to MLX format from [allenai/Olmo-3-7B-Think](https://huggingface.co/allenai/Olmo-3-7B-Think)
16
  using mlx-lm version **0.28.3**.
17
 
 
 
18
  ```bash
19
+ export MODEL1_MAJOR="allenai"
20
+ export MODEL1_MINOR="Olmo-3-7B-Think"
21
+ export MODEL1_Q_BITS="8"
22
+ export MODEL1_SD="$(stardate)"
23
+
24
+ echodo () { echo "$@" && "$@"; }
25
+ mkdir -p /tmp/mlx_lm/convert
26
+
27
+ echodo time mlx_lm.convert \
28
+ --hf-path "$MODEL1_MAJOR/$MODEL1_MINOR" \
29
+ --mlx-path "/tmp/mlx_lm/convert/$MODEL1_SD" \
30
+ --quantize \
31
+ --q-bits "$MODEL1_Q_BITS" \
32
+ --upload-repo \
33
+ "johnsmith968530/$MODEL1_MAJOR-$MODEL1_MINOR-MLX-${MODEL1_Q_BITS}bit"
34
+ ```