Hugging Face's logo

tensorblock
/
Phi-3-medium-4k-instruct-GGUF

Text Generation
GGUF
multilingual
nlp
code
TensorBlock
GGUF
conversational
Model card Files
xet
Community
morriszms commited on
Commit
e5f1ca9
·
verified ·
1 Parent(s): f81b7a2

Upload folder using huggingface_hub

Browse files
Files changed (13) hide show
  1. Phi-3-medium-4k-instruct-Q2_K.gguf +2 -2
  2. Phi-3-medium-4k-instruct-Q3_K_L.gguf +2 -2
  3. Phi-3-medium-4k-instruct-Q3_K_M.gguf +2 -2
  4. Phi-3-medium-4k-instruct-Q3_K_S.gguf +2 -2
  5. Phi-3-medium-4k-instruct-Q4_0.gguf +2 -2
  6. Phi-3-medium-4k-instruct-Q4_K_M.gguf +2 -2
  7. Phi-3-medium-4k-instruct-Q4_K_S.gguf +2 -2
  8. Phi-3-medium-4k-instruct-Q5_0.gguf +2 -2
  9. Phi-3-medium-4k-instruct-Q5_K_M.gguf +2 -2
  10. Phi-3-medium-4k-instruct-Q5_K_S.gguf +2 -2
  11. Phi-3-medium-4k-instruct-Q6_K.gguf +2 -2
  12. Phi-3-medium-4k-instruct-Q8_0.gguf +2 -2
  13. README.md +23 -31
Phi-3-medium-4k-instruct-Q2_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9fb9f379394bbd244df9e9a2540a7a2476227fa6373641c1847a206d585349b1
3
- size 5143000160
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab411a477d13d0fb61923b77d26960c8783272d66dc05d05a631048bceb9697b
3
+ size 5204447392
Phi-3-medium-4k-instruct-Q3_K_L.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8af8052c6b5097d31295672a51103564755c474d7713edc18b3313f00a6683f0
3
- size 7490297440
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bbf9cd9c899b6000e3e7a35cadd55b5cf6ab0d117e5bde0dae077c587355af8d
3
+ size 7348992672
Phi-3-medium-4k-instruct-Q3_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e9e38028ebe84a0b02301342fd0d042a21f797c8b3d173f6e6930b9c61260b3a
3
- size 6923411040
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb883ac66623fd8ea6dc7ee5617cc475357a9c6d28cca12fe78f58197fe7be28
3
+ size 6750976672
Phi-3-medium-4k-instruct-Q3_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f895ba9270e54e0be692c83ba942ec65d4a597e6794f71bb13cb079b8ac1075e
3
- size 6064889440
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d391d17d965b4c8fb769218c9ae1cafd994a54a642f57b3e70d8cce2ee0c1ec0
3
+ size 6064896672
Phi-3-medium-4k-instruct-Q4_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ce54b212b64a6429a04df0270978af9b4006fefdfcf08d5f486b56d8201fd206
3
- size 7897124960
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3357c77c2fec18e9b0655703171316f977713c0e436db29550de25553c89c16e
3
+ size 7897132192
Phi-3-medium-4k-instruct-Q4_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4ca431f3e0ccef8af8ff9a81e9f797f4c0e70484c35dabef594cb8289ca33237
3
- size 8566820960
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b9ded40380f3e2e1cb219b52c57cef992b423847382cd1faa1a2f96be82e81b
3
+ size 8404012192
Phi-3-medium-4k-instruct-Q4_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1de015231d7ee75f80ddcb8990962874689ada39d33aa3524c7b6bd4b0ea4f70
3
- size 7954468960
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11fc6f23100e2b10f70db92d7f569450fe04bac349866ec69051784e3c4c6500
3
+ size 7957752992
Phi-3-medium-4k-instruct-Q5_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9b5b944c7b2123b2c432c321be1350f1e361fd3dda8e9680f763fe2f0c9dc857
3
- size 9621581920
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1db29faccce31ff1f25309f3dd4d6e42e067a9dae368bf51c5898b3c92d9d66c
3
+ size 9621589152
Phi-3-medium-4k-instruct-Q5_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d9dd9e470d392fe5aab22ddb095e41105a494f93a741cb457e2f94e501cafcb5
3
- size 10074189920
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:563814fddc4e61c385ed8f6436a51a1269595dd570760e34c5468fd1b726ccab
3
+ size 9882709152
Phi-3-medium-4k-instruct-Q5_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d10f2cf1576ff37519c9aeeec46999fec841e24a9b3b215e6dbee8b76a51f023
3
- size 9621581920
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d18e7e96f2a3683531cbe8226d50bdf1bfdad86c610fd02b3c567658ac65e29
3
+ size 9621589152
Phi-3-medium-4k-instruct-Q6_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fd58d284092ea403771f35fca424301a74791259df09b75bf9289a676d0d8f33
3
- size 11453817440
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c81399cf50b6b293f67c8db62eed2b666a83fb79d3c5b2a123fdd378dffbc72
3
+ size 11453824672
Phi-3-medium-4k-instruct-Q8_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:41102ed2a0343f1f07856204b4b686de76a509d49aedc7a49f9c4d1cea5de992
3
- size 14834712160
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7cd0cda6efadc5641f715166b69f611b4dc684a0e06df905d90fb93641e92e5f
3
+ size 14834719392
README.md CHANGED
@@ -1,22 +1,16 @@
1
  ---
2
- license: mit
3
- license_link: https://huggingface.co/microsoft/Phi-3-medium-4k-instruct/resolve/main/LICENSE
4
  language:
5
- - multilingual
6
- pipeline_tag: text-generation
 
7
  tags:
8
- - nlp
9
- - code
 
 
10
  - TensorBlock
11
  - GGUF
12
- inference:
13
- parameters:
14
- temperature: 0.7
15
- widget:
16
- - messages:
17
- - role: user
18
- content: Can you provide ways to eat combinations of bananas and dragonfruits?
19
- base_model: microsoft/Phi-3-medium-4k-instruct
20
  ---
21
 
22
  <div style="width: auto; margin-left: auto; margin-right: auto">
@@ -30,12 +24,11 @@ base_model: microsoft/Phi-3-medium-4k-instruct
30
  </div>
31
  </div>
32
 
33
- ## microsoft/Phi-3-medium-4k-instruct - GGUF
34
 
35
- This repo contains GGUF format model files for [microsoft/Phi-3-medium-4k-instruct](https://huggingface.co/microsoft/Phi-3-medium-4k-instruct).
36
-
37
- The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
38
 
 
39
 
40
  <div style="text-align: left; margin: 20px 0;">
41
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
@@ -45,7 +38,6 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
45
 
46
  ## Prompt template
47
 
48
-
49
  ```
50
  <|user|>
51
  {prompt}<|end|>
@@ -56,18 +48,18 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
56
 
57
  | Filename | Quant type | File Size | Description |
58
  | -------- | ---------- | --------- | ----------- |
59
- | [Phi-3-medium-4k-instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q2_K.gguf) | Q2_K | 4.790 GB | smallest, significant quality loss - not recommended for most purposes |
60
- | [Phi-3-medium-4k-instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_S.gguf) | Q3_K_S | 5.648 GB | very small, high quality loss |
61
- | [Phi-3-medium-4k-instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_M.gguf) | Q3_K_M | 6.448 GB | very small, high quality loss |
62
- | [Phi-3-medium-4k-instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_L.gguf) | Q3_K_L | 6.976 GB | small, substantial quality loss |
63
- | [Phi-3-medium-4k-instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_0.gguf) | Q4_0 | 7.355 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
64
- | [Phi-3-medium-4k-instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_K_S.gguf) | Q4_K_S | 7.408 GB | small, greater quality loss |
65
- | [Phi-3-medium-4k-instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_K_M.gguf) | Q4_K_M | 7.978 GB | medium, balanced quality - recommended |
66
- | [Phi-3-medium-4k-instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_0.gguf) | Q5_0 | 8.961 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
67
- | [Phi-3-medium-4k-instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_K_S.gguf) | Q5_K_S | 8.961 GB | large, low quality loss - recommended |
68
- | [Phi-3-medium-4k-instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_K_M.gguf) | Q5_K_M | 9.382 GB | large, very low quality loss - recommended |
69
- | [Phi-3-medium-4k-instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q6_K.gguf) | Q6_K | 10.667 GB | very large, extremely low quality loss |
70
- | [Phi-3-medium-4k-instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q8_0.gguf) | Q8_0 | 13.816 GB | very large, extremely low quality loss - not recommended |
71
 
72
 
73
  ## Downloading instruction
 
1
  ---
 
 
2
  language:
3
+ - en
4
+ library_name: transformers
5
+ license: mit
6
  tags:
7
+ - unsloth
8
+ - phi3
9
+ - transformers
10
+ - phi
11
  - TensorBlock
12
  - GGUF
13
+ base_model: unsloth/Phi-3-medium-4k-instruct
 
 
 
 
 
 
 
14
  ---
15
 
16
  <div style="width: auto; margin-left: auto; margin-right: auto">
 
24
  </div>
25
  </div>
26
 
27
+ ## unsloth/Phi-3-medium-4k-instruct - GGUF
28
 
29
+ This repo contains GGUF format model files for [unsloth/Phi-3-medium-4k-instruct](https://huggingface.co/unsloth/Phi-3-medium-4k-instruct).
 
 
30
 
31
+ The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4242](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
32
 
33
  <div style="text-align: left; margin: 20px 0;">
34
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
 
38
 
39
  ## Prompt template
40
 
 
41
  ```
42
  <|user|>
43
  {prompt}<|end|>
 
48
 
49
  | Filename | Quant type | File Size | Description |
50
  | -------- | ---------- | --------- | ----------- |
51
+ | [Phi-3-medium-4k-instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q2_K.gguf) | Q2_K | 5.204 GB | smallest, significant quality loss - not recommended for most purposes |
52
+ | [Phi-3-medium-4k-instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_S.gguf) | Q3_K_S | 6.065 GB | very small, high quality loss |
53
+ | [Phi-3-medium-4k-instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_M.gguf) | Q3_K_M | 6.751 GB | very small, high quality loss |
54
+ | [Phi-3-medium-4k-instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_L.gguf) | Q3_K_L | 7.349 GB | small, substantial quality loss |
55
+ | [Phi-3-medium-4k-instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_0.gguf) | Q4_0 | 7.897 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
56
+ | [Phi-3-medium-4k-instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_K_S.gguf) | Q4_K_S | 7.958 GB | small, greater quality loss |
57
+ | [Phi-3-medium-4k-instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_K_M.gguf) | Q4_K_M | 8.404 GB | medium, balanced quality - recommended |
58
+ | [Phi-3-medium-4k-instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_0.gguf) | Q5_0 | 9.622 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
59
+ | [Phi-3-medium-4k-instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_K_S.gguf) | Q5_K_S | 9.622 GB | large, low quality loss - recommended |
60
+ | [Phi-3-medium-4k-instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_K_M.gguf) | Q5_K_M | 9.883 GB | large, very low quality loss - recommended |
61
+ | [Phi-3-medium-4k-instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q6_K.gguf) | Q6_K | 11.454 GB | very large, extremely low quality loss |
62
+ | [Phi-3-medium-4k-instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q8_0.gguf) | Q8_0 | 14.835 GB | very large, extremely low quality loss - not recommended |
63
 
64
 
65
  ## Downloading instruction