Update README.md
Browse files
README.md
CHANGED
|
@@ -58,7 +58,7 @@ To run GGUFs correctly, you need the most recent version of KoboldCPP, and to pa
|
|
| 58 |
|
| 59 |
Thanks to DaringDuck and tofumagnate for info how to apply this fix.
|
| 60 |
|
| 61 |
-
To run this model on vLLM, you'll need to build it from source from the git repo, full GLM4 support hasn't reached release yet.
|
| 62 |
|
| 63 |
ExLLaMAv2 currently doesn't properly support GLM-4-32B, unlike 9B. EXL3 should work, but it's untested.
|
| 64 |
|
|
|
|
| 58 |
|
| 59 |
Thanks to DaringDuck and tofumagnate for info how to apply this fix.
|
| 60 |
|
| 61 |
+
~~To run this model on vLLM, you'll need to build it from source from the git repo, full GLM4 support hasn't reached release yet.~~ Should work OOTB on vLLM >=0.8.5.
|
| 62 |
|
| 63 |
ExLLaMAv2 currently doesn't properly support GLM-4-32B, unlike 9B. EXL3 should work, but it's untested.
|
| 64 |
|