inclusionAI
/

Ring-mini-linear-2.0-GPTQ-int4

Text Generation

bailing_moe_linear

compressed-tensors

Model card Files Files and versions

zheyishine commited on Oct 5

Commit

6c6ccc9

·

verified ·

1 Parent(s): 5e5be73

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -87,6 +87,7 @@ vllm serve inclusionAI/Ring-mini-linear-2.0-GPTQ-int4 \
               --gpu-memory-utilization 0.90 \
               --max-num-seqs 128 \
               --no-enable-prefix-caching
 ```

               --gpu-memory-utilization 0.90 \
               --max-num-seqs 128 \
               --no-enable-prefix-caching
+              --api-key your-api-key
 ```