Serve With vLLM

by faheemraza1 - opened Sep 28

Sep 28

Can someone share the command to serve this model on an RTX 3090?

Volkin

Oct 27

NVFP4 is only supported on Nvidia 50 series GPU.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment