Can it run on NVIDIA A100 80G * 8 ？
I try to run with sglang ，but it says 'not support fp8e4nv'
Why?
I can run in latest llama.cpp on ONLY CPU server motherboard with 768Gb RAM total (by 64Gb, i can't find 128Gb RAM too rare). Memory is everything, speed is NOTHING esp with textual models. Server motherboards have CXL tech and 24 RAM slots for a reason! It looks like that https://www.gigabyte.com/Enterprise/Server-Motherboard
I've already run F16 quality which require 465Gb RAM/VRAM and i have a mixed results, the main problem of this model is too much thinking, but it show worse results than GLM4.5 - sometimes code perfect, sometimes broken. In Q8 quality (254Gb RAM) its much worse than GLM4.5.
P.S. my server motherboard is from year 2014, total PC cost with all RAM is kinda $1000. GPUs are usable in video, not in this.