Abrubptly Terminates Output
This model is stellar for evaluating complex concerns and coming up with reasoned solutions, especially when paired with BasedBase/Qwen3-Coder-30B-A3B-Instruct-480B-Distill-V2-Fp32 to do the heavy-lifting coding work.
Unfortunately, the model has a tendency to just stop output mid-flow. Running with --isq q8_0 --mem 16384 --penalty 1.1 --temperature 0.7 --top-p 0.95 --top-k 20. Any thoughts on what could be done to mitigate this concern?
Separately, if this can be fixed, this thinking treatment might be very handy for the coder model i mentioned above and BasedBase' distillation from the corresponding large model might apply to the Qwen3MoE base of this one (i'm guessing prior to the thinking pass to ensure appropriate logit dispersal) rather well too. Potentially doing that distillation once, then applying a thinking pass, and reinforcing the distillation again might give the "thinking expert" elements "more with which to work" so to speak during their phase of generation.
I added the think \n tag after "assistant\n" in the chat template to encourage the model to generate a chain of thought when handling certain complex problems. However, due to the relatively low proportion of the Qwen3-30B-A3B-Thinking-2507 model, the final output often contains only the chain of thought and lacks a conclusive summary answer.
Removing the think\n tag after "assistant\n" resolves the issue of the missing summary answer, but at the cost of the model no longer generating any chain of thought whatsoever.
Therefore, I have always regarded this model as merely a half-finished product — unexpectedly, it has gained widespread popularity among users. An improved version of the model is currently being uploaded and will be officially released after testing is complete.
Thank you very much. ran into a similar thing with BasedBase' DeepSeek distillation for Qwen3 however removal of the thinking tag from the chat template in that case actually allowed it to properly insert its own starter thinking tag which it was not doing prior (same as here).