This is an auto-thinking-switching model built with model merging and expert substitution techniques: it answers simple questions directly, gives brief thoughts to moderate ones, and delves deeply into difficult ones.

Model Highlights:

  • merge method: arcee_fusion

  • Highest precision: dtype: float32 + out_dtype: bfloat16

  • Context length: 262,144&1010000

Parameter Settings:

Auto-Thinking Mode

Temperature=0.6, TopP=0.95, TopK=20,MinP=0.

Step1: Hybrid Instruct Model and Thinking Model

Conduct initial mixing of the instruction model and reasoning model.

models:
  - model: Qwen/Qwen3-30B-A3B-Thinking-2507
merge_method: arcee_fusion
base_model: Qwen/Qwen3-30B-A3B-Instruct-2507
dtype: float32
out_dtype: bfloat16
tokenizer_source: base
name: Qwen3-30B-A3B-YOYO-AutoThink-preview

Step2: Expert replacement

Inspired by this paper , we use the following regular expression: ^model\.layers\.\d+\.mlp\.experts\.\d+\.(down_proj|gate_proj|up_proj)\.weight$ for expert replacement โ€” all experts in Qwen3-30B-A3B-YOYO-AutoThink-preview that match the regex are replaced with those from Qwen3-30B-A3B-Thinking-2507.

Downloads last month
228
Safetensors
Model size
31B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for YOYO-AI/Qwen3-30B-A3B-YOYO-AutoThink

Collection including YOYO-AI/Qwen3-30B-A3B-YOYO-AutoThink