metadata
license: apache-2.0
library_name: transformers
Introduction
SDAR(Synergy of Diffusion and AutoRegression)-model is a new large language model that integrates autoregressive (AR) and discrete diffusion modeling strategies. It combines the efficient training paradigm of AR models with the highly parallel inference capability of diffusion models, while delivering performance fully on par with SOTA opensource AR models. At the same time, SDAR sets a new benchmark as the most powerful diffusion language model to date.
The SDAR series models undergo continued training on Qwen3 models.
Performance of SDAR-1.7B-Chat on various benchmarks
evaluation settings:
- MMLU: 5-shot
- Math500: 0-shot
- GSM8K: 0-shot
- HumanEval: 0-shot
- Sanitized_MBPP: 0-shot
- IFEval: 0-shot
- MathBench: 0-shot
| Model | MMLU | Math500 | GSM8K | HumanEval | Sanitized_MBPP | IFEval | MathBench |
|---|---|---|---|---|---|---|---|
| SDAR-1.7B-Chat | 62.9 | 63.2 | 80.06 | 61.59 | 61.09 | 43.44 | 63.55 |
| SDAR-4B-Chat | |||||||
| SDAR-8B-Chat | |||||||
| SDAR-30B-A3B-Chat |
Note: The 4B, 8B, and 30B models are coming soon. Performance results for these models will be released in the near future.
Inference
The inference code will come soon
Hightlights
- Performance: SDAR-1.7B-Chat achieves state-of-the-art.
- Efficiency: SDAR provides over 2× faster inference speed compared to the same-size AR models, while maintaining comparable performance.