Improve model card with paper link, usage example, and citation (#2)
Browse files- Improve model card with paper link, usage example, and citation (be173834915ee86e243dae7f50a4b6e71998065e)
Co-authored-by: Niels Rogge <[email protected]>
README.md
CHANGED
|
@@ -1,12 +1,46 @@
|
|
| 1 |
---
|
| 2 |
-
license: mit
|
| 3 |
library_name: transformers
|
|
|
|
| 4 |
pipeline_tag: text-generation
|
| 5 |
---
|
| 6 |
-
# LLaDA 1.5
|
| 7 |
|
| 8 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
|
| 10 |
[Project Page](https://ml-gsai.github.io/LLaDA-1.5-Demo/)
|
| 11 |
|
| 12 |
-
[Code](https://github.com/ML-GSAI/LLaDA-1.5)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
library_name: transformers
|
| 3 |
+
license: mit
|
| 4 |
pipeline_tag: text-generation
|
| 5 |
---
|
|
|
|
| 6 |
|
| 7 |
+
# LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models
|
| 8 |
+
|
| 9 |
+
We introduce LLaDA 1.5, a competitive large diffusion language model, trained by variance-reduced preference optimization (VRPO), as presented in the paper [LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models](https://huggingface.co/papers/2505.19223).
|
| 10 |
+
|
| 11 |
+
Compared with LLaDA-8B-Instruct, LLaDA 1.5 achieves better performance on a wide range of tasks, including Math, Code, and Alignment tasks.
|
| 12 |
|
| 13 |
[Project Page](https://ml-gsai.github.io/LLaDA-1.5-Demo/)
|
| 14 |
|
| 15 |
+
[Code](https://github.com/ML-GSAI/LLaDA-1.5)
|
| 16 |
+
|
| 17 |
+
<div style="display: flex; justify-content: center; align-items: center; width: 100%; margin: 0 auto;">
|
| 18 |
+
<img src="https://github.com/ML-GSAI/LLaDA-1.5/raw/main/assets/llada_1_5.png" style="width: 50%; display: block; margin: 0 auto;" />
|
| 19 |
+
</div>
|
| 20 |
+
|
| 21 |
+
## Inference
|
| 22 |
+
|
| 23 |
+
The LLaDA 1.5 model is available on [Huggingface](https://huggingface.co/GSAI-ML/LLaDA-1.5). Please employ the [transformers](https://huggingface.co/docs/transformers/index) to load.
|
| 24 |
+
|
| 25 |
+
```python
|
| 26 |
+
from transformers import AutoModel, AutoTokenizer
|
| 27 |
+
import torch
|
| 28 |
+
|
| 29 |
+
tokenizer = AutoTokenizer.from_pretrained('GSAI-ML/LLaDA-1.5', trust_remote_code=True)
|
| 30 |
+
model = AutoModel.from_pretrained('GSAI-ML/LLaDA-1.5', trust_remote_code=True, torch_dtype=torch.bfloat16)
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
The model is based on LLaDA-8B-Instruct, you can use the code for [LLaDA-8B-Instruct](https://github.com/ML-GSAI/LLaDA/blob/main/generate.py) to inference.
|
| 34 |
+
|
| 35 |
+
## Citation
|
| 36 |
+
|
| 37 |
+
Please consider cite:
|
| 38 |
+
|
| 39 |
+
```bibtex
|
| 40 |
+
@article{zhu2025llada,
|
| 41 |
+
title={LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models},
|
| 42 |
+
author={Zhu, Fengqi and Wang, Rongzhen and Nie, Shen and Zhang, Xiaolu and Wu, Chunwei and Hu, Jun and Zhou, Jun and Chen, Jianfei and Lin, Yankai and Wen, Ji-Rong and others},
|
| 43 |
+
journal={arXiv preprint arXiv:2505.19223},
|
| 44 |
+
year={2025}
|
| 45 |
+
}
|
| 46 |
+
```
|