Monohydroxides nielsr HF Staff commited on
Commit
c8d3a40
·
verified ·
1 Parent(s): 4371625

Improve model card with paper link, usage example, and citation (#2)

Browse files

- Improve model card with paper link, usage example, and citation (be173834915ee86e243dae7f50a4b6e71998065e)


Co-authored-by: Niels Rogge <[email protected]>

Files changed (1) hide show
  1. README.md +38 -4
README.md CHANGED
@@ -1,12 +1,46 @@
1
  ---
2
- license: mit
3
  library_name: transformers
 
4
  pipeline_tag: text-generation
5
  ---
6
- # LLaDA 1.5
7
 
8
- We introduce LLaDA 1.5, a competitive large diffusion language model, trained by variance-reduced preference optimization (VRPO).
 
 
 
 
9
 
10
  [Project Page](https://ml-gsai.github.io/LLaDA-1.5-Demo/)
11
 
12
- [Code](https://github.com/ML-GSAI/LLaDA-1.5)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
2
  library_name: transformers
3
+ license: mit
4
  pipeline_tag: text-generation
5
  ---
 
6
 
7
+ # LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models
8
+
9
+ We introduce LLaDA 1.5, a competitive large diffusion language model, trained by variance-reduced preference optimization (VRPO), as presented in the paper [LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models](https://huggingface.co/papers/2505.19223).
10
+
11
+ Compared with LLaDA-8B-Instruct, LLaDA 1.5 achieves better performance on a wide range of tasks, including Math, Code, and Alignment tasks.
12
 
13
  [Project Page](https://ml-gsai.github.io/LLaDA-1.5-Demo/)
14
 
15
+ [Code](https://github.com/ML-GSAI/LLaDA-1.5)
16
+
17
+ <div style="display: flex; justify-content: center; align-items: center; width: 100%; margin: 0 auto;">
18
+ <img src="https://github.com/ML-GSAI/LLaDA-1.5/raw/main/assets/llada_1_5.png" style="width: 50%; display: block; margin: 0 auto;" />
19
+ </div>
20
+
21
+ ## Inference
22
+
23
+ The LLaDA 1.5 model is available on [Huggingface](https://huggingface.co/GSAI-ML/LLaDA-1.5). Please employ the [transformers](https://huggingface.co/docs/transformers/index) to load.
24
+
25
+ ```python
26
+ from transformers import AutoModel, AutoTokenizer
27
+ import torch
28
+
29
+ tokenizer = AutoTokenizer.from_pretrained('GSAI-ML/LLaDA-1.5', trust_remote_code=True)
30
+ model = AutoModel.from_pretrained('GSAI-ML/LLaDA-1.5', trust_remote_code=True, torch_dtype=torch.bfloat16)
31
+ ```
32
+
33
+ The model is based on LLaDA-8B-Instruct, you can use the code for [LLaDA-8B-Instruct](https://github.com/ML-GSAI/LLaDA/blob/main/generate.py) to inference.
34
+
35
+ ## Citation
36
+
37
+ Please consider cite:
38
+
39
+ ```bibtex
40
+ @article{zhu2025llada,
41
+ title={LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models},
42
+ author={Zhu, Fengqi and Wang, Rongzhen and Nie, Shen and Zhang, Xiaolu and Wu, Chunwei and Hu, Jun and Zhou, Jun and Chen, Jianfei and Lin, Yankai and Wen, Ji-Rong and others},
43
+ journal={arXiv preprint arXiv:2505.19223},
44
+ year={2025}
45
+ }
46
+ ```