Any-to-Any
Transformers
Safetensors
qwen3_vl
image-to-text
KaituoFeng nielsr HF Staff commited on
Commit
2a13d6e
Β·
verified Β·
1 Parent(s): b3d0e03

Improve model card: Add tags, license, detailed description, and performance (#1)

Browse files

- Improve model card: Add tags, license, detailed description, and performance (362721bcedbaeb7a246af06ac928b328521f4cf2)


Co-authored-by: Niels Rogge <[email protected]>

Files changed (1) hide show
  1. README.md +76 -4
README.md CHANGED
@@ -1,10 +1,82 @@
1
  ---
2
- datasets:
3
- - OneThink/OneThinker-train-data
4
  base_model:
5
  - Qwen/Qwen3-VL-8B-Instruct
 
 
 
 
 
6
  ---
7
 
8
- This repository contains the model presented in: [OneThinker: All-in-one Reasoning Model for Image and Video](https://arxiv.org/abs/2512.03043)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
 
10
- Code: https://github.com/tulerfeng/OneThinker
 
 
 
 
 
 
 
 
1
  ---
 
 
2
  base_model:
3
  - Qwen/Qwen3-VL-8B-Instruct
4
+ datasets:
5
+ - OneThink/OneThinker-train-data
6
+ pipeline_tag: any-to-any
7
+ library_name: transformers
8
+ license: apache-2.0
9
  ---
10
 
11
+ # OneThinker: All-in-one Reasoning Model for Image and Video
12
+
13
+ [[πŸ“– Paper](https://huggingface.co/papers/2512.03043)] [[πŸ€— OneThinker-8B-model](https://huggingface.co/OneThink/OneThinker-8B)] [[πŸ€— OneThinker-SFT-model](https://huggingface.co/OneThink/OneThinker-SFT-Qwen3-8B)] [[πŸ€— OneThinker-train-data](https://huggingface.co/datasets/OneThink/OneThinker-train-data)] [[πŸ€— OneThinker-eval](https://huggingface.co/datasets/OneThink/OneThinker-eval)] [[πŸ”— Code](https://github.com/tulerfeng/OneThinker)]
14
+
15
+ ## πŸ‘€ About OneThinker
16
+
17
+ <div align="center">
18
+ <img src="https://github.com/tulerfeng/OneThinker/raw/main/assets/teaser.png" alt="OneThinker Teaser Image" width="95%">
19
+ </div>
20
+
21
+ We introduce **OneThinker**, an all-in-one multimodal reasoning generalist that is **capable of thinking across a wide range of fundamental visual tasks within a single model**.
22
+
23
+ OneThinker unifies image and video understanding across diverse fundamental visual tasks, including question answering, captioning, spatial and temporal grounding, tracking, and segmentation. To achieve this, we construct the large-scale **OneThinker-600k** multi-task training corpus and build **OneThinker-SFT-340k** with high-quality CoT annotations for SFT cold start. Furthermore, we propose **EMA-GRPO**, a new RL method that balances heterogeneous reward signals across diverse visual tasks by tracking task-wise moving averages of reward standard deviations for balanced optimization.
24
+
25
+ OneThinker demonstrates **strong performance on 31 benchmarks across 10 fundamental vision tasks**, while showing effective knowledge transfer between certain tasks and promising zero-shot generalization ability, marking a step toward a unified multimodal reasoning generalist.
26
+
27
+ All code, models, and data are fully released.
28
+
29
+ ## πŸ”₯ News
30
+ - [2025/12/03] We release the code, model, data of OneThinker
31
+
32
+ ## πŸ“ Features
33
+
34
+ + Support Qwen3-VL Training
35
+ + Support Image-Video mixed training
36
+ + Support reward types in diverse visual tasks
37
+ + Provide full pipeline (dataset, SFT training, RL training, evaluation, etc)
38
+
39
+ ## πŸ” Dataset
40
+
41
+ Our dataset covers both image and video modalities and spans a series of fundamental visual reasoning tasks, including rule-based QA, open-ended QA, captioning, spatial grounding, temporal grounding, spatio-temporal grounding, tracking, and segmentation.
42
+
43
+ <div align="center">
44
+ <img src="https://github.com/tulerfeng/OneThinker/raw/main/assets/dataset.png" alt="OneThinker Dataset Overview" width="90%">
45
+ </div>
46
+
47
+ To enable effective SFT initialization for reasoning, we leverage a strong proprietary model, Seed1.5-VL to produce CoT annotations.
48
+
49
+ ## πŸ† Performance
50
+
51
+ Our model obtains significant performance gains after training based on Qwen3-VL-Instruct-8B across diverse visual tasks. For example, OneThinker-8B reaches 70.6% accuracy on MMMU, 64.3% on MathVerse, 66.2% on VideoMMMU, 93.7 on Refcoco-testA, 54.9 J&F on ReasonVOS.
52
+
53
+ <div align="center">
54
+ <img src="https://github.com/tulerfeng/OneThinker/raw/main/assets/performance.png" alt="OneThinker Performance Table" width="90%">
55
+ </div>
56
+
57
+ Besides, we also observe beneficial cross-task and cross-modality knowledge transfer, along with promising preliminary zero-shot generalization under unified training. This highlights the effectiveness and generalization ability of our unified training framework across diverse visual tasks.
58
+
59
+ ## πŸŽ₯ Demo
60
+
61
+ For detailed interactive demos with reasoning examples across various tasks (QA, Tracking, Segmentation), please refer to the [GitHub repository's Demo section](https://github.com/tulerfeng/OneThinker#demo).
62
+
63
+ ## πŸš€ Inference & Evaluation
64
+
65
+ For inference on a single example, you may refer to the provided script in the GitHub repository:
66
+ ```bash
67
+ python ./Evaluation/inference_single/inference.py
68
+ ```
69
+ For more detailed instructions on environment setup, training scripts, and comprehensive evaluation, please refer to the [OneThinker GitHub repository](https://github.com/tulerfeng/OneThinker).
70
+
71
+ ## πŸ“„ Citations
72
+
73
+ If you find our work helpful for your research, please consider citing our work.
74
 
75
+ ```bibtex
76
+ @article{feng2025onethinker,
77
+ title={OneThinker: All-in-one Reasoning Model for Image and Video},
78
+ author={Feng, Kaituo and Zhang, Manyuan and Li, Hongyu and Fan, Kaixuan and Chen, Shuang and Jiang, Yilei and Zheng, Dian and Sun, Peiwen and Zhang, Yiyuan and Sun, Haoze and others},
79
+ journal={arXiv preprint arXiv:2512.03043},
80
+ year={2025}
81
+ }
82
+ ```