Improve model card: Add tags, license, detailed description, and performance (#1)
Browse files- Improve model card: Add tags, license, detailed description, and performance (362721bcedbaeb7a246af06ac928b328521f4cf2)
Co-authored-by: Niels Rogge <[email protected]>
README.md
CHANGED
|
@@ -1,10 +1,82 @@
|
|
| 1 |
---
|
| 2 |
-
datasets:
|
| 3 |
-
- OneThink/OneThinker-train-data
|
| 4 |
base_model:
|
| 5 |
- Qwen/Qwen3-VL-8B-Instruct
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
---
|
| 7 |
|
| 8 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
| 2 |
base_model:
|
| 3 |
- Qwen/Qwen3-VL-8B-Instruct
|
| 4 |
+
datasets:
|
| 5 |
+
- OneThink/OneThinker-train-data
|
| 6 |
+
pipeline_tag: any-to-any
|
| 7 |
+
library_name: transformers
|
| 8 |
+
license: apache-2.0
|
| 9 |
---
|
| 10 |
|
| 11 |
+
# OneThinker: All-in-one Reasoning Model for Image and Video
|
| 12 |
+
|
| 13 |
+
[[π Paper](https://huggingface.co/papers/2512.03043)] [[π€ OneThinker-8B-model](https://huggingface.co/OneThink/OneThinker-8B)] [[π€ OneThinker-SFT-model](https://huggingface.co/OneThink/OneThinker-SFT-Qwen3-8B)] [[π€ OneThinker-train-data](https://huggingface.co/datasets/OneThink/OneThinker-train-data)] [[π€ OneThinker-eval](https://huggingface.co/datasets/OneThink/OneThinker-eval)] [[π Code](https://github.com/tulerfeng/OneThinker)]
|
| 14 |
+
|
| 15 |
+
## π About OneThinker
|
| 16 |
+
|
| 17 |
+
<div align="center">
|
| 18 |
+
<img src="https://github.com/tulerfeng/OneThinker/raw/main/assets/teaser.png" alt="OneThinker Teaser Image" width="95%">
|
| 19 |
+
</div>
|
| 20 |
+
|
| 21 |
+
We introduce **OneThinker**, an all-in-one multimodal reasoning generalist that is **capable of thinking across a wide range of fundamental visual tasks within a single model**.
|
| 22 |
+
|
| 23 |
+
OneThinker unifies image and video understanding across diverse fundamental visual tasks, including question answering, captioning, spatial and temporal grounding, tracking, and segmentation. To achieve this, we construct the large-scale **OneThinker-600k** multi-task training corpus and build **OneThinker-SFT-340k** with high-quality CoT annotations for SFT cold start. Furthermore, we propose **EMA-GRPO**, a new RL method that balances heterogeneous reward signals across diverse visual tasks by tracking task-wise moving averages of reward standard deviations for balanced optimization.
|
| 24 |
+
|
| 25 |
+
OneThinker demonstrates **strong performance on 31 benchmarks across 10 fundamental vision tasks**, while showing effective knowledge transfer between certain tasks and promising zero-shot generalization ability, marking a step toward a unified multimodal reasoning generalist.
|
| 26 |
+
|
| 27 |
+
All code, models, and data are fully released.
|
| 28 |
+
|
| 29 |
+
## π₯ News
|
| 30 |
+
- [2025/12/03] We release the code, model, data of OneThinker
|
| 31 |
+
|
| 32 |
+
## π Features
|
| 33 |
+
|
| 34 |
+
+ Support Qwen3-VL Training
|
| 35 |
+
+ Support Image-Video mixed training
|
| 36 |
+
+ Support reward types in diverse visual tasks
|
| 37 |
+
+ Provide full pipeline (dataset, SFT training, RL training, evaluation, etc)
|
| 38 |
+
|
| 39 |
+
## π Dataset
|
| 40 |
+
|
| 41 |
+
Our dataset covers both image and video modalities and spans a series of fundamental visual reasoning tasks, including rule-based QA, open-ended QA, captioning, spatial grounding, temporal grounding, spatio-temporal grounding, tracking, and segmentation.
|
| 42 |
+
|
| 43 |
+
<div align="center">
|
| 44 |
+
<img src="https://github.com/tulerfeng/OneThinker/raw/main/assets/dataset.png" alt="OneThinker Dataset Overview" width="90%">
|
| 45 |
+
</div>
|
| 46 |
+
|
| 47 |
+
To enable effective SFT initialization for reasoning, we leverage a strong proprietary model, Seed1.5-VL to produce CoT annotations.
|
| 48 |
+
|
| 49 |
+
## π Performance
|
| 50 |
+
|
| 51 |
+
Our model obtains significant performance gains after training based on Qwen3-VL-Instruct-8B across diverse visual tasks. For example, OneThinker-8B reaches 70.6% accuracy on MMMU, 64.3% on MathVerse, 66.2% on VideoMMMU, 93.7 on Refcoco-testA, 54.9 J&F on ReasonVOS.
|
| 52 |
+
|
| 53 |
+
<div align="center">
|
| 54 |
+
<img src="https://github.com/tulerfeng/OneThinker/raw/main/assets/performance.png" alt="OneThinker Performance Table" width="90%">
|
| 55 |
+
</div>
|
| 56 |
+
|
| 57 |
+
Besides, we also observe beneficial cross-task and cross-modality knowledge transfer, along with promising preliminary zero-shot generalization under unified training. This highlights the effectiveness and generalization ability of our unified training framework across diverse visual tasks.
|
| 58 |
+
|
| 59 |
+
## π₯ Demo
|
| 60 |
+
|
| 61 |
+
For detailed interactive demos with reasoning examples across various tasks (QA, Tracking, Segmentation), please refer to the [GitHub repository's Demo section](https://github.com/tulerfeng/OneThinker#demo).
|
| 62 |
+
|
| 63 |
+
## π Inference & Evaluation
|
| 64 |
+
|
| 65 |
+
For inference on a single example, you may refer to the provided script in the GitHub repository:
|
| 66 |
+
```bash
|
| 67 |
+
python ./Evaluation/inference_single/inference.py
|
| 68 |
+
```
|
| 69 |
+
For more detailed instructions on environment setup, training scripts, and comprehensive evaluation, please refer to the [OneThinker GitHub repository](https://github.com/tulerfeng/OneThinker).
|
| 70 |
+
|
| 71 |
+
## π Citations
|
| 72 |
+
|
| 73 |
+
If you find our work helpful for your research, please consider citing our work.
|
| 74 |
|
| 75 |
+
```bibtex
|
| 76 |
+
@article{feng2025onethinker,
|
| 77 |
+
title={OneThinker: All-in-one Reasoning Model for Image and Video},
|
| 78 |
+
author={Feng, Kaituo and Zhang, Manyuan and Li, Hongyu and Fan, Kaixuan and Chen, Shuang and Jiang, Yilei and Zheng, Dian and Sun, Peiwen and Zhang, Yiyuan and Sun, Haoze and others},
|
| 79 |
+
journal={arXiv preprint arXiv:2512.03043},
|
| 80 |
+
year={2025}
|
| 81 |
+
}
|
| 82 |
+
```
|