|
|
--- |
|
|
license: mit |
|
|
--- |
|
|
# internlm2.5_7b_distill_orpo |
|
|
## Architecture |
|
|
<div align="center"> |
|
|
<img src="distill_orpo.png" width="800"/> |
|
|
</div> |
|
|
|
|
|
## Base model |
|
|
[internlm2_5-7b-chat](https://huggingface.co/internlm/internlm2_5-7b-chat) |
|
|
|
|
|
## Datasets used for training |
|
|
Created a preference optimization dataset [**PKU-SafeRLHF-orpo-72k**](https://huggingface.co/datasets/juneup/PKU-SafeRLHF-orpo-72k) from [PKU-SafeRLHF-single-dimension](https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF-single-dimension) |
|
|
|
|
|
## Download model |
|
|
```bash |
|
|
git lfs install |
|
|
git clone https://huggingface.co/juneup/internlm2.5_7b_distill_orpo |
|
|
``` |
|
|
If you want to clone without large files - just their pointers |
|
|
```bash |
|
|
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/juneup/internlm2.5_7b_distill_orpo |
|
|
``` |
|
|
|
|
|
### Download at Ollama |
|
|
```bash |
|
|
ollama run Juneup/internlm2.5_7b_distill:orpo_q4_k_m |
|
|
|
