Files changed (2) hide show
  1. README.md +35 -35
  2. distill.png +2 -2
README.md CHANGED
@@ -1,35 +1,35 @@
1
- ---
2
- license: mit
3
- ---
4
- # internlm2.5_7b_distill
5
- ## 架构图
6
- <div align="center">
7
- <img src="distill.png" width="800"/>
8
- </div>
9
-
10
- ## 基座模型
11
- [internlm2.6-7b-chat](https://huggingface.co/internlm/internlm2_5-7b-chat)
12
-
13
- ## 数据集
14
- ### 数据集组成
15
- 5k条精选通用领域含思维链数据(https://huggingface.co/datasets/Congliu/Chinese-DeepSeek-R1-Distill-data-110k-SFT)+3k条含思维链心理辅导对话(https://huggingface.co/datasets/CAS-SIAT-XinHai/CPsyCoun)
16
-
17
- 1. 110k的数据怎么挑选10k条?:基于score
18
- 2. 3k条心理辅导对话:原始对话(alpaca,含history字段,不含思维链,来自于CPsyCounD),使用deepseek r1【通过设置max_token实现低成本获取思维链】给output增加思维链,获得含思维链的数据集
19
-
20
- ### 训练方式
21
- qlora,变长注意力,数据拼接
22
-
23
- ## 下载模型
24
- ```bash
25
- git lfs install
26
- git clone https://huggingface.co/juneup/internlm2.5_7b_distill
27
- ```
28
- 若不想克隆大型文件:
29
- ```bash
30
- GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/juneup/internlm2.5_7b_distill
31
- ```
32
-
33
- ### 在Ollama下载
34
- ```bash
35
- ollama run Juneup/internlm2.5_7b_distill:q4_k_m
 
1
+ ---
2
+ license: mit
3
+ ---
4
+ # internlm2.5_7b_distill
5
+ ## 架构图
6
+ <div align="center">
7
+ <img src="distill.png" width="800"/>
8
+ </div>
9
+
10
+ ## 基座模型
11
+ https://huggingface.co/internlm/internlm2_5-7b-chat
12
+
13
+ ## 数据集
14
+ ### 数据集组成
15
+ 5k条精选通用领域含思维链数据(https://huggingface.co/datasets/Congliu/Chinese-DeepSeek-R1-Distill-data-110k-SFT)+3k条含思维链心理辅导对话(https://huggingface.co/datasets/CAS-SIAT-XinHai/CPsyCoun)
16
+
17
+ 1. 110k的数据怎么挑选10k条?:基于score
18
+ 2. 3k条心理辅导对话:原始对话(alpaca,含history字段,不含思维链,来自于CPsyCounD),使用deepseek r1【通过设置max_token实现低成本获取思维链】给output增加思维链,获得含思维链的数据集
19
+
20
+ ### 训练方式
21
+ qlora,变长注意力,数据拼接
22
+
23
+ ## 下载模型
24
+ ```bash
25
+ git lfs install
26
+ git clone https://huggingface.co/juneup/internlm2.5_7b_distill
27
+ ```
28
+ 若不想克隆大型文件:
29
+ ```bash
30
+ GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/juneup/internlm2.5_7b_distill
31
+ ```
32
+
33
+ ### 在Ollama下载
34
+ ```bash
35
+ ollama run Juneup/internlm2.5_7b_distill:q4_k_m
distill.png CHANGED

Git LFS Details

  • SHA256: 7dce2fa42bf884305c8a8f4a8807954888987b5896d331a95f52c16656f5bf30
  • Pointer size: 131 Bytes
  • Size of remote file: 177 kB

Git LFS Details

  • SHA256: fb1f756b5320fad586ed52bd9b01a8c029327fb9576fcb630ed95c9475d27b52
  • Pointer size: 131 Bytes
  • Size of remote file: 202 kB