Upload 2 files (#2)

- Upload 2 files (24408cd0ce19b33be106d7663f117a0ffc802555)


Co-authored-by: kuroba <[email protected]>

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+distill.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,40 +1,32 @@
----
-license: mit
----
-## 数据集
-
-### 通用领域含思维链数据集：
-```bash
-git lfs install
-git clone https://huggingface.co/datasets/Congliu/Chinese-DeepSeek-R1-Distill-data-110k-SFT
-```
-若不想克隆大型文件：
-```bash
-GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/datasets/Congliu/Chinese-DeepSeek-R1-Distill-data-110k-SFT
-```
-### 含思维链心理辅导对话数据集：
-```bash
-git lfs install
-git clone https://huggingface.co/datasets/CAS-SIAT-XinHai/CPsyCoun
-```
-若不想克隆大型文件：
-```bash
-GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/datasets/CAS-SIAT-XinHai/CPsyCoun
-```
-
-## 模型
-
-### 下载
-```bash
-git lfs install
-git clone https://huggingface.co/juneup/internlm2.5_7b_distill
-```
-**若不想克隆大型文件：
-```bash
-GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/juneup/internlm2.5_7b_distill
-```
-
-### 在Ollama下载
-```bash
-ollama run Juneup/internlm2.5_7b_distill:q4_k_m
-```
+# internlm2.5_7b_distill 
+## 架构图
+<div align="center">
+  <img src="distill.png" width="800"/>
+</div>
+
+## 基座模型
+https://huggingface.co/Slipstream-Max/Emollm-InternLM2.5-7B-chat-GGUF-fp16
+
+## 数据集
+### 数据集组成
+5k条精选通用领域含思维链数据(https://huggingface.co/datasets/Congliu/Chinese-DeepSeek-R1-Distill-data-110k-SFT)+3k条含思维链心理辅导对话(https://huggingface.co/datasets/CAS-SIAT-XinHai/CPsyCoun)
+
+1. 110k的数据怎么挑选10k条？：基于score
+2. 3k条心理辅导对话：原始对话（alpaca，含history字段，不含思维链，来自于CPsyCounD），使用deepseek r1【通过设置max_token实现低成本获取思维链】给output增加思维链，获得含思维链的数据集
+
+### 训练方式
+qlora，变长注意力，数据拼接
+
+## 下载模型
+```bash
+git lfs install
+git clone https://huggingface.co/juneup/internlm2.5_7b_distill
+```
+若不想克隆大型文件：
+```bash
+GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/juneup/internlm2.5_7b_distill
+```
+
+### 在Ollama下载
+```bash
+ollama run Juneup/internlm2.5_7b_distill:q4_k_m