|
|
--- |
|
|
language: en |
|
|
license: apache-2.0 |
|
|
library_name: transformers |
|
|
tags: |
|
|
- llama |
|
|
- vicuna |
|
|
- eagle |
|
|
- text-generation |
|
|
pipeline_tag: text-generation |
|
|
datasets: |
|
|
- Aeala/ShareGPT_Vicuna_unfiltered |
|
|
base_model: |
|
|
- lmsys/vicuna-13b-v1.3 |
|
|
--- |
|
|
# Eagle-Vicuna-13B-v1.3 |
|
|
|
|
|
This is a fine-tuned version of Vicuna-13B using the EAGLE method for fast inference. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Base model**: [lmsys/vicuna-13b-v1.3](https://huggingface.co/lmsys/vicuna-13b-v1.3) |
|
|
- **Method**: EAGLE (Efficient speculative decoding) |
|
|
- **Training data**: ShareGPT, etc. |
|
|
|
|
|
|
|
|
|
|
|
## 模型配置 |
|
|
base_model: lmsys/vicuna-13b-v1.3 |
|
|
|
|
|
eagle-model |
|
|
|
|
|
```text |
|
|
Model( |
|
|
(embed_tokens): Embedding(32000, 4096, padding_idx=0) |
|
|
(layers): ModuleList( |
|
|
(0): LlamaDecoderLayer( |
|
|
(self_attn): LlamaAttention( |
|
|
(q_proj): Linear(in_features=4096, out_features=4096, bias=False) |
|
|
(k_proj): Linear(in_features=4096, out_features=4096, bias=False) |
|
|
(v_proj): Linear(in_features=4096, out_features=4096, bias=False) |
|
|
(o_proj): Linear(in_features=4096, out_features=4096, bias=False) |
|
|
(rotary_emb): LlamaRotaryEmbedding() |
|
|
) |
|
|
(mlp): LlamaMLP( |
|
|
(gate_proj): Linear(in_features=4096, out_features=11008, bias=False) |
|
|
(up_proj): Linear(in_features=4096, out_features=11008, bias=False) |
|
|
(down_proj): Linear(in_features=11008, out_features=4096, bias=False) |
|
|
(act_fn): SiLU() |
|
|
) |
|
|
(post_attention_layernorm): LlamaRMSNorm() |
|
|
) |
|
|
) |
|
|
(fc): Linear(in_features=8192, out_features=4096, bias=True) |
|
|
(act): SiLU() |
|
|
) |
|
|
``` |
|
|
|
|
|
vicuna-13B-config.json |
|
|
|
|
|
```json |
|
|
{ |
|
|
"architectures": [ |
|
|
"LlamaForCausalLM" |
|
|
], |
|
|
"bos_token_id": 1, |
|
|
"eos_token_id": 2, |
|
|
"hidden_act": "silu", |
|
|
"hidden_size": 5120, |
|
|
"initializer_range": 0.02, |
|
|
"intermediate_size": 13824, |
|
|
"max_position_embeddings": 2048, |
|
|
"model_type": "llama", |
|
|
"num_attention_heads": 40, |
|
|
"num_hidden_layers": 1, |
|
|
"pad_token_id": 0, |
|
|
"rms_norm_eps": 1e-06, |
|
|
"tie_word_embeddings": false, |
|
|
"torch_dtype": "float16", |
|
|
"transformers_version": "4.28.1", |
|
|
"use_cache": true, |
|
|
"vocab_size": 32000 |
|
|
} |
|
|
|
|
|
|
|
|
``` |
|
|
## 模型训练 |
|
|
|
|
|
### 数据生成 |
|
|
```bash |
|
|
python -m eagle.ge_data.allocation --outdir ../data |
|
|
``` |
|
|
|
|
|
### 训练 |
|
|
|
|
|
```bash |
|
|
accelerate launch -m --mixed_precision=bf16 eagle.train.main --tmpdir eagle/data/sharegpt_0_67999_mufp16 --cpdir eagle/checkpoint --configpath eagle/train/vicuna_13B_config.json |
|
|
``` |
|
|
|
|
|
|
|
|
### 模型上传 |
|
|
|
|
|
|
|
|
|
|
|
```py |
|
|
from huggingface_hub import HfApi |
|
|
|
|
|
api = HfApi() |
|
|
|
|
|
# 只上传修改后的 README.md 文件 |
|
|
api.upload_file( |
|
|
path_or_fileobj="checkpoints/eagle-vicuna-13B/README.md", # 本地修改后的 README 路径 |
|
|
path_in_repo="README.md", # 仓库中的目标路径（根目录） |
|
|
repo_id="Gavin1104/eagle-vicuna-13b-v1.3", |
|
|
repo_type="model" |
|
|
) |
|
|
``` |
