Commit
·
8f5cd82
1
Parent(s):
3bc5cf3
Update README.md
Browse files
README.md
CHANGED
|
@@ -14,6 +14,10 @@ Sheared-LLaMA-1.3B is a model pruned and further pre-trained from [meta-llama/Ll
|
|
| 14 |
model = AutoModelForCausalLM.from_pretrained("princeton-nlp/Sheared-LLaMA-1.3B")
|
| 15 |
```
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
## Downstream Tasks
|
| 18 |
|
| 19 |
We evaluate on an extensive set of downstream tasks including reasoning, reading comprehension, language modeling and knowledge intensive tasks. Our Sheared-LLaMA models outperform existing large language models.
|
|
|
|
| 14 |
model = AutoModelForCausalLM.from_pretrained("princeton-nlp/Sheared-LLaMA-1.3B")
|
| 15 |
```
|
| 16 |
|
| 17 |
+
- Smaller-scale
|
| 18 |
+
- Same vocabulary as LLaMA1 and LLaMA2
|
| 19 |
+
- Derived with 50B tokens by utilizing existing strong LLMs
|
| 20 |
+
|
| 21 |
## Downstream Tasks
|
| 22 |
|
| 23 |
We evaluate on an extensive set of downstream tasks including reasoning, reading comprehension, language modeling and knowledge intensive tasks. Our Sheared-LLaMA models outperform existing large language models.
|