Update README.md
Browse files
README.md
CHANGED
|
@@ -10,7 +10,27 @@ tags:
|
|
| 10 |
- trl
|
| 11 |
- sft
|
| 12 |
base_model: augmxnt/shisa-base-7b-v1
|
|
|
|
|
|
|
|
|
|
| 13 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
# Uploaded model
|
| 16 |
|
|
|
|
| 10 |
- trl
|
| 11 |
- sft
|
| 12 |
base_model: augmxnt/shisa-base-7b-v1
|
| 13 |
+
datasets:
|
| 14 |
+
- NilanE/ParallelFiction-Ja_En-100k
|
| 15 |
+
- mpasila/ParallelFiction-Ja_En-100k-alpaca
|
| 16 |
---
|
| 17 |
+
Experimental model, may not perform that well. Dataset used is [a modified](https://huggingface.co/datasets/mpasila/ParallelFiction-Ja_En-100k-alpaca) version of [NilanE/ParallelFiction-Ja_En-100k](https://huggingface.co/datasets/NilanE/ParallelFiction-Ja_En-100k).
|
| 18 |
+
|
| 19 |
+
After training with an 8k context length it didn't appear to improve performance much at all. Not sure if I should keep training it (which is costly) or if I should fix some issues with the dataset (like it starting with Ch or Chapter) or I go back to finetuning Finnish models.
|
| 20 |
+
|
| 21 |
+
### Prompt format: Alpaca
|
| 22 |
+
```
|
| 23 |
+
Below is a translation task, paired with an input that provides further context. Write a response that appropriately completes the request.
|
| 24 |
+
|
| 25 |
+
### Instruction:
|
| 26 |
+
{}
|
| 27 |
+
|
| 28 |
+
### Input:
|
| 29 |
+
{}
|
| 30 |
+
|
| 31 |
+
### Response:
|
| 32 |
+
{}
|
| 33 |
+
```
|
| 34 |
|
| 35 |
# Uploaded model
|
| 36 |
|