Update README.md
Browse files
README.md
CHANGED
|
@@ -27,7 +27,8 @@ And some various other data, viewable at openerotica/mixed-rp
|
|
| 27 |
Every line of data was run through a large model in order to filter for low quality, repetition, and underage content.
|
| 28 |
|
| 29 |
[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
|
| 30 |
-
|
|
|
|
| 31 |
base_model: mistralai/Mistral-Nemo-Base-2407
|
| 32 |
model_type: AutoModelForCausalLM
|
| 33 |
tokenizer_type: AutoTokenizer
|
|
@@ -108,7 +109,6 @@ special_tokens:
|
|
| 108 |
tokens:
|
| 109 |
- "<|im_start|>"
|
| 110 |
|
| 111 |
-
|
| 112 |
# fsdp:
|
| 113 |
# - full_shard
|
| 114 |
# - auto_wrap
|
|
@@ -124,4 +124,3 @@ tokens:
|
|
| 124 |
# fsdp_sharding_strategy: FULL_SHARD
|
| 125 |
# fsdp_forward_prefetch: false
|
| 126 |
# fsdp_backward_prefetch: BACKWARD_PRE
|
| 127 |
-
'''
|
|
|
|
| 27 |
Every line of data was run through a large model in order to filter for low quality, repetition, and underage content.
|
| 28 |
|
| 29 |
[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
|
| 30 |
+
|
| 31 |
+
```yaml
|
| 32 |
base_model: mistralai/Mistral-Nemo-Base-2407
|
| 33 |
model_type: AutoModelForCausalLM
|
| 34 |
tokenizer_type: AutoTokenizer
|
|
|
|
| 109 |
tokens:
|
| 110 |
- "<|im_start|>"
|
| 111 |
|
|
|
|
| 112 |
# fsdp:
|
| 113 |
# - full_shard
|
| 114 |
# - auto_wrap
|
|
|
|
| 124 |
# fsdp_sharding_strategy: FULL_SHARD
|
| 125 |
# fsdp_forward_prefetch: false
|
| 126 |
# fsdp_backward_prefetch: BACKWARD_PRE
|
|
|