File size: 1,353 Bytes
dcdcb91 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
---
license: apache-2.0
tags:
- merge
- mergekit
- lazymergekit
- NousResearch/Meta-Llama-3.1-8B-Instruct
- EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta
- nvidia/OpenMath2-Llama3.1-8B
---
# Llama-3.1-8B-Squareroot
Llama-3.1-8B-Squareroot is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
* [NousResearch/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3.1-8B-Instruct)
* [EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta](https://huggingface.co/EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta)
* [nvidia/OpenMath2-Llama3.1-8B](https://huggingface.co/nvidia/OpenMath2-Llama3.1-8B)
## 🧩 Configuration
```yaml
models:
- model: NousResearch/Meta-Llama-3.1-8B-Instruct # instruct model
parameters:
density: [1, 0.8, 0.5]
weight: 1.0
- model: EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta # Reasoning/Logic model
parameters:
density: [0.9, 0.6, 0.4]
weight: [0.3, 0.6, 0.9]
- model: nvidia/OpenMath2-Llama3.1-8B # Math model
parameters:
density: 0.7
weight:
- filter: mlp
value: 0.7
- value: 0.3
merge_method: ties
base_model: vicgalle/Configurable-Llama-3.1-8B-Instruct # General base model
parameters:
normalize: true
dtype: float16
``` |