README.md

metadata

base_model:
  - epfl-llm/meditron-7b
  - bofenghuang/vigogne-2-7b-instruct
library_name: transformers
tags:
  - mergekit
  - merge

vigomed-2-7b-slerp

This is a merged model combining Vigogne-2-7B-Instruct and Meditron-7B.
It was extracted using mergekit to create a model that retains both French language capabilities and medical expertise.

Merge Method

This model was merged using the SLERP merge method.

Models Merged

The following models were included in the merge:

• epfl-llm/meditron-7b
• bofenghuang/vigogne-2-7b-instruct

Available Adapters

• LoRA fine-tuning

Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
      - model: bofenghuang/vigogne-2-7b-instruct
        layer_range: [0, 32]
      - model: epfl-llm/meditron-7b
        layer_range: [0, 32]
merge_method: slerp
base_model: bofenghuang/vigogne-2-7b-instruct
parameters:
  t:
    - filter: self_attn
      value: [0, 0.6, 0.4, 0.8, 1]
    - filter: mlp
      value: [1, 0.4, 0.6, 0.2, 0]
    - value: 0.5
dtype: bfloat16