metadata
base_model:
- nbeerbower/Llama-3.1-Nemotron-lorablated-70B
- abacusai/Dracarys2-Llama-3.1-70B-Instruct
- Ppoyaa/MythoNemo-L3.1-70B-v1.0
- shisa-ai/shisa-v2-llama3.3-70b
- Doctor-Shotgun/L3.3-70B-Magnum-Diamond
library_name: transformers
tags:
- mergekit
- merge
merged
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the DARE TIES merge method using nbeerbower/Llama-3.1-Nemotron-lorablated-70B as a base.
Models Merged
The following models were included in the merge:
- abacusai/Dracarys2-Llama-3.1-70B-Instruct
- Ppoyaa/MythoNemo-L3.1-70B-v1.0
- shisa-ai/shisa-v2-llama3.3-70b
- Doctor-Shotgun/L3.3-70B-Magnum-Diamond
Configuration
The following YAML configuration was used to produce this model:
models:
- model: abacusai/Dracarys2-Llama-3.1-70B-Instruct
parameters:
weight: 0.25
density: 0.5
- model: shisa-ai/shisa-v2-llama3.3-70b
parameters:
weight: 0.25
density: 0.5
- model: Doctor-Shotgun/L3.3-70B-Magnum-Diamond
parameters:
weight: 0.25
density: 0.5
- model: Ppoyaa/MythoNemo-L3.1-70B-v1.0
parameters:
weight: 0.25
density: 0.5
merge_method: dare_ties
base_model: nbeerbower/Llama-3.1-Nemotron-lorablated-70B
parameters:
normalize: false
out_dtype: bfloat16
chat_template: llama3
tokenizer:
source: base
pad_to_multiple_of: 8