Create README.md

README.md

@@ -0,0 +1,62 @@
+---
+license: cc-by-nc-4.0
+tags:
+- not-for-all-audiences
+- nsfw
+- mistral
+- pretrained
+---
+
+Mistral-7B and airoboros-mistral2.2-7b glued together and finetuned with qlora of Pippa and LimaRPv3 dataset.
+
+<!-- description start -->
+## Description
+
+This repo contains fp16 files of Mistral-11B-Airoboros-RP-v1.
+
+<!-- description end -->
+<!-- description start -->
+## Model used
+
+- [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1/)
+- [airoboros-mistral2.2-7b](https://huggingface.co/teknium/airoboros-mistral2.2-7b/)
+- PIPPA dataset 11B qlora
+- LimaRPv3 dataset 11B qlora
+
+<!-- description end -->
+<!-- prompt-template start -->
+## Prompt template: Alpaca or default
+
+```
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+{prompt}
+
+### Response:
+
+```
+
+```
+USER: <prompt>
+ASSISTANT:
+```
+
+## The secret sauce
+
+```
+slices:
+  - sources:
+    - model: mistralai/Mistral-7B-v0.1
+      layer_range: [0, 24]
+  - sources:
+    - model: teknium/airoboros-mistral2.2-7b
+      layer_range: [8, 32]
+merge_method: passthrough
+dtype: float16
+```
+
+
+Special thanks to Sushi.
+
+If you want to support me, you can [here](https://ko-fi.com/undiai).