Update README.md

README.md

@@ -1,3 +1,52 @@
 ---
 license: gpl-3.0
 ---
+
+# WizardLM 13B GGML
+
+This is https://huggingface.co/winddude/wizardLM-LlaMA-LoRA-13 converted into GGML format, supported by [llama.cpp](https://github.com/ggerganov/llama.cpp).
+
+If used in an instruct mode this model is very picky about the newlines. For some reason it insists on having the following format:
+
+```
+### Instruction:
+
+(instruction text here)
+
+### Response:
+
+(leave this for the model to fill)
+```
+
+Note that generally the alpacas only have one newline before '### Instruction:' / '### Response' and the text. I might have somehow messed up conversion :(
+
+
+# Instructions for running this model in https://github.com/oobabooga/text-generation-webui:
+
+1. Make sure the pytorch and llama.cpp are recent enough:
+
+```
+pip install -U --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html
+pip install -U llamacpp
+```
+
+2. Move the model to the right directory:
+
+```
+mkdir models/wizardlm-13b/
+mv ~/Downloads/ggml-model-q5_1.bin models/wizardlm-13b/
+```
+
+2. Run the textgen, e.g. with:
+
+```
+python server.py --cpu --threads 8 --chat --model=wizardlm-13b --model_type=llama  --verbose
+```
+
+3. If using it in the instruction mode, you should also fix the prompt. Select the Alpaca template as a base, then go to 'Character' tab and modify Turn template with the following string: `<|user|>\n\n<|user-message|>\n\n<|bot|>\n\n<|bot-message|>\n\n`.
+
+4. Sometimes the model just won't shut up, so go to Parameters tab and add the following 'Custom stopping string': `"### Instruction:"`
+
+5. The 3) and 4) could probably be fixed by modifying settings.json
+
+Please share other tips and tricks in https://huggingface.co/execveat/wizardLM-13b-ggml-q5_1/discussions