Update README.md
Browse files
README.md
CHANGED
|
@@ -9,7 +9,12 @@ pipeline_tag: text-generation
|
|
| 9 |
## Mixtral-8x7B-v0.1-hf-2bit_g16_s128-HQQ
|
| 10 |
This is a version of the Mixtral-8x7B-v0.1 model (https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) quantized to 2-bit via Half-Quadratic Quantization (HQQ).
|
| 11 |
### Basic Usage
|
| 12 |
-
To run the model, install the HQQ library
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
``` Python
|
| 14 |
model_id = 'mobiuslabsgmbh/Mixtral-8x7B-v0.1-hf-2bit_g16_s128-HQQ'
|
| 15 |
|
|
|
|
| 9 |
## Mixtral-8x7B-v0.1-hf-2bit_g16_s128-HQQ
|
| 10 |
This is a version of the Mixtral-8x7B-v0.1 model (https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) quantized to 2-bit via Half-Quadratic Quantization (HQQ).
|
| 11 |
### Basic Usage
|
| 12 |
+
To run the model, install the HQQ library:
|
| 13 |
+
```
|
| 14 |
+
pip install hqq==0.1.8
|
| 15 |
+
pip install transformers==4.46.0
|
| 16 |
+
```
|
| 17 |
+
and use it as follows:
|
| 18 |
``` Python
|
| 19 |
model_id = 'mobiuslabsgmbh/Mixtral-8x7B-v0.1-hf-2bit_g16_s128-HQQ'
|
| 20 |
|