Update README.md
Browse files
README.md
CHANGED
|
@@ -18,10 +18,13 @@ I have no idea how well it works on this one, running metrics on this will take
|
|
| 18 |
If you "like" the model, I'll keep it in the collection.
|
| 19 |
|
| 20 |
```bash
|
| 21 |
-
Perplexity
|
| 22 |
-
|
|
|
|
| 23 |
```
|
| 24 |
|
|
|
|
|
|
|
| 25 |
-G
|
| 26 |
|
| 27 |
This model [KAT-Dev-72B-Exp-qx86-hi-mlx](https://huggingface.co/KAT-Dev-72B-Exp-qx86-hi-mlx) was
|
|
|
|
| 18 |
If you "like" the model, I'll keep it in the collection.
|
| 19 |
|
| 20 |
```bash
|
| 21 |
+
Model Perplexity Peak memory
|
| 22 |
+
qx86-hi-mlx 4.333 ± 0.031 71.47 GB
|
| 23 |
+
q8-hi 4.334 ± 0.031 86.59 GB
|
| 24 |
```
|
| 25 |
|
| 26 |
+
The mixed precision model is slightly outperforming q8. This is expected
|
| 27 |
+
|
| 28 |
-G
|
| 29 |
|
| 30 |
This model [KAT-Dev-72B-Exp-qx86-hi-mlx](https://huggingface.co/KAT-Dev-72B-Exp-qx86-hi-mlx) was
|