Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model:
|
| 4 |
+
- Nanbeige/Nanbeige4.1-3B
|
| 5 |
+
tags:
|
| 6 |
+
- llm-compressor
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
<div align="center">
|
| 10 |
+
<img
|
| 11 |
+
src="https://cdn-uploads.huggingface.co/production/uploads/64b93e6bd6c468ac7536607e/mj6xac74jHGLqymiovObc.png"
|
| 12 |
+
alt="The Kaitchup -- AI on a Budget"
|
| 13 |
+
style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;"
|
| 14 |
+
/>
|
| 15 |
+
<div style="display: flex; justify-content: center; gap: 0.5em; margin-bottom: 1em;">
|
| 16 |
+
<a href="https://kaitchup.substack.com/subscribe"><strong>Subscribe and Support</strong></a>
|
| 17 |
+
</div>
|
| 18 |
+
</div>
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
This is [Nanbeige/Nanbeige4.1-3B](https://huggingface.co/Nanbeige/Nanbeige4.1-3B) quantized with [llm-compressor](https://github.com/vllm-project/llm-compressor) to W8A8 (FP8) . The model is compatible with vLLM (tested: v0.15.1). Tested with an L4 (Google Colab).
|
| 22 |
+
|
| 23 |
+
- **Developed by:** [The Kaitchup](https://kaitchup.substack.com/)
|