Update README.md
Browse files
README.md
CHANGED
|
@@ -21,6 +21,12 @@ tags:
|
|
| 21 |
box-shadow: 0 4px 12px rgba(0,0,0,0.3);
|
| 22 |
border: 1px solid #3c3c3c;
|
| 23 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
.card-dark h1 {
|
| 25 |
font-size: 2.2em;
|
| 26 |
color: #ffffff;
|
|
@@ -70,11 +76,12 @@ tags:
|
|
| 70 |
padding: 3px 6px;
|
| 71 |
border-radius: 4px;
|
| 72 |
font-family: 'Fira Code', 'Courier New', monospace;
|
| 73 |
-
color: #
|
| 74 |
}
|
| 75 |
a {
|
| 76 |
color: #569cd6;
|
| 77 |
text-decoration: none;
|
|
|
|
| 78 |
}
|
| 79 |
a:hover {
|
| 80 |
text-decoration: underline;
|
|
@@ -89,7 +96,7 @@ tags:
|
|
| 89 |
|
| 90 |
<div class="container-dark">
|
| 91 |
|
| 92 |
-
<div class="card-dark">
|
| 93 |
<h1>EXL3 Quantization Requests</h1>
|
| 94 |
<p class="subtitle">Community hub for requesting EXL3 quants.</p>
|
| 95 |
</div>
|
|
@@ -139,11 +146,11 @@ tags:
|
|
| 139 |
<h2>How to Download and Use EXL Quants</h2>
|
| 140 |
<p>Each quantization size for a model is stored in a separate HF repository branch. You can download a specific quant size by its branch.</p>
|
| 141 |
<p>For example, to download the <code class="inline-code-dark">4.0bpw_H6</code> quant:</p>
|
| 142 |
-
<p>Install hugginface-cli:</p>
|
| 143 |
<pre><code>pip install -U "huggingface_hub[cli]"</code></pre>
|
| 144 |
-
<p>Download quant by targeting the specific quant size (revision):</p>
|
| 145 |
<pre><code>huggingface-cli download ArtusDev/MODEL_NAME --revision "4.0bpw_H6" --local-dir ./</code></pre>
|
| 146 |
-
<p
|
| 147 |
</div>
|
| 148 |
|
| 149 |
<div class="card-dark">
|
|
|
|
| 21 |
box-shadow: 0 4px 12px rgba(0,0,0,0.3);
|
| 22 |
border: 1px solid #3c3c3c;
|
| 23 |
}
|
| 24 |
+
.card-dark.card-dark-title h1 {
|
| 25 |
+
font-size: 1.5em;
|
| 26 |
+
color: #ffffff;
|
| 27 |
+
text-align: center;
|
| 28 |
+
margin-bottom: 10px;
|
| 29 |
+
}
|
| 30 |
.card-dark h1 {
|
| 31 |
font-size: 2.2em;
|
| 32 |
color: #ffffff;
|
|
|
|
| 76 |
padding: 3px 6px;
|
| 77 |
border-radius: 4px;
|
| 78 |
font-family: 'Fira Code', 'Courier New', monospace;
|
| 79 |
+
color: #c586c0;
|
| 80 |
}
|
| 81 |
a {
|
| 82 |
color: #569cd6;
|
| 83 |
text-decoration: none;
|
| 84 |
+
font-weight: 600;
|
| 85 |
}
|
| 86 |
a:hover {
|
| 87 |
text-decoration: underline;
|
|
|
|
| 96 |
|
| 97 |
<div class="container-dark">
|
| 98 |
|
| 99 |
+
<div class="card-dark card-dark-title">
|
| 100 |
<h1>EXL3 Quantization Requests</h1>
|
| 101 |
<p class="subtitle">Community hub for requesting EXL3 quants.</p>
|
| 102 |
</div>
|
|
|
|
| 146 |
<h2>How to Download and Use EXL Quants</h2>
|
| 147 |
<p>Each quantization size for a model is stored in a separate HF repository branch. You can download a specific quant size by its branch.</p>
|
| 148 |
<p>For example, to download the <code class="inline-code-dark">4.0bpw_H6</code> quant:</p>
|
| 149 |
+
<p><b>1. Install hugginface-cli:</b></p>
|
| 150 |
<pre><code>pip install -U "huggingface_hub[cli]"</code></pre>
|
| 151 |
+
<p><b>2. Download quant by targeting the specific quant size (revision):</b></p>
|
| 152 |
<pre><code>huggingface-cli download ArtusDev/MODEL_NAME --revision "4.0bpw_H6" --local-dir ./</code></pre>
|
| 153 |
+
<p>EXL3 quants can be run with any inference client that supports the EXL3 format, such as <a href="https://github.com/theroyallab/tabbyapi" target="_blank"><b>TabbyAPI</b></a>. Please refer to <a href="https://github.com/theroyallab/tabbyAPI/wiki/01.-Getting-Started" target="_blank">documentation</a> for set up instructions.</p>
|
| 154 |
</div>
|
| 155 |
|
| 156 |
<div class="card-dark">
|