Hugging Face's logo

ArtusDev
/
swiss-ai_Apertus-70B-Instruct-2509-EXL3

Text Generation
Transformers
multilingual
compliant
swiss-ai
apertus
exl3
Model card Files
xet
Community
ArtusDev commited on
Commit
707cf41
·
verified ·
1 Parent(s): e872b91

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +230 -28
README.md CHANGED
@@ -12,37 +12,239 @@ tags:
12
  - apertus
13
  - exl3
14
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
- ## EXL3 Quants of swiss-ai/Apertus-70B-Instruct-2509
17
 
18
- EXL3 quants of [swiss-ai/Apertus-70B-Instruct-2509](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509) using <a href="https://github.com/turboderp-org/exllamav3/">exllamav3</a> for quantization.
 
 
 
 
 
19
 
20
- ### Quants
21
- | Quant(Revision) | Bits per Weight | Head Bits |
22
- | -------- | ---------- | --------- |
23
- | [2.5_H6](https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/2.5bpw_H6) | 2.5 | 6 |
24
- | [3.0_H6](https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/3.0bpw_H6) | 3.0 | 6 |
25
- | [3.5_H6](https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/3.5bpw_H6) | 3.5 | 6 |
26
- | [4.0_H6](https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/4.0bpw_H6) | 4.0 | 6 |
27
- | [4.25_H6](https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/4.25bpw_H6) | 4.25 | 6 |
28
- | [5.0_H6](https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/5.0bpw_H6) | 5.0 | 6 |
29
- | [6.0_H6](https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/6.0bpw_H6) | 6.0 | 6 |
30
- | [8.0_H8](https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/8.0bpw_H8) | 8.0 | 8 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
 
32
- ### Downloading quants with huggingface-cli
 
 
 
 
 
 
 
 
 
 
 
 
 
33
 
34
- <details>
35
- <summary>Click to view download instructions</summary>
 
 
 
 
 
 
 
36
 
37
- Install hugginface-cli:
38
-
39
- ```bash
40
- pip install -U "huggingface_hub[cli]"
41
- ```
42
-
43
- Download quant by targeting the specific quant revision (branch):
44
-
45
- ```
46
- huggingface-cli download ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3 --revision "5.0bpw_H6" --local-dir ./
47
- ```
48
- </details>
 
12
  - apertus
13
  - exl3
14
  ---
15
+ <style>
16
+ .container-dark {
17
+ font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
18
+ line-height: 1.6;
19
+ color: #d4d4d4;
20
+ }
21
+ a {
22
+ color: #569cd6;
23
+ text-decoration: none;
24
+ font-weight: 600;
25
+ }
26
+ a:hover {
27
+ text-decoration: underline;
28
+ }
29
+ .card-dark {
30
+ background-color: #252526;
31
+ border-radius: 12px;
32
+ padding: 24px;
33
+ margin-bottom: 20px;
34
+ box-shadow: 0 4px 12px rgba(0,0,0,0.3);
35
+ border: 1px solid #3c3c3c;
36
+ }
37
+ .card-dark h1 {
38
+ font-size: 2.2em;
39
+ color: #ffffff;
40
+ text-align: center;
41
+ margin-bottom: 10px;
42
+ }
43
+ .card-dark.card-dark-title h1 {
44
+ font-size: 1.5em;
45
+ }
46
+ .card-dark .subtitle {
47
+ text-align: center;
48
+ font-size: 1.1em;
49
+ color: #a0a0a0;
50
+ }
51
+ .card-dark h2 {
52
+ font-size: 1.5em;
53
+ margin-top: 0;
54
+ padding-bottom: 10px;
55
+ border-bottom: 1px solid #3c3c3c;
56
+ color: #c586c0;
57
+ }
58
+ .card-dark h3 {
59
+ font-size: 1.2em;
60
+ color: #d4d4d4;
61
+ }
62
+ .styled-table {
63
+ display: table;
64
+ border: none;
65
+ width: 100%;
66
+ font-size: 0.95em;
67
+ margin-bottom: 0px;
68
+ }
69
+ .styled-table thead th {
70
+ background-color: #333333;
71
+ color: #c586c0;
72
+ text-align: left;
73
+ }
74
+ .styled-table th {
75
+ padding: 12px 15px;
76
+ }
77
+ .styled-table td {
78
+ padding: 0;
79
+ }
80
+ .styled-table table, .styled-table th, .styled-table td {
81
+ border-left: none;
82
+ border-right: none;
83
+ border-bottom: none;
84
+ }
85
+ .styled-table td {
86
+ border-bottom: 1px solid #3c3c3c;
87
+ }
88
+ .styled-table tbody tr {
89
+ transition: background-color 0.1s ease;
90
+ }
91
+ .styled-table tbody tr:hover {
92
+ background-color: #3a3a3a;
93
+ }
94
+ .styled-table tr:last-child td {
95
+ border-bottom: none;
96
+ }
97
+ .styled-table td a {
98
+ display: block;
99
+ padding: 12px 15px;
100
+ }
101
+ .styled-table td a.fake-link {
102
+ text-decoration:none;
103
+ color:inherit;
104
+ }
105
+ details {
106
+ margin-top: 20px;
107
+ border: 1px solid #3c3c3c;
108
+ border-radius: 8px;
109
+ overflow: hidden;
110
+ }
111
+ summary {
112
+ cursor: pointer;
113
+ padding: 12px 18px;
114
+ background-color: #6A5ACD;
115
+ font-weight: 600;
116
+ display: flex;
117
+ align-items: center;
118
+ gap: 10px;
119
+ justify-content: space-between;
120
+ list-style: none;
121
+ }
122
+ summary::-webkit-details-marker {
123
+ display: none;
124
+ }
125
+ summary:hover {
126
+ filter: brightness(1.1);
127
+ }
128
+ summary::after {
129
+ content: '';
130
+ display: inline-block;
131
+ width: 8px;
132
+ height: 8px;
133
+ border-bottom: 2px solid white;
134
+ border-right: 2px solid white;
135
+ transform: rotate(45deg);
136
+ transition: transform 0.3s ease;
137
+ }
138
+ details[open] > summary::after {
139
+ transform: rotate(225deg);
140
+ }
141
+ .details-content {
142
+ padding: 18px;
143
+ }
144
+ .btn-purple {
145
+ display: inline-block;
146
+ background-color: #6A5ACD;
147
+ color: white !important;
148
+ padding: 12px 24px;
149
+ border-radius: 8px;
150
+ text-decoration: none;
151
+ font-weight: 600;
152
+ transition: background-color 0.3s ease, transform 0.2s ease;
153
+ text-align: center;
154
+ }
155
+ .btn-purple:hover {
156
+ background-color: #7B68EE;
157
+ transform: translateY(-2px);
158
+ }
159
+ </style>
160
 
161
+ <div class="container-dark">
162
 
163
+ <div class="card-dark card-dark-title">
164
+ <h1>ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3</h1>
165
+ <p class="subtitle">
166
+ EXL3 quants of <a href="https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509" target="_blank">swiss-ai/Apertus-70B-Instruct-2509</a> using <a href="https://github.com/turboderp-org/exllamav3/" target="_blank">exllamav3</a> for quantization.
167
+ </p>
168
+ </div>
169
 
170
+ <div class="card-dark">
171
+ <h2>Quants</h2>
172
+ <table class="styled-table">
173
+ <thead>
174
+ <tr>
175
+ <th>Quant</th>
176
+ <th>BPW</th>
177
+ <th>Head Bits</th>
178
+ </tr>
179
+ </thead>
180
+ <tbody>
181
+ <tr>
182
+ <td><a href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/2.5bpw_H6" target="_blank">2.5_H6</a></td>
183
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/2.5bpw_H6" target="_blank">2.5</a></td>
184
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/2.5bpw_H6" target="_blank">6</a></td>
185
+ </tr>
186
+ <tr>
187
+ <td><a href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/3.0bpw_H6" target="_blank">3.0_H6</a></td>
188
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/3.0bpw_H6" target="_blank">3.0</a></td>
189
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/3.0bpw_H6" target="_blank">6</a></td>
190
+ </tr>
191
+ <tr>
192
+ <td><a href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/3.5bpw_H6" target="_blank">3.5_H6</a></td>
193
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/3.5bpw_H6" target="_blank">3.5</a></td>
194
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/3.5bpw_H6" target="_blank">6</a></td>
195
+ </tr>
196
+ <tr>
197
+ <td><a href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/4.0bpw_H6" target="_blank">4.0_H6</a></td>
198
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/4.0bpw_H6" target="_blank">4.0</a></td>
199
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/4.0bpw_H6" target="_blank">6</a></td>
200
+ </tr>
201
+ <tr>
202
+ <td><a href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/4.25bpw_H6" target="_blank">4.25_H6</a></td>
203
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/4.25bpw_H6" target="_blank">4.25</a></td>
204
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/4.25bpw_H6" target="_blank">6</a></td>
205
+ </tr>
206
+ <tr>
207
+ <td><a href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/5.0bpw_H6" target="_blank">5.0_H6</a></td>
208
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/5.0bpw_H6" target="_blank">5.0</a></td>
209
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/5.0bpw_H6" target="_blank">6</a></td>
210
+ </tr>
211
+ <tr>
212
+ <td><a href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/6.0bpw_H6" target="_blank">6.0_H6</a></td>
213
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/6.0bpw_H6" target="_blank">6.0</a></td>
214
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/6.0bpw_H6" target="_blank">6</a></td>
215
+ </tr>
216
+ <tr>
217
+ <td><a href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/8.0bpw_H8" target="_blank">8.0_H8</a></td>
218
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/8.0bpw_H8" target="_blank">8.0</a></td>
219
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3/tree/8.0bpw_H8" target="_blank">8</a></td>
220
+ </tr>
221
+ </tbody>
222
+ </table>
223
+ </div>
224
 
225
+ <div class="card-dark">
226
+ <h2>How to Download and Use Quants</h2>
227
+ <p>You can download quants by targeting specific size using the Hugging Face CLI.</p>
228
+ <details>
229
+ <summary>Click for download commands</summary>
230
+ <div class="details-content">
231
+ <b>1. Install huggingface-cli:</b>
232
+ <pre><code>pip install -U "huggingface_hub[cli]"</code></pre>
233
+ <b>2. Download a specific quant:</b>
234
+ <pre><code>huggingface-cli download ArtusDev/swiss-ai_Apertus-70B-Instruct-2509-EXL3 --revision "5.0bpw_H6" --local-dir ./</code></pre>
235
+ </div>
236
+ </details>
237
+ <p>EXL3 quants can be run with any inference client that supports EXL3, such as <a href="https://github.com/theroyallab/tabbyapi" target="_blank"><b>TabbyAPI</b></a>. Refer to <a href="https://github.com/theroyallab/tabbyAPI/wiki/01.-Getting-Started" target="_blank">documentation</a> for set up instructions.</p>
238
+ </div>
239
 
240
+ <div class="card-dark">
241
+ <h2>Quant Requests</h2>
242
+ <div style="text-align: center; margin-top: 25px;">
243
+ <a href="https://huggingface.co/ArtusDev/requests-exl/discussions/new?title=[MODEL_NAME_HERE]&description=[MODEL_HF_LINK_HERE]" class="btn-purple" target="_blank">Request EXL3 Quants</a>
244
+ </div>
245
+ <p class="subtitle">
246
+ See <a href="https://huggingface.co/ArtusDev/requests-exl" target="_blank">EXL community hub</a> for request guidelines.
247
+ </p>
248
+ </div>
249
 
250
+ </div>