update code snippet
Browse files
README.md
CHANGED
|
@@ -10,6 +10,7 @@ tags:
|
|
| 10 |
- text-classification
|
| 11 |
- neural-compressor
|
| 12 |
- int8
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
# Dynamically quantized DistilBERT base uncased finetuned SST-2
|
|
@@ -27,14 +28,17 @@ tags:
|
|
| 27 |
|
| 28 |
## How to Get Started With the Model
|
| 29 |
|
|
|
|
|
|
|
|
|
|
| 30 |
To load the quantized model and run inference using the Transformers [pipelines](https://huggingface.co/docs/transformers/main/en/main_classes/pipelines), you can do as follows:
|
| 31 |
|
| 32 |
```python
|
| 33 |
from transformers import AutoTokenizer, pipeline
|
| 34 |
-
from optimum.intel
|
| 35 |
|
| 36 |
model_id = "echarlaix/distilbert-base-uncased-finetuned-sst-2-english-int8-dynamic"
|
| 37 |
-
model =
|
| 38 |
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
| 39 |
cls_pipe = pipeline("text-classification", model=model, tokenizer=tokenizer)
|
| 40 |
text = "He's a dreadful magician."
|
|
|
|
| 10 |
- text-classification
|
| 11 |
- neural-compressor
|
| 12 |
- int8
|
| 13 |
+
- 8-bit
|
| 14 |
---
|
| 15 |
|
| 16 |
# Dynamically quantized DistilBERT base uncased finetuned SST-2
|
|
|
|
| 28 |
|
| 29 |
## How to Get Started With the Model
|
| 30 |
|
| 31 |
+
This requires to install Optimum :
|
| 32 |
+
`pip install optimum[neural-compressor]`
|
| 33 |
+
|
| 34 |
To load the quantized model and run inference using the Transformers [pipelines](https://huggingface.co/docs/transformers/main/en/main_classes/pipelines), you can do as follows:
|
| 35 |
|
| 36 |
```python
|
| 37 |
from transformers import AutoTokenizer, pipeline
|
| 38 |
+
from optimum.intel import INCModelForSequenceClassification
|
| 39 |
|
| 40 |
model_id = "echarlaix/distilbert-base-uncased-finetuned-sst-2-english-int8-dynamic"
|
| 41 |
+
model = INCModelForSequenceClassification.from_pretrained(model_id)
|
| 42 |
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
| 43 |
cls_pipe = pipeline("text-classification", model=model, tokenizer=tokenizer)
|
| 44 |
text = "He's a dreadful magician."
|