Update README.md
Browse files
README.md
CHANGED
|
@@ -6,16 +6,37 @@ tags:
|
|
| 6 |
- text-generation-inference
|
| 7 |
datasets:
|
| 8 |
- s-nlp/ru_paradetox
|
|
|
|
|
|
|
| 9 |
---
|
| 10 |
-
This is the detoxification baseline model trained on the [train](https://github.com/skoltech-nlp/russe_detox_2022/blob/main/data/input/train.tsv) part of "RUSSE 2022: Russian Text Detoxification Based on Parallel Corpora" competition. The source sentences are Russian toxic messages from Odnoklassniki, Pikabu, and Twitter platforms. The base model is [ruT5](https://huggingface.co/
|
| 11 |
|
| 12 |
**How to use**
|
| 13 |
```python
|
| 14 |
from transformers import T5ForConditionalGeneration, AutoTokenizer
|
| 15 |
|
| 16 |
-
base_model_name = '
|
| 17 |
-
model_name = '
|
| 18 |
|
| 19 |
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
|
| 20 |
model = T5ForConditionalGeneration.from_pretrained(model_name)
|
| 21 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
- text-generation-inference
|
| 7 |
datasets:
|
| 8 |
- s-nlp/ru_paradetox
|
| 9 |
+
base_model:
|
| 10 |
+
- ai-forever/ruT5-base
|
| 11 |
---
|
| 12 |
+
This is the detoxification baseline model trained on the [train](https://github.com/skoltech-nlp/russe_detox_2022/blob/main/data/input/train.tsv) part of "RUSSE 2022: Russian Text Detoxification Based on Parallel Corpora" competition. The source sentences are Russian toxic messages from Odnoklassniki, Pikabu, and Twitter platforms. The base model is [ruT5](https://huggingface.co/ai-forever/ruT5-base).
|
| 13 |
|
| 14 |
**How to use**
|
| 15 |
```python
|
| 16 |
from transformers import T5ForConditionalGeneration, AutoTokenizer
|
| 17 |
|
| 18 |
+
base_model_name = 'ai-forever/ruT5-base'
|
| 19 |
+
model_name = 's-nlp/ruT5-base-detox'
|
| 20 |
|
| 21 |
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
|
| 22 |
model = T5ForConditionalGeneration.from_pretrained(model_name)
|
| 23 |
+
|
| 24 |
+
input_ids = tokenizer.encode('Это полная хуйня!', return_tensors='pt')
|
| 25 |
+
output_ids = model.generate(input_ids, max_length=50, num_return_sequences=1)
|
| 26 |
+
output_text = tokenizer.decode(output_ids[0], skip_special_tokens=True)
|
| 27 |
+
print(output_text)
|
| 28 |
+
# Это полный бред!
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
## Citation
|
| 32 |
+
|
| 33 |
+
```
|
| 34 |
+
@article{dementievarusse,
|
| 35 |
+
title={RUSSE-2022: Findings of the First Russian Detoxification Shared Task Based on Parallel Corpora},
|
| 36 |
+
author={Dementieva, Daryna and Logacheva, Varvara and Nikishina, Irina and Fenogenova, Alena and Dale, David and Krotova, Irina and Semenov, Nikita and Shavrina, Tatiana and Panchenko, Alexander}
|
| 37 |
+
}
|
| 38 |
+
```
|
| 39 |
+
|
| 40 |
+
**License**
|
| 41 |
+
|
| 42 |
+
This model is licensed under the OpenRAIL++ License, which supports the development of various technologies—both industrial and academic—that serve the public good.
|