Update usage with infinity
Browse files
README.md
CHANGED
|
@@ -1072,6 +1072,16 @@ library_name: sentence-transformers
|
|
| 1072 |
|
| 1073 |
Please refer to our [technical report](https://arxiv.org/abs/2408.15710).
|
| 1074 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1075 |
## Citation
|
| 1076 |
|
| 1077 |
If you find our models / papers useful in your research, please consider giving ❤️ and citations. Thanks!
|
|
|
|
| 1072 |
|
| 1073 |
Please refer to our [technical report](https://arxiv.org/abs/2408.15710).
|
| 1074 |
|
| 1075 |
+
## Local usage with Infinity
|
| 1076 |
+
|
| 1077 |
+
OpenAI compatible API deployment with [Infinity](https://github.com/michaelfeil/infinity).
|
| 1078 |
+
|
| 1079 |
+
```bash
|
| 1080 |
+
docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
|
| 1081 |
+
michaelf34/infinity:0.0.70 \
|
| 1082 |
+
v2 --model-id TencentBAC/Conan-embedding-v1 --dtype float16 --batch-size 32 --engine torch --port 7997
|
| 1083 |
+
```
|
| 1084 |
+
|
| 1085 |
## Citation
|
| 1086 |
|
| 1087 |
If you find our models / papers useful in your research, please consider giving ❤️ and citations. Thanks!
|