--- base_model: minishlab/potion-base-2m datasets: - Intel/polite-guard library_name: model2vec license: mit model_name: enguard/tiny-guard-2m-en-general-politeness-binary-intel tags: - static-embeddings - text-classification - model2vec --- # enguard/tiny-guard-2m-en-general-politeness-binary-intel This model is a fine-tuned Model2Vec classifier based on [minishlab/potion-base-2m](https://huggingface.co/minishlab/potion-base-2m) for the general-politeness-binary found in the [Intel/polite-guard](https://huggingface.co/datasets/Intel/polite-guard) dataset. ## Installation ```bash pip install model2vec[inference] ``` ## Usage ```python from model2vec.inference import StaticModelPipeline model = StaticModelPipeline.from_pretrained( "enguard/tiny-guard-2m-en-general-politeness-binary-intel" ) # Supports single texts. Format input as a single text: text = "Example sentence" model.predict([text]) model.predict_proba([text]) ``` ## Why should you use these models? - Optimized for precision to reduce false positives. - Extremely fast inference: up to x500 faster than SetFit. ## This model variant Below is a quick overview of the model variant and core metrics. | Field | Value | |---|---| | Classifies | general-politeness-binary | | Base Model | [minishlab/potion-base-2m](https://huggingface.co/minishlab/potion-base-2m) | | Precision | 0.9843 | | Recall | 0.9889 | | F1 | 0.9866 | ### Confusion Matrix | True \ Predicted | FAIL | PASS | | --- | --- | --- | | **FAIL** | 2504 | 28 | | **PASS** | 40 | 7628 |
Full metrics (JSON) ```json { "FAIL": { "precision": 0.9842767295597484, "recall": 0.9889415481832543, "f1-score": 0.9866036249014972, "support": 2532.0 }, "PASS": { "precision": 0.9963427377220481, "recall": 0.9947835159102765, "f1-score": 0.9955625163142783, "support": 7668.0 }, "accuracy": 0.9933333333333333, "macro avg": { "precision": 0.9903097336408982, "recall": 0.9918625320467653, "f1-score": 0.9910830706078877, "support": 10200.0 }, "weighted avg": { "precision": 0.9933475286370538, "recall": 0.9933333333333333, "f1-score": 0.9933386032694584, "support": 10200.0 } } ```
Sample Predictions | Text | True Label | Predicted Label | |------|------------|-----------------| | I appreciate your interest in our vegetarian options. I can provide you with a list of our current dishes that cater to your dietary preferences. | PASS | PASS | | I understand you're concerned about the ski lessons, and I'll look into the options for rescheduling. | PASS | PASS | | Our technical skills course will cover the essential topics in data analysis, including data visualization and statistical modeling. The course materials will be available on our learning platform. | PASS | PASS | | Our buffet hours are from 11 AM to 9 PM. Please note that we have a limited selection of options available during the lunch break. | PASS | PASS | | I'll look into your policy details and see what options are available to you. | PASS | PASS | | I appreciate your interest in our vegetarian options. I can provide you with a list of our current dishes that cater to your dietary preferences. | PASS | PASS |
Prediction Speed Benchmarks | Dataset Size | Time (seconds) | Predictions/Second | |--------------|----------------|---------------------| | 1 | 0.0002 | 5108.77 | | 1000 | 0.0542 | 18439.74 | | 10000 | 0.6208 | 16108.79 |
## Other model variants Below is a general overview of the best-performing models for each dataset variant. | Classifies | Model | Precision | Recall | F1 | | --- | --- | --- | --- | --- | | general-politeness-binary | [enguard/tiny-guard-2m-en-general-politeness-binary-intel](https://huggingface.co/enguard/tiny-guard-2m-en-general-politeness-binary-intel) | 0.9843 | 0.9889 | 0.9866 | | general-politeness-multiclass | [enguard/tiny-guard-2m-en-general-politeness-multiclass-intel](https://huggingface.co/enguard/tiny-guard-2m-en-general-politeness-multiclass-intel) | 0.9875 | 0.9704 | 0.9789 | | general-politeness-binary | [enguard/tiny-guard-4m-en-general-politeness-binary-intel](https://huggingface.co/enguard/tiny-guard-4m-en-general-politeness-binary-intel) | 0.9831 | 0.9878 | 0.9854 | | general-politeness-multiclass | [enguard/tiny-guard-4m-en-general-politeness-multiclass-intel](https://huggingface.co/enguard/tiny-guard-4m-en-general-politeness-multiclass-intel) | 0.9896 | 0.9783 | 0.9839 | | general-politeness-binary | [enguard/tiny-guard-8m-en-general-politeness-binary-intel](https://huggingface.co/enguard/tiny-guard-8m-en-general-politeness-binary-intel) | 0.9828 | 0.9905 | 0.9866 | | general-politeness-multiclass | [enguard/tiny-guard-8m-en-general-politeness-multiclass-intel](https://huggingface.co/enguard/tiny-guard-8m-en-general-politeness-multiclass-intel) | 0.9873 | 0.9795 | 0.9833 | | general-politeness-binary | [enguard/small-guard-32m-en-general-politeness-binary-intel](https://huggingface.co/enguard/small-guard-32m-en-general-politeness-binary-intel) | 0.9858 | 0.9889 | 0.9874 | | general-politeness-multiclass | [enguard/small-guard-32m-en-general-politeness-multiclass-intel](https://huggingface.co/enguard/small-guard-32m-en-general-politeness-multiclass-intel) | 0.9897 | 0.9862 | 0.9879 | | general-politeness-binary | [enguard/medium-guard-128m-xx-general-politeness-binary-intel](https://huggingface.co/enguard/medium-guard-128m-xx-general-politeness-binary-intel) | 0.9831 | 0.9901 | 0.9866 | | general-politeness-multiclass | [enguard/medium-guard-128m-xx-general-politeness-multiclass-intel](https://huggingface.co/enguard/medium-guard-128m-xx-general-politeness-multiclass-intel) | 0.9881 | 0.9870 | 0.9876 | ## Resources - Awesome AI Guardrails: - Model2Vec: https://github.com/MinishLab/model2vec - Docs: https://minish.ai/packages/model2vec/introduction ## Citation If you use this model, please cite Model2Vec: ``` @software{minishlab2024model2vec, author = {Stephan Tulkens and {van Dongen}, Thomas}, title = {Model2Vec: Fast State-of-the-Art Static Embeddings}, year = {2024}, publisher = {Zenodo}, doi = {10.5281/zenodo.17270888}, url = {https://github.com/MinishLab/model2vec}, license = {MIT} } ```