v2: new checkpoint + updated metrics + changelog

| Date (UTC) | Version | Notes |
| ---------- | ------- | ------------------------------------------------- |
| 2025-07-18 | **v2** | new checkpoint ➜ test acc 0.9986, macro F1 0.9987 |

CHANGELOG.md

@@ -0,0 +1,9 @@
+# Changelog
+
+## v2 – 2025-07-18
+* Added 20-epoch fine-tuned checkpoint
+* Test accuracy ↑ 0.9986, macro F1 ↑ 0.9987
+* Refreshed tokenizer, model card, metrics
+
+## v1 – 2024-11-10
+* Initial public release

README.md

@@ -1,10 +1,73 @@
 ---
-license: mit
-datasets:
-- malakhovks/MeDeBERTa
-language:
-- en
-base_model:
-- microsoft/deberta-v3-small
-- microsoft/deberta-v3-xsmall
----
+license: apache-2.0
+language: en
+library_name: transformers
+tags:
+  - deberta
+  - sequence-classification
+  - medicine
+  - telerehabilitation
+metrics:
+  - name: accuracy
+    type: accuracy
+    value: 0.9986   # test_accuracy
+  - name: macro_f1
+    type: f1
+    value: 0.9987
+  - name: balanced_accuracy
+    type: balanced_accuracy
+    value: 0.9987
+  - name: auc_micro
+    type: auc
+    value: 0.999997
+  - name: ap_micro
+    type: average_precision
+    value: 0.99993
+---
+
+# **MeDeBERTa** – v2 (July 2025)
+
+Fine-tuned **microsoft/deberta-v3-xsmall** on 269 874 Q-A pairs (30 intent labels) for the *MeDeBERTa* telerehabilitation question-classification task.
+
+|                                    | Value |
+|------------------------------------|-------|
+| **Epochs**                         | 20 (best @ epoch 17) |
+| **Batch / Grad. Accum.**           | 16 / 4 (eff. 64) |
+| **Learning rate**                  | 5 × 10⁻⁵ |
+| **Best val. accuracy**        | **0.99855** |
+| **Test accuracy**                  | **0.99859** |
+| **Macro F1 (test)**           | **0.99867** |
+| **Balanced accuracy (test)**       | 0.99868 |
+| **Micro AUC**                      | 0.999997 |
+| **Micro average precision**        | 0.99993 |
+| **Loss (val | test)**              | 0.01371 \| 0.01305 |
+| **Hardware**                       | RTX 2080 Ti (11 GB) |
+
+<details>
+<summary>Per-class metrics (excerpt)</summary>
+
+| Label | Precision | Recall | F1 | Support |
+|-------|-----------|--------|----|---------|
+| any_code | 1.000 | 1.000 | 1.000 | 980 |
+| contexts | 0.988 | 0.987 | 0.988 | 923 |
+| treatment summary | 1.000 | 0.998 | 0.999 | 927 |
+| … | … | … | … | … |
+
+Full table: see `classification_report.json` / `classification_report.csv`.
+</details>
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
+tok = AutoTokenizer.from_pretrained("malakhovks/MeDeBERTa")
+model = AutoModelForSequenceClassification.from_pretrained("malakhovks/MeDeBERTa")
+
+inputs = tok("what are contraindications for TENS?", return_tensors="pt")
+pred   = model(**inputs).logits.argmax(-1).item()
+print(model.config.id2label[pred])
+```
+
+## Changelog
+See [CHANGELOG.md](./CHANGELOG.md) for full version history.

classification_report.csv

@@ -0,0 +1,35 @@
+,precision,recall,f1-score,support
+any_code,1.0000,1.0000,1.0000,980.0000
+any_icd,1.0000,1.0000,1.0000,950.0000
+articles,1.0000,1.0000,1.0000,1022.0000
+causes,1.0000,0.9990,0.9995,961.0000
+certain cpt,1.0000,1.0000,1.0000,239.0000
+certain g-code,1.0000,1.0000,1.0000,231.0000
+certain hcpcs,1.0000,1.0000,1.0000,56.0000
+certain icd 10,1.0000,1.0000,1.0000,918.0000
+certain icd 9,1.0000,1.0000,1.0000,884.0000
+contexts,0.9881,0.9870,0.9875,923.0000
+contraindications precautions,1.0000,0.9990,0.9995,965.0000
+cpt,1.0000,0.9990,0.9995,977.0000
+description,0.9864,0.9931,0.9897,875.0000
+diagnosis need for treatment,1.0000,0.9989,0.9995,940.0000
+g-code,0.9968,1.0000,0.9984,939.0000
+hcpcs,1.0000,0.9989,0.9995,947.0000
+icd 10,1.0000,1.0000,1.0000,907.0000
+icd 9,1.0000,0.9990,0.9995,954.0000
+indicatons,0.9978,0.9946,0.9962,921.0000
+pathogenesis,0.9979,0.9990,0.9985,970.0000
+patient education,1.0000,1.0000,1.0000,961.0000
+prognosis,0.9979,1.0000,0.9990,954.0000
+references,1.0000,1.0000,1.0000,967.0000
+reimbursement,0.9990,0.9990,0.9990,968.0000
+relations,0.9968,0.9968,0.9968,952.0000
+risk factors,1.0000,1.0000,1.0000,974.0000
+rule out,1.0000,1.0000,1.0000,867.0000
+symptoms,1.0000,0.9979,0.9990,975.0000
+synonyms,0.9990,1.0000,0.9995,954.0000
+test,0.9989,1.0000,0.9995,930.0000
+treatment summary,1.0000,0.9978,0.9989,927.0000
+accuracy,0.9986,0.9986,0.9986,0.9986
+macro avg,0.9987,0.9987,0.9987,26988.0000
+weighted avg,0.9986,0.9986,0.9986,26988.0000

classification_report.json

@@ -0,0 +1,201 @@
+{
+  "any_code": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 980.0
+  },
+  "any_icd": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 950.0
+  },
+  "articles": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 1022.0
+  },
+  "causes": {
+    "precision": 1.0,
+    "recall": 0.9989594172736732,
+    "f1-score": 0.9994794377928162,
+    "support": 961.0
+  },
+  "certain cpt": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 239.0
+  },
+  "certain g-code": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 231.0
+  },
+  "certain hcpcs": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 56.0
+  },
+  "certain icd 10": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 918.0
+  },
+  "certain icd 9": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 884.0
+  },
+  "contexts": {
+    "precision": 0.9880694143167028,
+    "recall": 0.9869989165763814,
+    "f1-score": 0.9875338753387534,
+    "support": 923.0
+  },
+  "contraindications precautions": {
+    "precision": 1.0,
+    "recall": 0.9989637305699481,
+    "f1-score": 0.9994815966822188,
+    "support": 965.0
+  },
+  "cpt": {
+    "precision": 1.0,
+    "recall": 0.9989764585465711,
+    "f1-score": 0.9994879672299027,
+    "support": 977.0
+  },
+  "description": {
+    "precision": 0.9863791146424518,
+    "recall": 0.9931428571428571,
+    "f1-score": 0.989749430523918,
+    "support": 875.0
+  },
+  "diagnosis need for treatment": {
+    "precision": 1.0,
+    "recall": 0.9989361702127659,
+    "f1-score": 0.9994678020223523,
+    "support": 940.0
+  },
+  "g-code": {
+    "precision": 0.9968152866242038,
+    "recall": 1.0,
+    "f1-score": 0.9984051036682615,
+    "support": 939.0
+  },
+  "hcpcs": {
+    "precision": 1.0,
+    "recall": 0.9989440337909187,
+    "f1-score": 0.9994717379820391,
+    "support": 947.0
+  },
+  "icd 10": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 907.0
+  },
+  "icd 9": {
+    "precision": 1.0,
+    "recall": 0.9989517819706499,
+    "f1-score": 0.9994756161510225,
+    "support": 954.0
+  },
+  "indicatons": {
+    "precision": 0.9978213507625272,
+    "recall": 0.99457111834962,
+    "f1-score": 0.9961935834692768,
+    "support": 921.0
+  },
+  "pathogenesis": {
+    "precision": 0.9979402677651905,
+    "recall": 0.9989690721649485,
+    "f1-score": 0.9984544049459042,
+    "support": 970.0
+  },
+  "patient education": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 961.0
+  },
+  "prognosis": {
+    "precision": 0.997907949790795,
+    "recall": 1.0,
+    "f1-score": 0.9989528795811519,
+    "support": 954.0
+  },
+  "references": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 967.0
+  },
+  "reimbursement": {
+    "precision": 0.9989669421487604,
+    "recall": 0.9989669421487604,
+    "f1-score": 0.9989669421487604,
+    "support": 968.0
+  },
+  "relations": {
+    "precision": 0.9968487394957983,
+    "recall": 0.9968487394957983,
+    "f1-score": 0.9968487394957983,
+    "support": 952.0
+  },
+  "risk factors": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 974.0
+  },
+  "rule out": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1-score": 1.0,
+    "support": 867.0
+  },
+  "symptoms": {
+    "precision": 1.0,
+    "recall": 0.997948717948718,
+    "f1-score": 0.9989733059548255,
+    "support": 975.0
+  },
+  "synonyms": {
+    "precision": 0.9989528795811519,
+    "recall": 1.0,
+    "f1-score": 0.9994761655316919,
+    "support": 954.0
+  },
+  "test": {
+    "precision": 0.9989258861439313,
+    "recall": 1.0,
+    "f1-score": 0.999462654486835,
+    "support": 930.0
+  },
+  "treatment summary": {
+    "precision": 1.0,
+    "recall": 0.9978425026968716,
+    "f1-score": 0.9989200863930886,
+    "support": 927.0
+  },
+  "accuracy": 0.9985919668000592,
+  "macro avg": {
+    "precision": 0.9986654139119842,
+    "recall": 0.9986780793189832,
+    "f1-score": 0.998671010625762,
+    "support": 26988.0
+  },
+  "weighted avg": {
+    "precision": 0.9985949747289834,
+    "recall": 0.9985919668000592,
+    "f1-score": 0.9985927033253673,
+    "support": 26988.0
+  }
+}

config.json

@@ -5,7 +5,7 @@
   "attention_probs_dropout_prob": 0.1,
   "hidden_act": "gelu",
   "hidden_dropout_prob": 0.1,
-  "hidden_size": 768,
+  "hidden_size": 384,
   "id2label": {
     "0": "any_code",
     "1": "any_icd",
@@ -40,7 +40,7 @@
     "30": "treatment summary"
   },
   "initializer_range": 0.02,
-  "intermediate_size": 3072,
+  "intermediate_size": 1536,
   "label2id": {
     "any_code": 0,
     "any_icd": 1,
@@ -80,12 +80,12 @@
   "max_relative_positions": -1,
   "model_type": "deberta-v2",
   "norm_rel_ebd": "layer_norm",
-  "num_attention_heads": 12,
-  "num_hidden_layers": 6,
+  "num_attention_heads": 6,
+  "num_hidden_layers": 12,
   "pad_token_id": 0,
   "pooler_dropout": 0,
   "pooler_hidden_act": "gelu",
-  "pooler_hidden_size": 768,
+  "pooler_hidden_size": 384,
   "pos_att_type": [
     "p2c",
     "c2p"

metrics.json

@@ -0,0 +1,17 @@
+{
+  "best_val_epoch": 17.0,
+  "best_val_accuracy": 0.9985548597472858,
+  "best_val_loss": 0.01370711624622345,
+  "final_train_loss": 0.0005,
+  "test_accuracy": 0.9985919668000592,
+  "test_loss": 0.013053071685135365,
+  "accuracy": 0.9985919668000592,
+  "balanced_accuracy": 0.9986780793189832,
+  "macro_precision": 0.9986654139119842,
+  "macro_recall": 0.9986780793189832,
+  "macro_f1": 0.998671010625762,
+  "weighted_f1": 0.9985927033253673,
+  "micro_f1": 0.9985919668000592,
+  "auc_micro": 0.9999974629488196,
+  "ap_micro": 0.9999324746733634
+}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c5fcf2526dbf5f8326da79da28b3f5c3e698c359f908439338c2e1a1933ad3b5
-size 567687764
+oid sha256:7a4c2aa3a73027a5d45bb686cdf20d077b50c1fa3a6e73c7faedba2a30445ec4
+size 283392108

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:39720f29c338c54d2269d57b08eaff7d4403c932a6e4ae333eacd6e392357241
+oid sha256:cf3f6bdff1f26ed479f70a4c50c0079d364eefb37e4e306f42ca7a97ac497df0
 size 5713