🔍 AVI-M3 — Fine-tuned BGE-M3 for Retrieval

shaipro/avi-m3 is a fine-tuned version of BAAI/bge-m3, adapted for domain-specific retrieval tasks.
It significantly improves retrieval accuracy over the base model on our custom dataset.

---

🧠 Model Details

• Base model: BAAI/bge-m3
• Architecture: Multi-vector dense retriever (BGE-M3)
• Fine-tuned by: shaipro
• Framework: FlagEmbedding
• Language(s): English (domain-specific)
• Task: Text retrieval / semantic search

---

📚 Intended Use

• ✅ Semantic search within a specific corpus
• ✅ Dense retrieval for RAG pipelines
• ✅ Re-ranking candidate documents

Not recommended for:

• ❌ General-purpose question answering without retrieval
• ❌ Open-domain reasoning
• ❌ Generation

---

🧪 Evaluation

We evaluated both the base BGE-M3 and the fine-tuned AVI-M3 on a held-out test set (≈20 % of the dataset).

Metric	Base BGE-M3	Fine-tuned AVI-M3
Accuracy	0.7106	0.9121
MRR	0.8131	0.9524
Recall@1	0.7106	0.9121
Recall@5	0.9377	0.9963
Recall@10	0.9927	1.0000

✅ Fine-tuning led to a ~20 % absolute gain in Recall@1 and Accuracy, with near-perfect Recall@10.

---

📚 Dataset

• Training set: 1088 examples
• Evaluation set: 273 examples (~20% held-out split)
• Task: Query → Positive passage retrieval

---

📊 Metrics Explained

• Accuracy: proportion of queries where the top-1 retrieved document is correct.
• MRR (Mean Reciprocal Rank): average of reciprocal ranks of the correct document.
• Recall@k: proportion of queries where the correct document appears in the top-k retrieved results.

---

💻 Hardware

• GPU: 1× NVIDIA A40 (48 GB VRAM)
• Precision: FP16 with gradient checkpointing
• Effective batch size: 32 (8 × grad accumulation 4)

---

🛠️ Training

• Evaluation dataset: 273 examples (~20 % held-out split)
• Epochs: 10
• Learning rate: 2e-5
• Per-device batch size: 8
• Gradient accumulation: 4
• Pooling method: cls
• Temperature: 0.02
• Loss: m3_kd_loss (knowledge distillation + contrastive)
• Knowledge distillation: Enabled
• Self-distillation: Enabled
• Unified fine-tuning: Enabled
• Encoder freezing: Disabled
• Optimizer: AdamW
• Scheduler: Linear with 10% warmup

---