README.md

---
language: en
tags:
- information-retrieval
- LLM
- Embedding
- text-retrieval
- disaster-management

task_categories:
- text-retrieval

license: apache-2.0
model_name: DMRetriever
---

# DMRetriever: A Family of Models for Improved Text Retrieval in Disaster Management

This repository provides an overview of **DMRetriever**, a family of embedding and retrieval models designed for **disaster-management retrieval tasks**.  
For details, please refer to the [paper](https://www.arxiv.org/abs/2510.15087) and the [GitHub repository](https://github.com/KaiYin97/DMRETRIEVER).

DMRetriever includes model variants with **33M, 109M, 335M, 596M, 4B, and 7.6B parameters**.  
These models are trained via a **three-stage learning framework** consisting of:

1. **Bidirectional Attention Adaptation**  
2. **Unsupervised Contrastive Pre-training**  
3. **Difficulty-aware Progressive Instruction Fine-tuning**

All stages leverage high-quality data generated through an advanced data-refinement pipeline.  
DMRetriever achieves **state-of-the-art (SOTA)** performance across six retrieval intents at all model scales.


<p align="center">
  <img src="https://huggingface.co/DMIR01/DMRetriever/resolve/main/DMRetriever_workflow_new.png" 
       alt="DMRetriever Workflow" width="750"/>
</p>


## 📚 Dataset  
Training data are publicly available on [DMRetriever_MTT](https://huggingface.co/datasets/DMIR01/DMRetriever_MTT).

---

## 🧪 Evaluation

Performance across six retrieval intents on the **[DisastIR-Test](https://huggingface.co/datasets/DMIR01/DisastIR-DevLite)** benchmark.
The evaluation is conducted using this [code](https://github.com/KaiYin97/DMRETRIEVER/tree/main/DMRetriever/eva).

### 🧩 Small Size (≤109M)

| Model | Scale | QA | QAdoc | TW | FC | NLI | STS | Avg. |
|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
| thenlper-gte-small | 33M | 18.04 | 9.13 | 10.95 | 49.63 | 37.51 | 55.55 | 30.14 |
| arctic-embed-m | 109M | 33.15 | 14.04 | 8.48 | 35.07 | 38.67 | 56.20 | 30.94 |
| thenlper-gte-base | 109M | 9.18 | 5.42 | 37.91 | 60.45 | 42.52 | 46.07 | 33.59 |
| arctic-embed-m-v1.5 | 109M | 25.76 | 30.41 | 17.95 | 47.97 | 42.88 | 64.16 | 38.19 |
| arctic-embed-s | 33M | 38.58 | 28.81 | 21.33 | 47.21 | 39.85 | 66.96 | 40.46 |
| bge-small-en-v1.5 | 33M | 56.91 | 51.19 | 25.15 | 55.17 | 32.87 | 64.54 | 47.64 |
| bge-base-en-v1.5 | 109M | 51.50 | 52.78 | 46.72 | 59.93 | 41.16 | <u>68.63</u> | 53.45 |
| **DMRetriever-33M (ours)** | 33M | <u>62.47</u>† | <u>57.03</u>† | <u>57.22</u>† | <u>60.81</u>† | <u>46.56</u>† | 67.57 | <u>58.61</u>† |
| **DMRetriever-109M (ours)** | 109M | **63.19**† | **59.55**† | **58.88**† | **62.48**† | **46.93**† | **68.79**† | **59.97**† |

---

### ⚙️ Medium Size (137M–335M)

| Model | Scale | QA | QAdoc | TW | FC | NLI | STS | Avg. |
|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
| arctic-embed-m-long | 137M | 21.51 | 10.86 | 19.24 | 36.13 | 41.67 | 54.94 | 30.73 |
| arctic-embed-l | 335M | 40.56 | 30.19 | 14.98 | 32.64 | 34.20 | 56.10 | 34.78 |
| bge-large-en-v1.5 | 335M | 56.76 | 54.45 | 32.20 | 54.90 | 35.11 | 64.47 | 49.65 |
| gte-base-en-v1.5 | 137M | 60.51 | 55.62 | 46.26 | 52.24 | 39.59 | <u>70.40</u> | 54.10 |
| mxbai-embed-large-v1 | 335M | <u>64.24</u> | <u>62.63</u> | 39.94 | <u>58.12</u> | 40.18 | 68.01 | 55.52 |
| arctic-embed-m-v2.0 | 305M | 61.22 | 62.20 | <u>47.01</u> | 57.79 | <u>42.29</u> | 64.51 | <u>55.84</u> |
| **DMRetriever-335M (ours)** | 335M | **67.44**† | **62.69**† | **62.16**† | **64.42**† | **49.69**† | **70.71**† | **62.85**† |

---

### 🚀 Large Size (434M–1.5B)

| Model | Scale | QA | QAdoc | TW | FC | NLI | STS | Avg. |
|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
| arctic-embed-l-v2.0 | 568M | 55.23 | 59.11 | 38.11 | 60.10 | 41.07 | 62.61 | 52.70 |
| gte-large-en-v1.5 | 434M | 67.37 | 58.18 | 39.43 | 52.66 | 34.45 | 66.47 | 53.09 |
| Qwen3-Embedding-0.6B | 596M | 66.10 | 52.31 | 62.38 | 64.89 | 50.30 | 67.39 | 60.56 |
| mulling-e5-large-instruct | 560M | 67.97 | <u>64.64</u> | 62.25 | <u>66.78</u> | 48.51 | 63.42 | 62.26 |
| mulling-e5-large | 560M | 66.99 | 64.01 | 62.81 | 59.87 | 50.93 | <u>74.12</u> | 63.12 |
| gte-Qwen2-1.5B-instruct | 1.5B | <u>69.85</u> | 59.17 | <u>65.09</u> | 62.73 | <u>55.51</u> | 73.58 | 64.32 |
| inf-retriever-v1-1.5b | 1.5B | 69.41 | 64.29 | 62.99 | 65.39 | 54.03 | 73.92 | <u>65.01</u> |
| **DMRetriever-596M (ours)** | 596M | **72.44**† | **67.50**† | **65.79**† | **69.15**† | **55.71**† | **74.73**† | **67.55**† |

---

### 🧠 XL Size (≥4B)

| Model | Scale | QA | QAdoc | TW | FC | NLI | STS | Avg. |
|:--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
| Qwen3-Embedding-8B | 7.6B | 44.21 | 34.38 | 41.56 | 42.04 | 32.53 | 42.95 | 39.61 |
| gte-Qwen2-7B-instruct | 7.6B | 70.24 | 47.41 | 63.08 | 31.62 | 53.71 | 74.88 | 56.82 |
| NV-Embed-v1 | 7.9B | 68.06 | 62.70 | 56.02 | 59.64 | 48.05 | 67.06 | 60.26 |
| Qwen3-Embedding-4B | 4B | 67.20 | 59.14 | 65.28 | 67.16 | 53.61 | 58.51 | 61.82 |
| e5-mistral-7b-instruct | 7.1B | 65.57 | 64.97 | 63.31 | 67.86 | 47.55 | 66.48 | 62.58 |
| NV-Embed-v2 | 7.9B | 74.47 | 69.37 | 42.40 | 68.32 | <u>58.20</u> | 76.07 | 64.80 |
| inf-retriever-v1 | 7.1B | 72.84 | 66.74 | 66.23 | 65.53 | 51.86 | 75.98 | 66.53 |
| SFR-Embedding-Mistral | 7.1B | 71.41 | 67.14 | 69.45 | 70.31 | 50.93 | 72.67 | 66.99 |
| Linq-Embed-Mistral | 7.1B | 74.40 | 70.31 | 64.11 | 70.64 | 52.46 | 71.25 | 67.19 |
| **DMRetriever-4B (ours)** | 4B | <u>75.32</u>† | <u>70.23</u>† | <u>70.55</u>† | <u>71.44</u>† | 57.63 | <u>77.38</u>† | <u>70.42</u>† |
| **DMRetriever-7.6B (ours)** | 7.6B | **76.19**† | **71.27**† | **71.11**† | **72.47**† | **58.81**† | **78.36**† | **71.37**† |

---

## 📦 DMRetriever Series Model List

| **Model** | **Description** | **Backbone** | **Backbone Type** | **Hidden Size** | **#Layers** |
|:--|:--|:--|:--|:--:|:--:|
| [DMRetriever-33M](https://huggingface.co/DMIR01/DMRetriever-33M) | Base 33M variant | MiniLM | Encoder-only | 384 | 12 |
| [DMRetriever-33M-PT](https://huggingface.co/DMIR01/DMRetriever-33M-PT) | Pre-trained version of 33M | MiniLM | Encoder-only | 384 | 12 |
| [DMRetriever-109M](https://huggingface.co/DMIR01/DMRetriever-109M) | Base 109M variant | BERT-base-uncased | Encoder-only | 768 | 12 |
| [DMRetriever-109M-PT](https://huggingface.co/DMIR01/DMRetriever-109M-PT) | Pre-trained version of 109M | BERT-base-uncased | Encoder-only | 768 | 12 |
| [DMRetriever-335M](https://huggingface.co/DMIR01/DMRetriever-335M) | Base 335M variant | BERT-large-uncased-WWM | Encoder-only | 1024 | 24 |
| [DMRetriever-335M-PT](https://huggingface.co/DMIR01/DMRetriever-335M-PT) | Pre-trained version of 335M | BERT-large-uncased-WWM | Encoder-only | 1024 | 24 |
| [DMRetriever-596M](https://huggingface.co/DMIR01/DMRetriever-596M) | Base 596M variant | Qwen3-0.6B | Decoder-only | 1024 | 28 |
| [DMRetriever-596M-PT](https://huggingface.co/DMIR01/DMRetriever-596M-PT) | Pre-trained version of 596M | Qwen3-0.6B | Decoder-only | 1024 | 28 |
| [DMRetriever-4B](https://huggingface.co/DMIR01/DMRetriever-4B) | Base 4B variant | Qwen3-4B | Decoder-only | 2560 | 36 |
| [DMRetriever-4B-PT](https://huggingface.co/DMIR01/DMRetriever-4B-PT) | Pre-trained version of 4B | Qwen3-4B | Decoder-only | 2560 | 36 |
| [DMRetriever-7.6B](https://huggingface.co/DMIR01/DMRetriever-7.6B) | Base 7.6B variant | Qwen3-8B | Decoder-only | 4096 | 36 |
| [DMRetriever-7.6B-PT](https://huggingface.co/DMIR01/DMRetriever-7.6B-PT) | Pre-trained version of 7.6B | Qwen3-8B | Decoder-only | 4096 | 36 |


---

## 🚀 Usage  
Please refer to each model’s [Hugging Face page](https://huggingface.co/DMIR01) for specific usage instructions, including input format, embedding extraction, and evaluation examples.

---

## 🧾 Citation  
If you find this repository helpful, please consider citing the corresponding paper:

```bibtex
@article{yin2025dmretriever,
  title={DMRetriever: A Family of Models for Improved Text Retrieval in Disaster Management},
  author={Yin, Kai and Dong, Xiangjue and Liu, Chengkai and Lin, Allen and Shi, Lingfeng and Mostafavi, Ali and Caverlee, James},
  journal={arXiv preprint arXiv:2510.15087},
  year={2025}
}