HUME: Measuring the Human-Model Performance Gap in Text Embedding Task
Paper
•
2510.10062
•
Published
•
10
HUME
The HUME benchmark is designed to evaluate the performance of text embedding models and humans on a comparable set of tasks. This captures areas wh...
-
-
mteb/HUMEEmotionClassification
Viewer • Updated • 16k • 21 -
mteb/HUMEToxicConversationsClassification
Viewer • Updated • 8.05k • 26 -
mteb/HUMETweetSentimentExtractionClassification
Viewer • Updated • 27.5k • 20 -
mteb/HUMEMultilingualSentimentClassification
Viewer • Updated • 15.2k • 11 -
mteb/mteb-human-arxiv-clustering
Viewer • Updated • 1 • 11 -
mteb/mteb-human-reddit-clustering
Viewer • Updated • 1 • 11 -
mteb/mteb-human-wikicities-clustering
Viewer • Updated • 1 • 14 -
mteb/mteb-human-sib200-clustering
Viewer • Updated • 10 • 14 -
mteb/HUMECore17InstructionReranking
Viewer • Updated • 320 • 43 -
mteb/HUMENews21InstructionReranking
Viewer • Updated • 496 • 28 -
mteb/HUMERobust04InstructionReranking
Viewer • Updated • 784 • 26 -
mteb/HUMEWikipediaRerankingMultilingual
Viewer • Updated • 1.44k • 32 -
mteb/mteb-human-sickr-sts
Viewer • Updated • 40 • 44 -
mteb/mteb-human-sts12-sts
Viewer • Updated • 50 • 45 -
mteb/mteb-human-stsbenchmark-sts
Viewer • Updated • 50 • 41 -
mteb/mteb-human-sts22-sts
Viewer • Updated • 240 • 52