tokyotech-llm
/

Llama-3.1-Swallow-8B-Instruct-v0.5

Text Generation

text-generation-inference

Model card Files Files and versions

s-mizuki-nlp commited on Jun 25

Commit

3ad0e74

·

verified ·

1 Parent(s): 1fbe3b6

updated benchmark description.

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -144,7 +144,7 @@ We used llm-jp-eval(v1.3.0), JP Language Model Evaluation Harness(commit #9b42d4
 - Automatic summarization (XL-Sum [Hasan et al., 2021])
 - Machine translation (WMT2020 ja-en [Barrault et al., 2020])
 - Machine translation (WMT2020 en-ja [Barrault et al., 2020])
-- Mathematical reasoning (MGSM [Shi et al., 2023])
 - Academic exams (JMMLU [尹ら, 2024])
 - Code generation (JHumanEval [佐藤ら, 2024])
@@ -157,7 +157,7 @@ We used the Language Model Evaluation Harness(v.0.4.2) and Code Generation LM Ev
 - Machine reading comprehension (SQuAD2 [Rajpurkar et al., 2018])
 - Commonsense reasoning (XWINO [Tikhonov and Ryabinin, 2021])
 - Natural language inference (HellaSwag [Zellers et al., 2019])
-- Mathematical reasoning (GSM8K [Cobbe et al., 2021])
 - Mathematical reasoning (MATH [Hendrycks et al., 2022][Lightman et al., 2024])
 - Reasoning (BBH (BIG-Bench-Hard) [Suzgun et al., 2023])
 - Academic exams (MMLU [Hendrycks et al., 2021])

 - Automatic summarization (XL-Sum [Hasan et al., 2021])
 - Machine translation (WMT2020 ja-en [Barrault et al., 2020])
 - Machine translation (WMT2020 en-ja [Barrault et al., 2020])
+- Arithmetic reasoning (MGSM [Shi et al., 2023])
 - Academic exams (JMMLU [尹ら, 2024])
 - Code generation (JHumanEval [佐藤ら, 2024])
 - Machine reading comprehension (SQuAD2 [Rajpurkar et al., 2018])
 - Commonsense reasoning (XWINO [Tikhonov and Ryabinin, 2021])
 - Natural language inference (HellaSwag [Zellers et al., 2019])
+- Arithmetic reasoning (GSM8K [Cobbe et al., 2021])
 - Mathematical reasoning (MATH [Hendrycks et al., 2022][Lightman et al., 2024])
 - Reasoning (BBH (BIG-Bench-Hard) [Suzgun et al., 2023])
 - Academic exams (MMLU [Hendrycks et al., 2021])