tokyotech-llm
/

Llama-3.1-Swallow-8B-Instruct-v0.5

Text Generation

text-generation-inference

Model card Files Files and versions

maym15 commited on Jun 25

Commit

b1f8317

·

verified ·

1 Parent(s): 3ad0e74

Update README.md

Files changed (1) hide show

README.md +0 -1

README.md CHANGED Viewed

@@ -106,7 +106,6 @@ The website [https://swallow-llm.github.io/](https://swallow-llm.github.io/index
 |   |4-shot|4-shot|4-shot|4-shot|4-shot|5-shot|4-shot|4-shot | 3-shot|0-shot|   |
 |   |Acc|EM acc|Acc|EM acc|Acc|Acc|EM acc|CoT EM Acc| CoT EM Acc| pass@1|   |
 | llm-jp-3-7.2b-instruct3                   | 0.328 | 0.479 | 0.563 | 0.501 | 0.876 | 0.462 | 0.264 | 0.028 | 0.420 | 0.219 | 0.414 |
-| Qwen2-7B-Instruct                         | 0.396 | 0.547 | 0.615 | 0.593 | 0.886 | 0.707 | 0.626 | 0.504 | 0.304 | 0.643 | 0.582 |
 | Qwen2.5-7B-Instruct                       | 0.428 | 0.519 | 0.624 | 0.569 | 0.877 | 0.742 | 0.739 | 0.688 | 0.217 | 0.636 | 0.604 |
 | Tanuki-8B-dpo-v1.0                        | 0.334 | 0.283 | 0.469 | 0.501 | 0.816 | 0.377 | 0.487 | 0.178 | 0.333 | 0.288 | 0.406 |
 | Llama 3 8B Instruct                       | 0.388 | 0.670 | 0.583 | 0.611 | 0.892 | 0.657 | 0.745 | 0.306 | 0.646 | 0.554 | 0.605 |

 |   |4-shot|4-shot|4-shot|4-shot|4-shot|5-shot|4-shot|4-shot | 3-shot|0-shot|   |
 |   |Acc|EM acc|Acc|EM acc|Acc|Acc|EM acc|CoT EM Acc| CoT EM Acc| pass@1|   |
 | llm-jp-3-7.2b-instruct3                   | 0.328 | 0.479 | 0.563 | 0.501 | 0.876 | 0.462 | 0.264 | 0.028 | 0.420 | 0.219 | 0.414 |
 | Qwen2.5-7B-Instruct                       | 0.428 | 0.519 | 0.624 | 0.569 | 0.877 | 0.742 | 0.739 | 0.688 | 0.217 | 0.636 | 0.604 |
 | Tanuki-8B-dpo-v1.0                        | 0.334 | 0.283 | 0.469 | 0.501 | 0.816 | 0.377 | 0.487 | 0.178 | 0.333 | 0.288 | 0.406 |
 | Llama 3 8B Instruct                       | 0.388 | 0.670 | 0.583 | 0.611 | 0.892 | 0.657 | 0.745 | 0.306 | 0.646 | 0.554 | 0.605 |