Correct HumanEval scores
#79
by
Muennighoff - opened
Previous scores did not strip end of sequence tokens. The updated scores ignore the end of sequence tokens (</s>).
This is equivalent to evaluating on code generations decoded with tokenizer.decode(code_tokens, skip_special_tokens=True)
Muennighoff changed pull request title from
Update README.md
to Correct HumanEval scores
Muennighoff changed pull request status to
merged