Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
dudeperf3ct
/
codellm-tokenizer
like
0
tokyotech-llm/swallow-code-v2
License:
apache-2.0
Model card
Files
Files and versions
xet
Community
main
codellm-tokenizer
2.31 MB
1 contributor
History:
6 commits
dudeperf3ct
Pin tokenizer to baseline run
5ed926a
verified
17 days ago
.gitattributes
1.52 kB
initial commit
17 days ago
README.md
143 Bytes
Create README.md
17 days ago
tokenizer.json
2.31 MB
Pin tokenizer to baseline run
17 days ago