kn1ght / tokenizer.json

Commit History

Revert to NFD normalization
9babf3e
verified

InterwebAlchemy commited on

Rename tokenizer_config.json to tokenizer.json
11d7238
verified

InterwebAlchemy commited on

Rename tokenizer.json to tokenizer_config.json
7e5462a
verified

InterwebAlchemy commited on