README.md

---
license: mit
tags:
- chess
- pgn
- tokenizer
pretty_name: PGN Tokenizer
source_datasets:
- InterwebAlchemy/pgn-dataset-including-special-tokens
task_categories:
  - token-classification
task_ids:
  - parsing
---

# PGN Tokenizer

![PGN Tokenizer Visualization](https://github.com/DVDAGames/pgn-tokenizer/raw/main/docs/assets/pgn-tokenizer.png)

This is a Byte Pair Encoding (BPE) tokenizer for chess Portable Game Notation (PGN).