readme: sync with upstream hmByT5 repo

README.md

@@ -7,4 +7,41 @@ language:
 - fi
 - sv
 - nl
----
+---
+
+# hmByT5
+
+Upcoming Historic Multilingual ByT5 Model. It covers the following languages:
+
+* English (British Library Corpus - Books)
+* German (Europeana Newspaper)
+* French (Europeana Newspaper)
+* Finnish (Europeana Newspaper)
+* Swedish (Europeana Newspaper)
+* Dutch (Delpher Corpus)
+
+# Pretraining
+
+We pretrain hmByT5 on a v3-32 TPU Pod. Details about the training can be found
+[here](https://github.com/stefan-it/hmByT5/tree/main/hmbyt5).
+
+# Evaluation on Downstream Tasks (NER)
+
+We use Flair to fine-tune hmByT5 on HIPE-2022 data. Details about the fine-tuning can be found
+[here](https://github.com/stefan-it/hmByT5/tree/main/bench).
+
+# **New**: Logbook
+
+* 07.04.2022: Pretraining for 200k steps on the English corpus finished without crashes! TensorBoard logs can be found
+              on the [Model Hub](https://huggingface.co/stefan-it/byt5-small-historic-multilingual/tensorboard). We
+              also uploaded all checkpoints (we checkpoint every 25k steps)
+              [here](https://huggingface.co/stefan-it/byt5-small-historic-multilingual). Fine-Tuning experiments (NER)
+              on the English part of AjMC corpus from HIPE-2022 are running.
+* 03.04.2022: We start ByT5 pretraining with official ByT5 implementation on a v3-32 TPU Pod - thankfully provided by
+              [TPU Research Cloud](https://sites.research.google/trc/about/) (TRC) program. Plan is to pretrain on the
+              English corpus for 200k steps and use the original ByT5 Small model as init checkpoint.
+
+# Acknowledgements
+
+Research supported with Cloud TPUs from Google's [TPU Research Cloud](https://sites.research.google/trc/about/) (TRC).
+Many Thanks for providing access to the TPUs ❤️