Update tokenizer config

README.md

@@ -12,6 +12,9 @@ inference:
 
 # Model Card for Mistral-7B-v0.1
 
+**NOTE**: This is a fork of [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) intended to have a non-null pad token. This has been done in order to
+facilitate usage of this model with off-the-shelf PEFT tuners, such as what is offered by Google Cloud Vertex AI.
+
 The Mistral-7B-v0.1 Large Language Model (LLM) is a pretrained generative text model with 7 billion parameters. 
 Mistral-7B-v0.1 outperforms Llama 2 13B on all benchmarks we tested.
 
@@ -43,4 +46,4 @@ Mistral 7B is a pretrained base model and therefore does not have any moderation
 
 ## The Mistral AI Team
  
-Albert Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed.
+Albert Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed.

tokenizer_config.json

@@ -33,7 +33,7 @@
   "eos_token": "</s>",
   "legacy": true,
   "model_max_length": 1000000000000000019884624838656,
-  "pad_token": null,
+  "pad_token": "<unk>",
   "sp_model_kwargs": {},
   "spaces_between_special_tokens": false,
   "tokenizer_class": "LlamaTokenizer",