Upload folder using huggingface_hub

README.md

@@ -0,0 +1,164 @@
+---
+license: mit
+language:
+  - en
+library_name: transformers
+tags:
+  - text-generation
+  - gpt2
+  - pytorch
+  - fine-tuned
+  - creative-writing
+  - fiction
+  - fantasy
+datasets:
+  - custom
+pipeline_tag: text-generation
+widget:
+  - text: "The witcher drew his sword and"
+    example_title: "Action Scene"
+  - text: "Geralt looked at Yennefer and said,"
+    example_title: "Dialogue"
+  - text: "The monster emerged from the shadows,"
+    example_title: "Monster Encounter"
+inference:
+  parameters:
+    max_length: 100
+    temperature: 0.9
+    top_p: 0.95
+    do_sample: true
+---
+
+# GPT-2 Fine-tuned on The Witcher Books
+
+A GPT-2 Medium model fine-tuned on text from The Witcher book series by Andrzej Sapkowski for creative text generation in a dark fantasy style.
+
+## Model Description
+
+This model is part of the [Deepstory](https://github.com/thetobysiu/deepstory) project, which combines Natural Language Generation, Text-to-Speech, and animation technologies to create interactive storytelling experiences.
+
+The model has been fine-tuned on The Witcher novels to generate text that captures the narrative style, vocabulary, and themes of the original books.
+
+## Model Architecture
+
+This model is based on **GPT-2 Medium** architecture.
+
+| Parameter | Value |
+|-----------|-------|
+| Architecture | GPT2LMHeadModel |
+| Model Size | GPT-2 Medium |
+| Number of Layers (n_layer) | 24 |
+| Hidden Size (n_embd) | 1024 |
+| Attention Heads (n_head) | 16 |
+| Context Length (n_ctx) | 1024 |
+| Max Positions (n_positions) | 1024 |
+| Vocabulary Size | 50,257 |
+| Activation Function | GELU (new) |
+| Attention Dropout | 0.1 |
+| Embedding Dropout | 0.1 |
+| Residual Dropout | 0.1 |
+| Layer Norm Epsilon | 1e-05 |
+
+## Usage
+
+### With Transformers Library
+
+```python
+from transformers import GPT2Tokenizer, GPT2LMHeadModel
+import torch
+
+# Load model and tokenizer
+tokenizer = GPT2Tokenizer.from_pretrained("thetobysiu/gpt2-witcher-books")
+model = GPT2LMHeadModel.from_pretrained("thetobysiu/gpt2-witcher-books")
+
+# Generate text
+prompt = "The witcher drew his silver sword and"
+input_ids = tokenizer.encode(prompt, return_tensors='pt')
+
+output = model.generate(
+    input_ids=input_ids,
+    max_length=150,
+    temperature=0.9,
+    top_p=0.95,
+    top_k=50,
+    do_sample=True,
+    num_return_sequences=1
+)
+
+generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
+print(generated_text)
+```
+
+### Generation Parameters
+
+For best results, we recommend the following generation parameters:
+
+| Parameter | Recommended Value | Description |
+|-----------|-------------------|-------------|
+| temperature | 0.7 - 1.0 | Higher values = more creative |
+| top_p | 0.9 - 0.95 | Nucleus sampling threshold |
+| top_k | 40 - 50 | Top-k sampling |
+| max_length | 100 - 300 | Maximum tokens to generate |
+| do_sample | True | Enable sampling |
+
+## Training Data
+
+The model was fine-tuned on text from The Witcher book series, including:
+- The Last Wish
+- Sword of Destiny
+- Blood of Elves
+- Time of Contempt
+- Baptism of Fire
+- The Tower of the Swallow
+- The Lady of the Lake
+- Season of Storms
+
+## Training Procedure
+
+The model was fine-tuned using the Hugging Face Transformers library starting from the pre-trained GPT-2 Medium checkpoint.
+
+## Intended Use
+
+This model is intended for:
+- Creative writing assistance in fantasy genre
+- Interactive storytelling applications
+- Fan fiction generation
+- Research in language model fine-tuning
+- Educational purposes
+
+## Limitations
+
+- The model may generate content that doesn't perfectly match the original author's style
+- Context length is limited to 1024 tokens
+- May occasionally generate repetitive or incoherent text
+- The training data is based on copyrighted material
+
+## Ethical Considerations
+
+- This model is trained on copyrighted material and should be used for personal/research purposes
+- Generated content should not be presented as original work by the book author
+- Users should be aware that generated text may contain mature themes consistent with the source material
+
+## Citation
+
+If you use this model, please cite the Deepstory project:
+
+```bibtex
+@misc{deepstory,
+  author = {Siu King Wai},
+  title = {Deepstory},
+  year = {2020},
+  publisher = {GitHub},
+  url = {https://github.com/thetobysiu/deepstory}
+}
+```
+
+## License
+
+This model is released under the MIT License. Please note that The Witcher series is the intellectual property of Andrzej Sapkowski and CD Projekt.
+
+## Acknowledgments
+
+- OpenAI for the original GPT-2 model
+- Hugging Face for the Transformers library
+- Andrzej Sapkowski for The Witcher book series

config.json

@@ -0,0 +1,33 @@
+{
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 1024,
+  "n_head": 16,
+  "n_layer": 24,
+  "n_positions": 1024,
+  "n_special": 0,
+  "predict_special_tokens": true,
+  "resid_pdrop": 0.1,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "vocab_size": 50257
+}

generation_config.json

@@ -0,0 +1,11 @@
+{
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "max_length": 200,
+  "do_sample": true,
+  "top_k": 50,
+  "top_p": 0.95,
+  "temperature": 0.9,
+  "repetition_penalty": 1.1,
+  "transformers_version": "4.0.0"
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02b346ed5f42b00724e1e5b05ea95df69cd0a392baf228dd322f8d425a8099cc
+size 1444527986

special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "unk_token": "<|endoftext|>"}

tokenizer_config.json

@@ -0,0 +1,8 @@
+{
+  "model_max_length": 1024,
+  "tokenizer_class": "GPT2Tokenizer",
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>",
+  "pad_token": "<|endoftext|>"
+}