Add files using upload-large-folder tool

.gitattributes

@@ -1,35 +1,2 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+**.safetensors filter=lfs diff=lfs merge=lfs -text
+tokenizer.model filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,125 @@
+# ERNIE-4.5-VL-424B-A47B-Base
+
+## ERNIE 4.5 Highlights
+
+The advanced capabilities of the ERNIE 4.5 models, particularly the MoE-based A47B and A3B series, are underpinned by several key technical innovations:
+
+- **Multimodal MoE Pretraining:** Our models are jointly trained on both textual and visual modalities to better capture the nuances of multimodal information and improve performance on tasks involving text generation, image understanding, and cross-modal reasoning. To achieve this without one modality hindering the learning of another, we designed a heterogeneous MoE structure, incorporated three-dimensional rotary embeddings, and employed router orthogonal loss and multimodal token-balanced loss. These architectural choices ensure that both modalities are effectively represented, allowing for mutual reinforcement during training.
+- **Scaling-Efficient Architecture and Infrastructure:** To train the large multimodal MoE models efficiently, we introduce a novel heterogeneous hybrid parallelism and multi-level load balancing strategy for efficient training of ERNIE 4.5 models. By using on-device expert parallelism, memory-efficient pipeline scheduling, and FP8 mixed precision, we achieve ideal pre-training performance. For inference, we propose a quantization method with collaborative parallelism among multiple experts to achieve lossless quantization. Built on PaddlePaddle, ERNIE 4.5 delivers high-performance inference across a wide range of hardware platforms.
+- **Modality-Specific Post-training:** To meet the diverse requirements of real-world applications, we fine-tuned variants of the pretrained model for specific modalities. Our LLMs are optimized for general-purpose language understanding and generation. The VLMs focuses on visual-language understanding and supports both thinking and no-thinking mode. Each model employed a combination of Supervised Fine-tuning (SFT), Direct Preference Optimization (DPO) or a modified reinforcement learning method named Unified Preference Optimization (UPO) for post-training, using targeted datasets aligned with its intended usage scenario.
+
+To ensure the stability of multimodal joint training, we adopt a staged training strategy. In the first and second stage, we train only the text-related parameters, enabling the model to develop strong fundamental language understanding as well as long-text processing capabilities. The final multimodal stage extends capabilities to images and videos by introducing additional parameters including a ViT for image feature extraction, an adapter for feature transformation, and visual experts for multimodal understanding. At this stage, text and visual modalities mutually enhance each other. After pretraining trillions tokens, we obtained ERNIE-4.5-VL-424B-A47B-Base.
+
+## Model Overview
+
+ERNIE-4.5-VL-424B-A47B-Base is a multimodal MoE Base model, with 424B total parameters and 47B activated parameters for each token. The following are the model configuration details:
+
+| Key                               | Value         |
+| --------------------------------- | ------------- |
+| Modality                          | Text & Vision |
+| Training Stage                    | Pretraining   |
+| Params(Total / Activated)         | 424B / 47B    |
+| Layers                            | 54            |
+| Heads(Q/KV)                       | 64 / 8        |
+| Text Experts(Total / Activated)   | 64 / 8        |
+| Vision Experts(Total / Activated) | 64 / 8        |
+| Context Length                    | 131072        |
+
+## Benchmark
+
+| Capability        | Benchmark           | ERNIE-4.5-VL-424B-A47B-Base | GPT-4.1 |
+| ----------------- | ------------------- | --------------------------- | ------- |
+| Average           |                     |                             |         |
+| Visual Perception | CVBench             |                             | 82.49   |
+|                   | CountBench          |                             |         |
+|                   | RealWorldQA         |                             | 77.25   |
+|                   | VLMAreBlind         |                             |         |
+| Knowledge         | CCBench             |                             | 78.65   |
+| Chart&Doc&OCR     | OCRBench            |                             | 83.00   |
+|                   | TableVQA            |                             | 72.13   |
+|                   | ChartQA             |                             | 82.56   |
+|                   | DocVQA(val)         |                             | 87.84   |
+|                   | ChartXiv-Reasoning  |                             | 58.30   |
+| Vision-Reasoning  | VisualPuzzle        |                             | 45.63   |
+|                   | Logicvista          |                             |         |
+| STEM              | OlympiadBench       |                             | 39.95   |
+|                   | MathVista(testmini) |                             | 70.90   |
+|                   | MathVerse           |                             | 62.46   |
+|                   | MMMU(val)           |                             | 73.07   |
+|                   | AI2D                |                             | 95.34   |
+|                   | MathVision          |                             | 50.46   |
+| Video             | MVBench             |                             | 64.15   |
+|                   | VideoMME w/o subs   |                             | 74.49   |
+|                   | VideoMME w/ subs    |                             | 78.90   |
+|                   | MLVU                |                             | 73.33   |
+|                   | LongVideoBench      |                             | 63.47   |
+
+## Quickstart
+
+### Using `transformers` library
+
+Here is an example of how to use the transformers library for inference:
+
+```bash
+from transformers import AutoProcessor, AutoTokenizer, AutoModelForCausalLM
+
+model_path = 'Baidu/ERNIE-4.5-VL-424B-A47B-Base'
+model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto", torch_dtype=torch.bfloat16)
+
+processor = AutoProcessor.from_pretrained(model_path)
+processor.eval()
+model.add_image_preprocess(processor)
+
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {"type": "text", "text": "Describe the image."},
+            {"type": "image_url", "image_url": {"url": "https://paddlenlp.bj.bcebos.com/datasets/paddlemix/demo_images/example1.jpg}},
+        ]
+    },
+]
+
+texts, images, videos = processor.pre_process(messages)
+inputs = processor(texts, images, videos)
+
+device = next(model.parameters()).device
+inputs = inputs.to(device)
+
+generated_ids = model.generate(
+    inputs=inputs['input_ids'].to(device),
+    **inputs,
+    max_new_tokens=128
+    )
+output_text = processor.decode(generated_ids[0])
+print(output_text)
+```
+
+### vLLM inference
+
+vLLM is currently being adapted, priority can be given to using our fork repository [vllm](https://github.com/CSWYF3634076/vllm/tree/ernie)
+
+```bash
+# 80G * 16 GPU
+vllm serve Baidu/ERNIE-4.5-VL-424B-A47B-Base --trust-remote-code
+```
+
+## License
+
+The ERNIE 4.5 models are provided under the Apache License 2.0. This license permits commercial use, subject to its terms and conditions. Copyright © 2025 Baidu, Inc. All Rights Reserved.
+
+## Citation
+
+If you find ERNIE 4.5 useful or wish to use it in your projects, please kindly cite our technical report:
+
+```bibtex
+@misc{ernie2025technicalreport,
+      title={ERNIE 4.5 Technical Report},
+      author={Baidu ERNIE Team},
+      year={2025},
+      eprint={},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={}
+}
+```

model-00002-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e161da7ec614797e6cc84d2879ad82aa0852c882736652b238959f8c8689c027
+size 4932555976

model-00005-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52ae6d9b59e4facf5a76a4253ed745511ee6d72224efdd1a36673996d50b12d3
+size 4919965224

model-00007-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:276dc5fd687896683b3185a594d2748c7c1d62f1f1867127052f48862e6e74b4
+size 4974451832

model-00013-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28d44a8a25d1bf598e15db260832473674be6be3cad811a8d56e7fce70957122
+size 4932508544

model-00014-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28120c3fd82f9c077e1e0a648f8fe3cd870f265e79d22bca4e5962b8b6aae950
+size 4999621048

model-00019-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d65d447cceb67d1e15c6b2cc51425452135d9060813e79b624b3203b772327d0
+size 4932508528

model-00020-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa3c507bcc7f9a3919a8c095ec226e26471af1f50ee62fec1b882b18d36c5168
+size 4932508544

model-00021-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:febdd2842cc2fe3930bc29cc82e0bb0db781c217cd3b52ecdc4cafdb8a7ce025
+size 4999624432

model-00022-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33ec983c4ee0229ef76a6be1e802c9698c7690e7192a05536a73073cb1c3667d
+size 4894795608

model-00031-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f922d4aa32c30b35b08227dfa6bad24281a1894eeb54241c3034b7a8bcca6c50
+size 4974458976

model-00033-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90425eb72c83002dc66e18c2983223bd06ade7fe3046bd5cf46e9d06419ecb5c
+size 4932508600

model-00034-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:698a03d7a05d3808f598e6c1fc4e53d671c99a8d42994a7722f1de29aebe7362
+size 4974455944

model-00040-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b07531792a1a2309407db6d88272bbefd97417901881b973271d6bcc94aa9e34
+size 4932508600

model-00041-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b1e008beb6c00aad35d6f507fd7e53cdea459692d95938674c840374aa04948
+size 4999625184

model-00042-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f9ac37cb9b9d5bf8be34318f724c8dca92ac9087a9a954d4c7f5d94c3077b7e
+size 4894795048

model-00045-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a504ffeb6bbc5829e9c022bd62bd8172e803ead4b6196b2bb3d0fc68ee512cd
+size 4894798112

model-00048-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac77abee7474dc7366aef31058a15b38f1ea49329c06961b844dba7ff75e49c9
+size 4894801160

model-00051-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bff2ad129fae75ac8ecc61ffd4a088446f59ec485146e21a1631310fd12737cf
+size 4999625568

model-00054-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca6a195509d6a38a7f420eb0e50a9efb978b29c24728bcb2b136b1854d09d7c5
+size 4999622528

model-00059-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4710cc205c2dad14be124090ddb97f89ec5d32b8276fd6f97e8016b51da62e81
+size 4932508584

model-00073-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ea4aa61d1053151ec564690f77ec161dd4c688c8d7bdb7c2063fe83172ecc03
+size 4932508600

model-00076-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e47575991d04f5f3f0ab29433a6ce8b3769518c68300a0977279b5f733488a3
+size 4932508600

model-00079-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81f81ca82b410e370e53893e71be04d6e9d34bbdbce177a35fc70454fd131c55
+size 4932508592

model-00082-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b188e0a7aa794841963dfcd6a7f346e893bbcd901c2c4633c5dd3652f5d49a1f
+size 4894793664

model-00089-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea8d76cfc2c3158d3ab1f38ac8c3c27f0651338b2b7cce4a86aa8561675c9cbe
+size 4932508592

model-00092-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d21db8f2ac91222cf1868b41056254f494e249c2c05db5ddac9da16d8c91e0c1
+size 4940913544

model-00093-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e2cfb61cfd726e0b9132d5c7061dc065a5d1018a84dd1c320582c73b2dae0e6
+size 4932508600

model-00096-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a003ff348a5f4ac1abc49a672fe252a0a8487f95e2c3055b186ce57d3cac6642
+size 4932508600

model-00097-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5dc54f1f193bb86e5a9a4c05896a25def63d9ba6db047b203ddc29c05b908f9
+size 4974454928

model-00103-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87d16c897823dd566f6f6e9b15ef5975bc0e7411f0ba6a4a128ad3c509ddd22c
+size 4932508600

model-00104-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b854b28c5ef667b53ef3b7d35a04c1a7cc0b8aa06654f2c98837b3f585b71d8b
+size 4999624168

model-00108-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4bd5a82c9f3e47a6b8e01ed2c25e2772da74c223f6dc995f82b9a332f78d1f6
+size 4894799128

model-00110-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b4c73fcac01e8e364c0cd16580b2fee1b52571e2b97a31620a7ab248bd7114e
+size 4999618096

model-00113-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d73f5bedd36bc7008c6656c7637f2058035dabbd6df662e3c2962365ad82f40
+size 4932508600

model-00125-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7b4db0fb8d993678673da4f49cef2c3679419967494f72282b3ce7fbafb8372
+size 4919961256

model-00129-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:11b0c9c2a9ea9986ec0ad1e1822672b049410a61a7b001a3313c81bb79f0996f
+size 4932508600

model-00132-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b354f27213f579c4c57734c7c53375938e53326153aed96c6ec68134cf72ca4
+size 4932508584

model-00139-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:405647585c2b09a46b7609665e21a33533c1641fc15d27118de92c2346b61e57
+size 4932508600

model-00143-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22d74c78a4ad4724d26478d3bba352c1bb8d816e7c6e3dd9e5a5f165e25ec0b5
+size 4932508600

model-00144-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eff03749864a8d4a2525dbb89d9e5d878c24309d11c17a3ad693ebc354282ea7
+size 4999625568

model-00146-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b90289094628d05142093bbeb0cf0288eb18e3ee823ee734010355185235f75d
+size 4932508600

model-00149-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f2ff3c11c4324465d1432cf7a607a4458d50bc7b9255097b2ff10a6fa96db9ac
+size 4932508600

model-00152-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b3e315e927ef536fd99e7154c59eef2891c0cc0ff70f2816c2795543fa633ed
+size 4932508584

model-00155-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f87f364297d1c8ff1fb217f3eef9905a17d7f3a1a6475131dd975a3e7c11a40
+size 4919960240

model-00156-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c37b57e17982e7900b1202ce29352b55bc52ed4d82410b89f75a9c00c73b736
+size 4932508600

model-00167-of-00172.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2bf77f5bd9eda27bf7b7fd61aa1579085355b82bb45c2865081fea543d41acf
+size 4999623152

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ed2203974453df691287a0432c06737f1b17f20f5ab325fb33e31844d90ddb0
+size 1614362

tokenizer_config.json

@@ -0,0 +1,16 @@
+{
+    "bos_token": "<s>",
+    "eos_token": "</s>",
+    "pad_token": "<unk>",
+    "unk_token": "<unk>",
+    "cls_token": "<|begin_of_sentence|>",
+    "sep_token": "<|end_of_sentence|>",
+    "mask_token": "<mask:1>",
+    "sys_start_token": "<mask:4>",
+    "sys_end_token": "<mask:5>",
+    "header_start_token": "<mask:6>",
+    "header_end_token": "<mask:7>",
+    "additional_special_tokens": null,
+    "tokenizer_class": "Ernie4_5_VLTokenizer",
+    "chat_template": "\n{%- set image_count = namespace(value=0) -%}\n{%- set video_count = namespace(value=0) -%}\n{{- '<|begin_of_sentence|>' }}\n{%- for message in messages -%}\n    {%- if message.role in ['system', 'user'] -%}\n        {%- if message.role == 'user' -%}\n            {{- 'User: ' -}}\n        {%- endif -%}\n        {%- if message.content is string -%}\n            {{- message.content -}}\n        {%- else -%}\n            {%- for content_item in message.content -%}\n                {%- if content_item.type == 'text' -%}\n                    {{- content_item.text -}}\n                {%- elif content_item.type == 'image_url' -%}\n                    {%- set image_count.value = image_count.value + 1 -%}\n                    Picture {{ image_count.value }}:<|IMAGE_START|><|image@placeholder|><|IMAGE_END|>\n                {%- elif content_item.type == 'video_url' -%}\n                    {%- set video_count.value = video_count.value + 1 -%}\n                    Video {{ video_count.value }}:<|VIDEO_START|><|video@placeholder|><|VIDEO_END|>\n                {%- endif -%}\n            {%- endfor -%}\n        {%- endif -%}\n        {%- if message.role == 'system' -%}\n            {{- '\n' -}}\n        {%- endif -%}\n    {%- elif message.role == 'assistant' -%}\n        {%- macro extract_text_content(content_field) -%}\n            {%- if content_field is string -%}\n                {{- content_field -}}\n            {%- elif content_field is iterable and content_field is not string -%}\n                {%- set ns = namespace(text_parts=[]) -%}\n                {%- set text_parts = [] -%}\n                {%- for item in content_field -%}\n                    {%- if item.type == 'text' -%}\n                        {%- set ns.text_parts = ns.text_parts + [item.text] -%}\n                    {%- endif -%}\n                {%- endfor -%}\n                {{- ns.text_parts | join('') -}}\n            {%- else -%}\n                {{- '' -}}\n            {%- endif -%}\n        {%- endmacro -%}\n        {%- set reasoning_content = extract_text_content(message.reasoning_content) -%}\n        {%- set content = extract_text_content(message.content) -%}\n        {%- if '</think>' in content %}\n            {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}\n            {%- set content = content.split('</think>')[-1].lstrip('\n') %}\n        {%- endif %}\n        {%- if reasoning_content %}\n            {{- '\n' + 'Assistant: ' + '<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}\n        {%- else %}\n            {{- '\n' + 'Assistant: ' + content }}\n        {%- endif %}\n        {{- '<|end_of_sentence|>' }}\n    {%- endif -%}\n{%- endfor -%}\n{%- if add_generation_prompt is not defined or add_generation_prompt is true %}\n    {{- '\nAssistant: ' -}}\n    {%- if enable_thinking is defined and enable_thinking is false %}\n        {{- '<think>\n\n</think>\n\n' }}\n    {%- endif %}\n    {%- if enable_thinking is not defined or enable_thinking is true %}\n        {{- '<think>' }}\n    {%- endif %}\n{%- endif %}\n"
+}