Final improvements

README.md

@@ -14,9 +14,13 @@ language:
 - ja
 - ko
 - ar
+library_name: transformers.js
+tags:
+- onnxruntime
+pipeline_tag: image-text-to-text
 ---
 > [!Tip]
-> This model was contributed by [Joshua](https://huggingface.co/Xenova) from Hugging Face. We sincerely appreciate the integration and community collaboration. While preliminary functionality checks have been performed, comprehensive testing has not yet been completed. We recommend you to proceed with caution and conducting your own evaluations for specific use cases. If any issues arise, open a PR/Issue here and we will try to address them promptly.
+> This model was contributed by [Xenova](https://huggingface.co/Xenova) from Hugging Face. We sincerely appreciate the integration and community collaboration. While preliminary functionality checks have been performed, comprehensive testing has not yet been completed. We recommend you to proceed with caution and conducting your own evaluations for specific use cases. If any issues arise, open a PR/Issue here and we will try to address them promptly.
 
 # Ministral 3 3B Instruct 2512
 The smallest model in the Ministral 3 family, **Ministral 3 3B** is a powerful, efficient tiny language model with vision capabilities.
@@ -119,6 +123,139 @@ We compare Ministral 3 to similar sized models.
 
 ## Usage
 
+
+### Transformers.js
+
+#### Online WebGPU Demo
+
+You can test out the model directly in your browser using our online WebGPU demo: https://huggingface.co/spaces/mistralai/Ministral_3B_WebGPU.
+
+<video src="https://cdn-uploads.huggingface.co/production/uploads/61b253b7ac5ecaae3d1efe0c/LA0XUsH9pFZaphbfF-HGm.mp4" controls autoplay muted></video>
+
+#### JavaScript
+
+If you haven't already, you can install the [Transformers.js](https://huggingface.co/docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@huggingface/transformers) using:
+```bash
+npm i @huggingface/transformers
+```
+
+You can then run the model as follows:
+```js
+import {
+    AutoProcessor,
+    AutoModelForImageTextToText,
+    load_image,
+    TextStreamer,
+} from "@huggingface/transformers";
+
+// Load processor and model
+const model_id = "mistralai/Ministral-3-3B-Instruct-2512-ONNX";
+const processor = await AutoProcessor.from_pretrained(model_id);
+const model = await AutoModelForImageTextToText.from_pretrained(model_id, {
+    device: "webgpu",
+});
+
+// Prepare inputs
+const messages = [
+    {
+        role: "user",
+        content: [
+            { type: "image" },
+            {
+                type: "text",
+                text: "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.",
+            },
+        ],
+    },
+]
+const prompt = processor.apply_chat_template(messages);
+const url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438";
+const image = await load_image(url);
+const inputs = await processor(image, prompt, { add_special_tokens: false });
+
+// Generate response
+const outputs = await model.generate({
+    ...inputs,
+    max_new_tokens: 2048,
+    streamer: new TextStreamer(processor.tokenizer, {
+        skip_prompt: true,
+        // callback_function: (text) => { /* Do something with the streamed output */ },
+    }),
+});
+
+// Decode output
+const decoded = processor.batch_decode(
+    outputs.slice(null, [inputs.input_ids.dims.at(-1), null]),
+    { skip_special_tokens: true },
+);
+console.log(decoded[0]);
+```
+
+<details>
+
+<summary>See example output</summary>
+
+```
+In this Pokémon game screenshot, you have several potential actions to consider, depending on the situation and your strategy. Here are the possible actions and their implications:
+
+---
+
+### **1. FIGHT**
+**Description:** Use the Pikachu in your possession to attack the Pidgey.
+
+#### **Pros:**
+- **Potential to defeat Pidgey:** If Pikachu has strong moves (e.g., Thunderbolt, Electric Move) and Pidgey is weak to Electric-type attacks, this could be a strong choice.
+- **Experience Gain:** Winning battles typically rewards you with experience points (XP) and sometimes drops Pidgey as a captured Pokémon.
+- **Training Opportunity:** Helps you level up Pikachu, which can improve its stats and access new moves.
+
+#### **Cons:**
+- **Risk of Losing Pikachu:** If Pikachu is already at a low HP (e.g., 83/83 is not bad, but if it's lower), a loss could be detrimental.
+- **Move Selection Matters:** If Pikachu doesn’t have a strong move against Pidgey, you might lose the battle unnecessarily.
+- **Stamina Cost:** Battles can consume stamina if you're on a timer-based system (e.g., Pokémon Red/Blue).
+
+---
+
+### **2. RUN**
+**Description:** Use the "Run" option to escape the battle.
+
+#### **Pros:**
+- **Avoids Risk:** If you're unsure about Pikachu’s moves or HP, running could save you from a potentially bad outcome.
+- **Preserves Resources:** Running avoids losing Pikachu’s HP, which could be useful if you're running low.
+- **Flexibility:** Allows you to explore or use other Pokémon later.
+
+#### **Cons:**
+- **No XP Gain:** Running means you won’t earn XP for defeating Pidgey.
+- **Potential Consequences:** If you run too often, you might miss out on capturing Pidgey or gaining experience.
+- **No Training:** Pikachu won’t level up or gain new moves if you avoid battles.
+
+---
+
+### **3. POKEMON (Select Another Pokémon)**
+**Description:** If you have another Pokémon in your party, you could switch to it.
+
+#### **Pros:**
+- **Better Move Selection:** If you have a stronger Pokémon (e.g., a Fire-type or Ground-type) that can handle Pidgey, switching could be beneficial.
+- **Balanced Strategy:** Helps you manage your team better.
+- **Avoids Weaknesses:** If Pikachu is weak to something Pidgey might use, switching could prevent a loss.
+
+#### **Cons:**
+- **No Immediate XP Gain:** Switching doesn’t defeat Pidgey, so you won’t earn XP.
+- **Stamina Cost:** Switching might still consume stamina if you're on a timer.
+- **Potential to Lose Pikachu:** If you switch and Pikachu is already at a low HP, you might lose it.
+
+---
+
+### **Best Strategy Based on the Image:**
+- **If you want to capture Pidgey:** Fighting is the best option, as it rewards you with experience and potentially captures Pidgey.
+- **If you're unsure about Pikachu’s moves:** Running could be safer, but you might miss out on XP.
+- **If you have another Pokémon:** Switching could be a good idea if you have a stronger option.
+
+Would you like to know what moves Pikachu might have, or do you have another Pokémon in your party? That could help decide the best action!
+```
+
+</details>
+
+
 ### ONNXRuntime
 
 ```py
@@ -223,9 +360,114 @@ print()
 print(processor.batch_decode(generated_tokens, skip_special_tokens=True)[0])
 ```
 
-### Transformers.js
+<details>
+<summary>See example output</summary>
+
+```
+In this *Pokémon* game screenshot, you are presented with a battle scenario between **Pidgey** and **Pikachu**. Here are the possible actions along with an analysis of their potential outcomes:
+
+---
+
+### **1. FIGHT**
+**Explanation:**
+- You choose to engage in battle.
+- Pidgey is a **Level 17** Pokémon with basic stats, while Pikachu is a **Level 42** Pokémon with significantly higher HP, Attack, and overall power.
+- **Pros:**
+  - Pikachu is stronger and likely to win if it lands a hit.
+  - Pidgey may be able to land a few moves before being overwhelmed.
+- **Cons:**
+  - Pidgey is likely to lose quickly due to its lower stats and experience.
+  - Pikachu may have a high chance of winning in one or two turns.
+
+---
+
+### **2. BAG**
+**Explanation:**
+- You choose to **Bag** Pikachu.
+- **Pros:**
+  - You can keep Pikachu in your bag for later use.
+  - If you need to switch Pokémon or use Pikachu in another battle, this is a viable option.
+- **Cons:**
+  - You miss out on the opportunity to potentially defeat Pidgey and gain experience.
+  - If Pidgey is a rare or special Pokémon, bagging it might not be ideal.
+
+---
+
+### **3. RUN**
+**Explanation:**
+- You choose to **Run** (escape the battle).
+- **Pros:**
+  - You avoid losing HP and experience points.
+  - You can continue your journey without facing a weaker opponent.
+- **Cons:**
+  - You might miss out on a chance to level up Pikachu or gain experience.
+  - If you are in a competitive setting, running might not be the best strategy.
+
+---
+
+### **Summary of Best Options:**
+- **If you want to continue the game and gain experience:**
+  - **Fight** Pikachu, but be prepared for a potentially quick loss.
+- **If you want to save Pikachu for later:**
+  - **Bag** Pikachu and continue your journey.
+- **If you want to avoid unnecessary battles:**
+  - **Run** away from the battle.
+
+Since Pikachu is significantly stronger, **Fighting** is the most straightforward choice, but you might want to consider **Bagging** Pikachu if you want to keep it for later battles. If you're not in a rush, **Running** is also a viable option.
+In this *Pokémon* game screenshot, you are presented with a battle scenario between **Pidgey** and **Pikachu**. Here are the possible actions along with an analysis of their potential outcomes:
+
+---
+
+### **1. FIGHT**
+**Explanation:**
+- You choose to engage in battle.
+- Pidgey is a **Level 17** Pokémon with basic stats, while Pikachu is a **Level 42** Pokémon with significantly higher HP, Attack, and overall power.
+- **Pros:**
+  - Pikachu is stronger and likely to win if it lands a hit.
+  - Pidgey may be able to land a few moves before being overwhelmed.
+- **Cons:**
+  - Pidgey is likely to lose quickly due to its lower stats and experience.
+  - Pikachu may have a high chance of winning in one or two turns.
+
+---
+
+### **2. BAG**
+**Explanation:**
+- You choose to **Bag** Pikachu.
+- **Pros:**
+  - You can keep Pikachu in your bag for later use.
+  - If you need to switch Pokémon or use Pikachu in another battle, this is a viable option.
+- **Cons:**
+  - You miss out on the opportunity to potentially defeat Pidgey and gain experience.
+  - If Pidgey is a rare or special Pokémon, bagging it might not be ideal.
+
+---
+
+### **3. RUN**
+**Explanation:**
+- You choose to **Run** (escape the battle).
+- **Pros:**
+  - You avoid losing HP and experience points.
+  - You can continue your journey without facing a weaker opponent.
+- **Cons:**
+  - You might miss out on a chance to level up Pikachu or gain experience.
+  - If you are in a competitive setting, running might not be the best strategy.
+
+---
+
+### **Summary of Best Options:**
+- **If you want to continue the game and gain experience:**
+  - **Fight** Pikachu, but be prepared for a potentially quick loss.
+- **If you want to save Pikachu for later:**
+  - **Bag** Pikachu and continue your journey.
+- **If you want to avoid unnecessary battles:**
+  - **Run** away from the battle.
+
+Since Pikachu is significantly stronger, **Fighting** is the most straightforward choice, but you might want to consider **Bagging** Pikachu if you want to keep it for later battles. If you're not in a rush, **Running** is also a viable option.
+```
+
+</details>
 
-TODO
 
 ## License
 

config.json

@@ -88,6 +88,11 @@
   },
   "vision_feature_layer": -1,
   "transformers.js_config": {
+    "dtype": {
+      "decoder_model_merged": "q4f16",
+      "vision_encoder": "fp16",
+      "embed_tokens": "fp16"
+    },
     "use_external_data_format": {
       "embed_tokens.onnx": 1,
       "embed_tokens_fp16.onnx": 1,