fix readme

README.md

@@ -23,14 +23,12 @@ metrics:
 
 **GroundNext-7B-V0** is a state-of-the-art vision-language model for GUI element grounding, developed as part of the **GroundCUA** project. This model features:
 
-- **Superior grounding accuracy** achieving 48.9% on ScreenSpot-Pro, 55.6% on OSWorld-G, and 31.3% on UI-Vision benchmarks
-- **Exceptional cross-platform generalization** with 83.7% accuracy on MMBench-GUI and 92.8% on ScreenSpot-v2 despite desktop-only training
+- **Superior grounding accuracy** achieving 52.9% on ScreenSpot-Pro, 67.7% on OSWorld-G, and 60.3% on UI-Vision benchmarks
+- **Exceptional cross-platform generalization** with 81.1% accuracy on MMBench-GUI and 90.4% on ScreenSpot-v2 despite desktop-only training
 - **Data-efficient training** achieving state-of-the-art results with only 700K training examples vs 9M+ in prior work
 - **Strong agentic capabilities** reaching 50.6% overall success rate on OSWorld when paired with reasoning models
 - **Native tool-calling support** with built-in computer use action space for mouse, keyboard, and screen interactions
 
-![Performance Comparison](https://via.placeholder.com/800x400?text=GroundNext+Performance+Visualization)
-
 ## Model Overview
 
 **GroundNext-7B-V0** has the following characteristics:
@@ -53,15 +51,15 @@ For more details about the training methodology, dataset, and comprehensive benc
 | **ScreenSpot-Pro** | 29.7          | 38.1        | **52.9**          |
 | **OSWorld-G**      | 42.7          | 57.1        | **67.7**          |
 | **UI-Vision**      | 16.5          | 25.5        | **60.3**          |
-| **Avg**            | 29.6          | 40.2        | **60.3**          |
+| **Avg (Desktop)**  | 29.6          | 40.2        | **60.3**          |
 
 ### Cross-Platform Generalization (Desktop, Mobile & Web)
 
-|                      | Qwen2.5-VL-7B | UI-TARS-72B | **GroundNext-7B** |
+|                      | Qwen2.5-VL-7B | UI-TARS-72B | **GroundNext-7B-V0** |
 | -------------------- | ------------- | ----------- | ----------------- |
 | **MMBench-GUI**      | 33.9          | 74.3        | **81.1**          |
 | **ScreenSpot-v2**    | 88.8          | 90.3        | **90.4**          |
-| **Avg**              | 61.4          | 82.3        | **85.8**          |
+| **Avg (Mobile/Web)** | 61.4          | 82.3        | **85.8**          |
 
 
 ### Agentic Performance on OSWorld
@@ -98,31 +96,14 @@ The following code snippet demonstrates how to use GroundNext-7B-V0 for GUI elem
 
 ```python
 import torch
-from transformers import Qwen2VLForConditionalGeneration, AutoTokenizer, AutoProcessor
-from qwen_vl_utils.vision_process import smart_resize
+from transformers import Qwen2_5_VLForConditionalGeneration, AutoTokenizer, AutoProcessor
 from PIL import Image
-
-# System prompt for computer use grounding
-GROUNDNEXT_SYSTEM_PROMPT = """You are a helpful assistant.
-
-# Tools
-
-You may call one or more functions to assist with the user query.
-
-You are provided with function signatures within <tools></tools> XML tags:
-<tools>
-{{"type": "function", "function": {{"name": "computer_use", "description": "Use a mouse and keyboard to interact with a computer, and take screenshots.\n* This is an interface to a desktop GUI. You do not have access to a terminal or applications menu. You must click on desktop icons to start applications.\n* Some applications may take time to start or process actions, so you may need to wait and take successive screenshots to see the results of your actions. E.g. if you click on Firefox and a window doesn't open, try wait and taking another screenshot.\n* The screen's resolution is {width}x{height}.\n* Whenever you intend to move the cursor to click on an element like an icon, you should consult a screenshot to determine the coordinates of the element before moving the cursor.\n* If you tried clicking on a program or link but it failed to load, even after waiting, try adjusting your cursor position so that the tip of the cursor visually falls on the element that you want to click.\n* Make sure to click any buttons, links, icons, etc with the cursor tip in the center of the element. Don't click boxes on their edges unless asked.", "parameters": {{"properties": {{"action": {{"description": "The action to perform. The available actions are:\n* `key`: Performs key down presses on the arguments passed in order, then performs key releases in reverse order.\n* `type`: Type a string of text on the keyboard.\n* `mouse_move`: Move the cursor to a specified (x, y) pixel coordinate on the screen.\n* `left_click`: Click the left mouse button.\n* `left_click_drag`: Click and drag the cursor to a specified (x, y) pixel coordinate on the screen.\n* `right_click`: Click the right mouse button.\n* `middle_click`: Click the middle mouse button.\n* `double_click`: Double-click the left mouse button.\n* `scroll`: Performs a scroll of the mouse scroll wheel.\n* `wait`: Wait specified seconds for the change to happen.\n* `terminate`: Terminate the current task and report its completion status.", "enum": ["key", "type", "mouse_move", "left_click", "left_click_drag", "right_click", "middle_click", "double_click", "scroll", "wait", "terminate"], "type": "string"}}, "keys": {{"description": "Required only by `action=key`.", "type": "array"}}, "text": {{"description": "Required only by `action=type`.", "type": "string"}}, "coordinate": {{"description": "(x, y): The x (pixels from the left edge) and y (pixels from the top edge) coordinates to move the mouse to. Required only by `action=mouse_move`, `action=left_click_drag`, `action=left_click`, `action=right_click`, `action=double_click`.", "type": "array"}}, "pixels": {{"description": "The amount of scrolling to perform. Positive values scroll up, negative values scroll down. Required only by `action=scroll`.", "type": "number"}}, "time": {{"description": "The seconds to wait. Required only by `action=wait`.", "type": "number"}}, "status": {{"description": "The status of the task. Required only by `action=terminate`.", "type": "string", "enum": ["success", "failure"]}}}}, "required": ["action"], "type": "object"}}}}}}
-</tools>
-
-For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
-<tool_call>
-{{"name": <function-name>, "arguments": <args-json-object>}}
-</tool_call>"""
+import groundcua
 
 model_name = "ServiceNow/GroundNext-7B-V0"
 
 # Load model and processor
-model = Qwen2VLForConditionalGeneration.from_pretrained(
+model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
     model_name,
     torch_dtype=torch.bfloat16,
     attn_implementation="flash_attention_2",
@@ -134,67 +115,26 @@ processor = AutoProcessor.from_pretrained(model_name)
 tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
 
 # Configure generation
-model.generation_config.temperature = 0.0
+model.generation_config.temperature = groundcua.DEFAULT_TEMPERATURE
 model.generation_config.do_sample = False
 model.generation_config.use_cache = True
 
 # Load and prepare image
-image_path = "./screenshot.png"
-image = Image.open(image_path).convert('RGB')
-width, height = image.size
-
-# Resize image using smart_resize
-resized_height, resized_width = smart_resize(
-    height,
-    width,
-    min_pixels=78_400,
-    max_pixels=6_000_000,
-)
-image = image.resize((resized_width, resized_height))
-
-# Create messages
+url = "https://huggingface.co/datasets/ServiceNow/GroundCUA/resolve/main/images/LibreOffice Writer/00c4bac63f95985ccd9a4210fa752e8a5148a5f69ecb8bcfb3e499f5a3becc0d.png"
+image = Image.open(io.BytesIO(urlopen(url).read()))
+image, (width, height) = groundcua.prepare_image(image)
+
+# Create messages and generate
 instruction = "Click on the 'Save' icon"
-messages = [
-    {
-        "role": "system",
-        "content": GROUNDNEXT_SYSTEM_PROMPT.format(width=resized_width, height=resized_height)
-    },
-    {
-        "role": "user",
-        "content": [
-            {"type": "image", "image": image},
-            {"type": "text", "text": instruction},
-        ],
-    }
-]
-
-# Prepare inputs
-input_text = tokenizer.apply_chat_template(
-    messages,
-    add_generation_prompt=True,
-    tokenize=False
-)
-
-inputs = processor(
-    text=[input_text],
-    images=[image],
-    videos=None,
-    padding=True,
-    return_tensors="pt",
-).to(model.device)
-
-# Generate response
-generated_ids = model.generate(**inputs, max_new_tokens=128)
-generated_ids_trimmed = [
-    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
-]
-
-response = processor.batch_decode(
-    generated_ids_trimmed,
-    skip_special_tokens=True,
-    clean_up_tokenization_spaces=False
-)[0]
+messages = groundcua.create_messages(instruction, image, width, height)
+
+input_text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
+inputs = processor(text=[input_text], images=[image], videos=None, padding=True, return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(**inputs, max_new_tokens=groundcua.DEFAULT_MAX_NEW_TOKENS)
+generated_ids_trimmed = [out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
 
+response = processor.batch_decode(generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
 print(response)
 # Expected output: <tool_call>{"name": "computer_use", "arguments": {"action": "left_click", "coordinate": [x, y]}}</tool_call>
 ```