Abstract
With the development of multimodal reasoning models, Computer Use Agents(CUAs), akin to Jarvis from \textit{"Iron Man"}, are becoming a reality. GUIgrounding is a core component for CUAs to execute actual actions, similar tomechanical control in robotics, and it directly leads to the success or failureof the system. It determines actions such as clicking and typing, as well asrelated parameters like the coordinates for clicks. Current end-to-endgrounding models still achieve less than 65\% accuracy on challengingbenchmarks like ScreenSpot-pro and UI-Vision, indicating they are far frombeing ready for deployment. % , as a single misclick can result in unacceptableconsequences. In this work, we conduct an empirical study on the training ofgrounding models, examining details from data collection to model training.Ultimately, we developed the \textbf{Phi-Ground} model family, which achievesstate-of-the-art performance across all five grounding benchmarks for modelsunder $10B$ parameters in agent settings. In the end-to-end model setting, ourmodel still achieves SOTA results with scores of \textit{\textbf{43.2}} onScreenSpot-pro and \textit{\textbf{27.2}} on UI-Vision. We believe that thevarious details discussed in this paper, along with our successes and failures,not only clarify the construction of grounding models but also benefit otherperception tasks. Project homepage:\href{https://zhangmiaosen2000.github.io/Phi-Ground/}{https://zhangmiaosen2000.github.io/Phi-Ground/}