Abstract
In robot-assisted laparoscopic radical prostatectomy (RALP), the location ofthe instrument tip is important to register the ultrasound frame with thelaparoscopic camera frame. A long-standing limitation is that the instrumenttip position obtained from the da Vinci API is inaccurate and requires hand-eyecalibration. Thus, directly computing the position of the tool tip in thecamera frame using the vision-based method becomes an attractive solution.Besides, surgical instrument tip detection is the key component of other tasks,like surgical skill assessment and surgery automation. However, this task ischallenging due to the small size of the tool tip and the articulation of thesurgical instrument. Surgical instrument segmentation becomes relatively easydue to the emergence of the Segmentation Foundation Model, i.e., SegmentAnything. Based on this advancement, we explore the deep learning-basedsurgical instrument tip detection approach that takes the part-level instrumentsegmentation mask as input. Comparison experiments with a hand-craftedimage-processing approach demonstrate the superiority of the proposed method onsimulated and real datasets.