Abstract
Traditional control and task automation have been successfully demonstratedin a variety of structured, controlled environments through the use of highlyspecialized modeled robotic systems in conjunction with multiple sensors.However, application of autonomy in endoscopic surgery is very challenging,particularly in soft tissue work, due to the lack of high-quality images andthe unpredictable, constantly deforming environment. In this work, we propose anovel surgical perception framework, SuPer, for surgical robotic control. Thisframework continuously collects 3D geometric information that allows formapping of a deformable surgical field while tracking rigid instruments withinthe field. To achieve this, a model-based tracker is employed to localize thesurgical tool with a kinematic prior in conjunction with a model-free trackerto reconstruct the deformable environment and provide an estimated point cloudas a mapping of the environment. The proposed framework was implemented on theda Vinci Surgical System in real-time with an end-effector controller where thetarget configurations are set and regulated through the framework. Our proposedframework successfully completed autonomous soft tissue manipulation tasks withhigh accuracy. The demonstration of this novel framework is promising for thefuture of surgical autonomy. In addition, we provide our dataset for furthersurgical research.