-
AutoCam: Hierarchical Path Planning for an Autonomous Auxiliary Camera in Surgical Robotics
Authors:
Alexandre Banks,
Randy Moore,
Sayem Nazmuz Zaman,
Alaa Eldin Abdelaal,
Septimiu E. Salcudean
Abstract:
Incorporating an autonomous auxiliary camera into robot-assisted minimally invasive surgery (RAMIS) enhances spatial awareness and eliminates manual viewpoint control. Existing path planning methods for auxiliary cameras track two-dimensional surgical features but do not simultaneously account for camera orientation, workspace constraints, and robot joint limits. This study presents AutoCam: an au…
▽ More
Incorporating an autonomous auxiliary camera into robot-assisted minimally invasive surgery (RAMIS) enhances spatial awareness and eliminates manual viewpoint control. Existing path planning methods for auxiliary cameras track two-dimensional surgical features but do not simultaneously account for camera orientation, workspace constraints, and robot joint limits. This study presents AutoCam: an automatic auxiliary camera placement method to improve visualization in RAMIS. Implemented on the da Vinci Research Kit, the system uses a priority-based, workspace-constrained control algorithm that combines heuristic geometric placement with nonlinear optimization to ensure robust camera tracking. A user study (N=6) demonstrated that the system maintained 99.84% visibility of a salient feature and achieved a pose error of 4.36 $\pm$ 2.11 degrees and 1.95 $\pm$ 5.66 mm. The controller was computationally efficient, with a loop time of 6.8 $\pm$ 12.8 ms. An additional pilot study (N=6), where novices completed a Fundamentals of Laparoscopic Surgery training task, suggests that users can teleoperate just as effectively from AutoCam's viewpoint as from the endoscope's while still benefiting from AutoCam's improved visual coverage of the scene. These results indicate that an auxiliary camera can be autonomously controlled using the da Vinci patient-side manipulators to track a salient feature, laying the groundwork for new multi-camera visualization methods in RAMIS.
△ Less
Submitted 15 May, 2025;
originally announced May 2025.
-
Latent Space Inference For Spatial Transcriptomics
Authors:
J. Ding,
S. N. Zaman,
P. Y. Chen,
D. Wang
Abstract:
In order to understand the complexities of cellular biology, researchers are interested in two important metrics: the genetic expression information of cells and their spatial coordinates within a tissue sample. However, state-of-the art methods, namely single-cell RNA sequencing and image based spatial transcriptomics can only recover a subset of this information, either full genetic expression w…
▽ More
In order to understand the complexities of cellular biology, researchers are interested in two important metrics: the genetic expression information of cells and their spatial coordinates within a tissue sample. However, state-of-the art methods, namely single-cell RNA sequencing and image based spatial transcriptomics can only recover a subset of this information, either full genetic expression with loss of spatial information, or spatial information with loss of resolution in sequencing data. In this project, we investigate a probabilistic machine learning method to obtain the full genetic expression information for tissues samples while also preserving their spatial coordinates. This is done through mapping both datasets to a joint latent space representation with the use of variational machine learning methods. From here, the full genetic and spatial information can be decoded and to give us greater insights on the understanding of cellular processes and pathways.
△ Less
Submitted 1 November, 2023;
originally announced November 2023.
-
Gaze-based Learning from Demonstration In Surgical Robotics
Authors:
A. E. Abdelaal,
S. N. Zaman,
P. Y Chen,
T. Suzuki,
J. Ingleton
Abstract:
Surgical robotics is a rising field in medical technology and advanced robotics. Robot assisted surgery, or robotic surgery, allows surgeons to perform complicated surgical tasks with more precision, automation, and flexibility than is possible for traditional surgical approaches. The main type of robot assisted surgery is minimally invasive surgery, which could be automated and result in a faster…
▽ More
Surgical robotics is a rising field in medical technology and advanced robotics. Robot assisted surgery, or robotic surgery, allows surgeons to perform complicated surgical tasks with more precision, automation, and flexibility than is possible for traditional surgical approaches. The main type of robot assisted surgery is minimally invasive surgery, which could be automated and result in a faster healing time for the patient. The surgical robot we are particularly interested in is the da Vinci surgical system, which is developed and manufactured by Intuitive Surgical. In the current iteration of the system, the endoscopic camera arm on the da Vinci robot has to be manually controlled and calibrated by the surgeon during a surgical task, which interrupts the flow of the operation. The main goal of this capstone project is to automate the motion of the camera arm using a probabilistic model based on surgeon eye gaze data and da Vinci robot kinematic data.
△ Less
Submitted 1 November, 2023;
originally announced November 2023.
-
The Quiet Eye Phenomenon in Minimally Invasive Surgery
Authors:
Alaa Eldin Abdelaal,
Rachelle Van Rumpt,
Sayem Nazmuz Zaman,
Irene Tong,
Anthony Jarc,
Gary L. Gallia,
Masaru Ishii,
Gregory D. Hager,
Septimiu E. Salcudean
Abstract:
In this paper, we report our discovery of a gaze behavior called Quiet Eye (QE) in minimally invasive surgery. The QE behavior has been extensively studied in sports training and has been associated with higher level of expertise in multiple sports. We investigated the QE behavior in two independently collected data sets of surgeons performing tasks in a sinus surgery setting and a robotic surgery…
▽ More
In this paper, we report our discovery of a gaze behavior called Quiet Eye (QE) in minimally invasive surgery. The QE behavior has been extensively studied in sports training and has been associated with higher level of expertise in multiple sports. We investigated the QE behavior in two independently collected data sets of surgeons performing tasks in a sinus surgery setting and a robotic surgery setting, respectively. Our results show that the QE behavior is more likely to occur in successful task executions and in performances of surgeons of high level of expertise. These results open the door to use the QE behavior in both training and skill assessment in minimally invasive surgery.
△ Less
Submitted 6 September, 2023;
originally announced September 2023.
