-
MITI: SLAM Benchmark for Laparoscopic Surgery
Authors:
Regine Hartwig,
Daniel Ostler,
Jean-Claude Rosenthal,
Hubertus Feußner,
Dirk Wilhelm,
Dirk Wollherr
Abstract:
We propose a new benchmark for evaluating stereoscopic visual-inertial computer vision algorithms (SLAM/ SfM/ 3D Reconstruction/ Visual-Inertial Odometry) for minimally invasive surgical (MIS) interventions in the abdomen. Our MITI Dataset available at [https://mediatum.ub.tum.de/1621941] provides all the necessary data by a complete recording of a handheld surgical intervention at Research Hospit…
▽ More
We propose a new benchmark for evaluating stereoscopic visual-inertial computer vision algorithms (SLAM/ SfM/ 3D Reconstruction/ Visual-Inertial Odometry) for minimally invasive surgical (MIS) interventions in the abdomen. Our MITI Dataset available at [https://mediatum.ub.tum.de/1621941] provides all the necessary data by a complete recording of a handheld surgical intervention at Research Hospital Rechts der Isar of TUM. It contains multimodal sensor information from IMU, stereoscopic video, and infrared (IR) tracking as ground truth for evaluation. Furthermore, calibration for the stereoscope, accelerometer, magnetometer, the rigid transformations in the sensor setup, and time-offsets are available. We wisely chose a suitable intervention that contains very few cutting and tissue deformation and shows a full scan of the abdomen with a handheld camera such that it is ideal for testing SLAM algorithms. Intending to promote the progress of visual-inertial algorithms designed for MIS application, we hope that our clinical training dataset helps and enables researchers to enhance algorithms.
△ Less
Submitted 23 February, 2022;
originally announced February 2022.
-
Constrained Visual-Inertial Localization With Application And Benchmark in Laparoscopic Surgery
Authors:
Regine Hartwig,
Daniel Ostler,
Jean-Claude Rosenthal,
Hubertus Feußner,
Dirk Wilhelm,
Dirk Wollherr
Abstract:
We propose a novel method to tackle the visual-inertial localization problem for constrained camera movements. We use residuals from the different modalities to jointly optimize a global cost function. The residuals emerge from IMU measurements, stereoscopic feature points, and constraints on possible solutions in SE(3). In settings where dynamic disturbances are frequent, the residuals reduce the…
▽ More
We propose a novel method to tackle the visual-inertial localization problem for constrained camera movements. We use residuals from the different modalities to jointly optimize a global cost function. The residuals emerge from IMU measurements, stereoscopic feature points, and constraints on possible solutions in SE(3). In settings where dynamic disturbances are frequent, the residuals reduce the complexity of the problem and make localization feasible. We verify the advantages of our method in a suitable medical use case and produce a dataset capturing a minimally invasive surgery in the abdomen. Our novel clinical dataset MITI is comparable to state-of-the-art evaluation datasets, contains calibration and synchronization and is available at https://mediatum.ub.tum.de/1621941.
△ Less
Submitted 22 February, 2022;
originally announced February 2022.
-
Surgical Data Science -- from Concepts toward Clinical Translation
Authors:
Lena Maier-Hein,
Matthias Eisenmann,
Duygu Sarikaya,
Keno März,
Toby Collins,
Anand Malpani,
Johannes Fallert,
Hubertus Feussner,
Stamatia Giannarou,
Pietro Mascagni,
Hirenkumar Nakawala,
Adrian Park,
Carla Pugh,
Danail Stoyanov,
Swaroop S. Vedula,
Kevin Cleary,
Gabor Fichtinger,
Germain Forestier,
Bernard Gibaud,
Teodor Grantcharov,
Makoto Hashizume,
Doreen Heckmann-Nötzel,
Hannes G. Kenngott,
Ron Kikinis,
Lars Mündermann
, et al. (25 additional authors not shown)
Abstract:
Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applica…
▽ More
Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.
△ Less
Submitted 30 July, 2021; v1 submitted 30 October, 2020;
originally announced November 2020.
-
TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks
Authors:
Tobias Czempiel,
Magdalini Paschali,
Matthias Keicher,
Walter Simson,
Hubertus Feussner,
Seong Tae Kim,
Nassir Navab
Abstract:
Automatic surgical phase recognition is a challenging and crucial task with the potential to improve patient safety and become an integral part of intra-operative decision-support systems. In this paper, we propose, for the first time in workflow analysis, a Multi-Stage Temporal Convolutional Network (MS-TCN) that performs hierarchical prediction refinement for surgical phase recognition. Causal,…
▽ More
Automatic surgical phase recognition is a challenging and crucial task with the potential to improve patient safety and become an integral part of intra-operative decision-support systems. In this paper, we propose, for the first time in workflow analysis, a Multi-Stage Temporal Convolutional Network (MS-TCN) that performs hierarchical prediction refinement for surgical phase recognition. Causal, dilated convolutions allow for a large receptive field and online inference with smooth predictions even during ambiguous transitions. Our method is thoroughly evaluated on two datasets of laparoscopic cholecystectomy videos with and without the use of additional surgical tool information. Outperforming various state-of-the-art LSTM approaches, we verify the suitability of the proposed causal MS-TCN for surgical phase recognition.
△ Less
Submitted 24 March, 2020;
originally announced March 2020.
-
Surgical Data Science: A Consensus Perspective
Authors:
Lena Maier-Hein,
Matthias Eisenmann,
Carolin Feldmann,
Hubertus Feussner,
Germain Forestier,
Stamatia Giannarou,
Bernard Gibaud,
Gregory D. Hager,
Makoto Hashizume,
Darko Katic,
Hannes Kenngott,
Ron Kikinis,
Michael Kranzfelder,
Anand Malpani,
Keno März,
Beat Müuller-Stich,
Nassir Navab,
Thomas Neumuth,
Nicolas Padoy,
Adrian Park,
Carla Pugh,
Nicolai Schoch,
Danail Stoyanov,
Russell Taylor,
Martin Wagner
, et al. (3 additional authors not shown)
Abstract:
Surgical data science is a scientific discipline with the objective of improving the quality of interventional healthcare and its value through capturing, organization, analysis, and modeling of data. The goal of the 1st workshop on Surgical Data Science was to bring together researchers working on diverse topics in surgical data science in order to discuss existing challenges, potential standards…
▽ More
Surgical data science is a scientific discipline with the objective of improving the quality of interventional healthcare and its value through capturing, organization, analysis, and modeling of data. The goal of the 1st workshop on Surgical Data Science was to bring together researchers working on diverse topics in surgical data science in order to discuss existing challenges, potential standards and new research directions in the field. Inspired by current open space and think tank formats, it was organized in June 2016 in Heidelberg. While the first day of the workshop, which was dominated by interactive sessions, was open to the public, the second day was reserved for a board meeting on which the information gathered on the public day was processed by (1) discussing remaining open issues, (2) deriving a joint definition for surgical data science and (3) proposing potential strategies for advancing the field. This document summarizes the key findings.
△ Less
Submitted 8 June, 2018;
originally announced June 2018.
-
Quantitative Characterization of Components of Computer Assisted Interventions
Authors:
Asli Okur,
Ralf Stauder,
Hubertus Feussner,
Nassir Navab
Abstract:
Purpose: We propose a mathematical framework for quantitative analysis weighting the impact of heterogeneous components of a surgery. While multi-level approaches, surgical process modeling and other workflow analysis methods exist, this is to our knowledge the first quantitative approach. Methods: Inspired by the group decision making problem from the field of operational research, we define even…
▽ More
Purpose: We propose a mathematical framework for quantitative analysis weighting the impact of heterogeneous components of a surgery. While multi-level approaches, surgical process modeling and other workflow analysis methods exist, this is to our knowledge the first quantitative approach. Methods: Inspired by the group decision making problem from the field of operational research, we define event impact factors, which combine independent and very diverse low-level functions. This allows us to rate surgical events by their importance. Results: We conducted surveys with 4 surgeons to determine the importance of roles, phases and their combinations within a laparoscopic cholecystectomy. Applying this data on a recorded surgery, we showed that it is possible to define a quantitative measure for deciding on acception or rejection of calls to different roles and at different phases of surgery. Conclusions: This methodology allows us to use components such as expertise and role of the surgical staff and other aspects of a given surgery in order to quantitatively analyze and evaluate events, actions, user interfaces or procedures.
△ Less
Submitted 2 February, 2017;
originally announced February 2017.
-
Surgical Data Science: Enabling Next-Generation Surgery
Authors:
Lena Maier-Hein,
Swaroop Vedula,
Stefanie Speidel,
Nassir Navab,
Ron Kikinis,
Adrian Park,
Matthias Eisenmann,
Hubertus Feussner,
Germain Forestier,
Stamatia Giannarou,
Makoto Hashizume,
Darko Katic,
Hannes Kenngott,
Michael Kranzfelder,
Anand Malpani,
Keno März,
Thomas Neumuth,
Nicolas Padoy,
Carla Pugh,
Nicolai Schoch,
Danail Stoyanov,
Russell Taylor,
Martin Wagner,
Gregory D. Hager,
Pierre Jannin
Abstract:
This paper introduces Surgical Data Science as an emerging scientific discipline. Key perspectives are based on discussions during an intensive two-day international interactive workshop that brought together leading researchers working in the related field of computer and robot assisted interventions. Our consensus opinion is that increasing access to large amounts of complex data, at scale, thro…
▽ More
This paper introduces Surgical Data Science as an emerging scientific discipline. Key perspectives are based on discussions during an intensive two-day international interactive workshop that brought together leading researchers working in the related field of computer and robot assisted interventions. Our consensus opinion is that increasing access to large amounts of complex data, at scale, throughout the patient care process, complemented by advances in data science and machine learning techniques, has set the stage for a new generation of analytics that will support decision-making and quality improvement in interventional medicine. In this article, we provide a consensus definition for Surgical Data Science, identify associated challenges and opportunities and provide a roadmap for advancing the field.
△ Less
Submitted 31 January, 2017; v1 submitted 23 January, 2017;
originally announced January 2017.
-
The TUM LapChole dataset for the M2CAI 2016 workflow challenge
Authors:
Ralf Stauder,
Daniel Ostler,
Michael Kranzfelder,
Sebastian Koller,
Hubertus Feußner,
Nassir Navab
Abstract:
In this technical report we present our collected dataset of laparoscopic cholecystectomies (LapChole). Laparoscopic videos of a total of 20 surgeries were recorded and annotated with surgical phase labels, of which 15 were randomly pre-determined as training data, while the remaining 5 videos are selected as test data. This dataset was later included as part of the M2CAI 2016 workflow detection c…
▽ More
In this technical report we present our collected dataset of laparoscopic cholecystectomies (LapChole). Laparoscopic videos of a total of 20 surgeries were recorded and annotated with surgical phase labels, of which 15 were randomly pre-determined as training data, while the remaining 5 videos are selected as test data. This dataset was later included as part of the M2CAI 2016 workflow detection challenge during MICCAI 2016 in Athens.
△ Less
Submitted 31 August, 2017; v1 submitted 28 October, 2016;
originally announced October 2016.
