Our research goal is to develop and evaluate computer- and robotic-assisted systems which optimize the surgical therapy of the individual patient by turning the available data into useful information. Therefore, we are investigating the entire process chain along the surgical treatment path including pre-, intra- and postoperative patient information. The research and methodological focuses include the following areas:

The purpose of intraoperative navigation is to provide information of hidden risk and target structures based on preoperative patient planning data. The visualization can be done via Augmented Reality (AR), either by overlaying the information directly in the endoscopic view or by using glasses. In soft-tissue navigation, this process is substantially complicated due to the organ deformation, induced by forces applied to the organs, cutting of the tissue, different patient poses and the patient's breathing.


The estimation of organ-internal deformation and movement of risk/target structures from intraoperative sensor data (e.g. surgical video streams) and pre-operative data in real time is an open research topic. Our team uses novel machine learning techniques to tackle this registration problem.

Projects: OP4.1
Team: Micha Pfeiffer, Dr. Georges Hattab, Prof. Dr. Stefanie Speidel

During laparoscopic surgeries, stereo endoscopes allow computer vision algorithms to extract 3D information from the surgical scene. This information makes it possible to provide the surgeon with quantitative assistance functions. Examples are measuring distances along the surface of a given organ or determining perfusion speed of indiocyanine green contrast agents (ICG) as an indicator for blood flow. We provide the surgeons with quantitative real-time methods using computer vision and machine-learning methods.


Team: Dr. Sebastian Bodenstedt, Prof. Dr. Stefanie Speidel

Context-aware assistance systems provide the right information at the right time. Therefore the  state of surgery (e.g. the current surgical phase) has to be determined constantly. Here, we focus on analysing intraoperative sensor data (e.g. from the laparoscopic image stream, surgical devices, …) to locate indicators that correlate to the surgical workflow. For analysing surgical sensor data, we employ state of the art machine learning methods to extract semantic information such as instrument usage, organ of focus or surgical phase. An integral part of the approach is the use of ontologies to represent medical background knowledge.

@ C. Feldmann, DKFZ

Projects: OP4.1
Team: Dr. Sebastian Bodenstedt, Prof. Dr. Stefanie Speidel

In order to ensure high-quality patient care, it is crucial to train novice surgeons effectively and efficiently. This especially holds for challenging surgical techniques such as laparoscopic or robot-assisted surgery. For this reason, we investigate how to enhance conventional surgical training by means of modern technology. In particular, we use a variety of sensor modalities to perceive the surgeon’s activities and novel machine learning algorithms to analyze the collected sensor data. Our focus is the development of smart algorithms to provide automatic constructive feedback to the novice surgeon along with an automatic objective assessment of their surgical skill.


Team: Isabel Funke, Prof. Dr. Stefanie Speidel

A large and important part of the patient data available to physicians is three-dimensional. CT and MRI scans, for example, can be used to generate models of a patient’s data, which can in turn be used to plan an operation. However, this data is usually viewed on two-dimensional screens, resulting in the loss of depth perception and making it difficult to interpret.

Using modern Virtual Reality (VR) Technology, we are developing tools which allow physicians to view patient data in a virtual and three-dimensional environment. By building intuitive surgical VR applications, we aim to aid in surgery planning, medical data analysis, and in researching advantages, caveats and chances of medical VR.

Team: Micha Pfeiffer, Prof. Dr. Stefanie Speidel
Publications: M. Pfeiffer, H. Kenngott, A. Preukschas, M. Huber, L. Bettscheider, B. Müller-Stich, S. Speidel: "IMHOTEP: virtual reality framework for surgical applications", Int. J. Computer Assisted Radiology and Surgery 13(5): 741-748 (2018), DOI 10.1007/s11548-018-1730-x