Projects

Projects Description

Modern ML needs large, diverse datasets, but in Surgical Data Science data are scarce, heterogeneous, and rare complications are underrepresented. Training algorithms, robots, and trainees therefore requires datasets that cover edge cases and represent patients fairly. Simulation can expand data, yet typical simulators model only basic scenarios.

SIMSURGE will semi-automatically generate large volumes of diverse, realistic simulations of rare surgical situations. We combine controllable physical simulation (with ground truth for every scene) and vision–language models to diversify content, scale case numbers, and balance datasets. We will evaluate the system via studies on data bias, reinforcement-learning tasks for automation, and user studies with surgeons and medical students.

While high-risk, the approach promises low-cost, high-fidelity synthetic data with major impact on computer-assisted intervention, surgical robotics, and data-driven medicine.

Amid the rapid progress in the field of Artificial Intelligence (AI), serious issues with computing, IT infrastructures, and networked systems have become increasingly known worldwide in recent years. These can severely limit the further development of AI and future technologies based on it—particularly in communications, medicine, and robotics—or bring them to a halt in the event of energy supply issues, as AI applications require enormous amounts of energy.

Renowned scientists from the Technical University of Dresden (TUD), Ludwig Maximilian University of Munich (LMU), and Technical University of Munich (TUM) now address this issue together as part of the pilot project "GAIn" (Next Generation AI Computing). The development of new AI hardware and software concepts within the GAIn project is of great importance for our group Department of Translational Surgical Oncology (TSO) to enable further progress in medical AI and robotics research. By reducing energy consumption and increasing the stability of AI systems, we can ensure that surgical assistance systems work more precisely and efficiently. These advances mean that surgical procedures can be carried out more safely and targeted, ultimately benefiting the patients.
More information about the GAIn project can be found on the website of TU Dresden.                                                

A robust robotics industry will form the basis for a flourishing society and economy in the coming years. The Robotics Institute Germany (RIG) is set to become a driving force for German robotics in order to meet challenges such as labour shortages, demographic change, and climate change.

As part of this project, TU Dresden focuses on the development and implementation of innovative educational modules and student research and incubator programs within the RIG educational program. Additionally, the establishment of a medical robotics cluster and participation in other AI-based robotics clusters within the RIG is planned. TU Dresden also supports the implementation of outreach concepts to increase the visibility of the RIG.

A key focus is the provision of robotics kits and interactive educational modules. Furthermore, teacher training sessions for general education schools are planned to fully harness the educational potential. The promotion of student research projects, particularly in collaboration with industry, as well as the design and establishment of an incubator program to support spin-offs at TU Dresden, complete this comprehensive initiative.

The project also includes the conceptualization of a medical robotics cluster together with partners within the RIG, as well as the further development of the medical robotics testbed at TU Dresden, which is to be expanded as a testing platform within the RIG.

Outreach activities include integrating initiatives such as CeTI and 6G-life, contributing to the visual identity of the RIG, and providing best practice examples for effective communication and branding.

Proof of Concept for adaptive meshing under topology changes using neural networks: In the event of a positive outcome regarding feasibility, the method should be validated with data that is as close to real-world conditions as possible. The code and API should reach the maturity level of a prototype, enabling further development of the method as a component for future products or integration into existing software. In this context, "code" refers to the code for the neural network. Additionally, a visualization of the remeshing process by the network is planned for demonstration purposes. The code for integration with the simulation software used, the training data, and the evaluation data are considered valuable interim results.

The Surgical AI Hub Germany project aims to measure and improve surgical quality using AI. By establishing a data-centric technology platform, we will significantly accelerate the development of AI methods in surgery, overcoming legal, organizational, and technological barriers. Objectives include creating a demonstrator platform for surgical data handling, providing a global platform for surgeons to measure and improve surgery quality, collecting surgery videos for AI training, building a European surgeon network for AI research, developing business models for SMEs, establishing best practices for data sharing, and researching new AI concepts in surgery. This project will connect surgical excellence from renowned university hospitals, key digital technologies from German companies, and the latest insights from leading scientists. It will facilitate the scalability of innovative digital business models and services in surgery, ultimately enhancing surgical quality and outcomes globally.

The School of Embedded Composite Artificial Intelligence (SECAI) is a project of TU Dresden and Leipzig University to promote artificial intelligence (AI) in research and education. SECAI awards scholarships, strengthens teaching, funds researchers and promotes international exchange.

CoBot 2.0 (2022 - 2026)
Machine learning-based surgical guidance system for robot-assisted rectal surgery – a first-in-human interventional study (CoBot 2.0)

The CoBot 2.0, the follow-up project to CoBot, is a pioneering proof-of-concept trial and the first-in-human interventional study aiming to evaluate the effectiveness of a machine learning-based CoBot surgical guidance system in robot-assisted rectal surgery. This groundbreaking study focuses on improving surgical outcomes by leveraging advanced AI technologies to enhance precision during oncological rectal resections, which often lead to complications like sexual dysfunction and incontinence due to nerve damage. The CoBot system aims to recognize and preserve critical anatomical structures, reducing these risks. The objectives of the CoBot 2.0 project include reducing postoperative complications by enhancing anatomical recognition during surgery, evaluating and improving the usability of AI-guided systems in robot-assisted rectal surgery, and setting a precedent for integrating AI in complex surgical procedures from preclinical to clinical stages. This study could revolutionize surgical practices, leading to better patient outcomes and setting new standards for AI-assisted surgery.

NAIV (2022 - 2026)
3D Navigation for Intraoperative Visualization of Tumor NAIV

In higher tumor stages of prostate carcinoma, one treatment option is complete surgical removal of the prostate. With this operation, affected men run the risk of postoperative incontinence as well as limitations in potency. To reduce this risk, it is possible to spare the responsible nerves that run on the prostate capsule. However, with this nerve-sparing operation, there is a risk that tumor parts remain and the resection is not performed in healthy tissue. Therefore, it is essential to remove the tumor completely so that no tumor cells remain at the incision margin. Intraoperative rapid incisions of the sedimentation margins increase oncologic safety, but are often associated with increased blood loss, longer operative time, and increased costs.

The 6G-life Transfer Hub bridges cutting-edge 6G communication research and real-world applications through the Universities of Excellence in Dresden and Munich. In close collaboration with industry partners and the incubator launchhub42, the initiative drives the development and deployment of next-generation communication technologies for cooperative mobile robotics in industrial and medical settings. 

Within 6G-life, our group develops intelligent and connected robotic systems for computer-assisted surgery. Our research focuses on AI-based vision-language-action models surgical robotics, teleoperation and shared autonomy, surgical skill modeling with LLMs and VLMs, and efficient transmission of surgical data over future 6G networks. We also develop foundation models for multimodal surgical scene understanding and federated learning methods for privacy-preserving clinical collaboration, enabling trustworthy next-generation healthcare robotics.

The Centre for Tactile Internet with Human-in-the-Loop (CeTI) is a Cluster of Excellence at the TU Dresden. CeTI aims to enable real-time interaction between humans and cyber-physical systems through intelligent communication networks. By combining expertise in communication technologies, artificial intelligence, robotics, sensing, and control, CeTI develops the foundations for future applications in medicine, industry, and the Internet of Skills.

Within CeTI, our group contributes to AI-driven computer-assisted surgery and robotics in healthcare. Prof. Stefanie Speidel serves as co-speaker of the AI and robotics in healthcare research area. Our research focuses on developing intelligent surgical systems that tightly integrate artificial intelligence and robotics to enable context-aware assistance in the operating room. We investigate methods for perception, scene understanding, and decision support in surgical environments, as well as robotic assistance strategies that improve precision, safety, and efficiency in minimally invasive and robotic-assisted interventions. By combining AI, robotics, and real-time communication concepts, we aim to advance next-generation computer-assisted surgery within the CeTI framework.

Projects Description

The transCampus is a unique partnership established by King’s College London and Technische Universität Dresden as a transnational strategic partnership based on the idea of true cooperation and an intense dedication for collaboration in all fields. Within the framework of TransCampus, IRTG 2251: ICSMD aims at providing outstanding PhD students with a specialized but interdisciplinary programme that offers tandem supervision by two principal investigators and the opportunity to acquire certificates from both institutions. We will focus on the negative latency for shared-autonomy in remote surgery for metabolic diseases. Predicting tasks and events in surgery is crucial for remote surgery, which requires high data rates and low latency. Robotic devices and sensors in the operating room can create a compact, data-driven digital twin of the surgical scene, reducing data transmission. This digital twin also forms the basis for predicting events and workflows using machine learning, enabling shared autonomy by compensating for transmission latencies and ensuring synchronization with the actual operating room.

To help to materialize the European bid on a true continuum in the next few years, CloudSkin pursues to build a cognitive cloud continuum platform with three main innovations: 1. The CloudSkin platform will leverage AI/ML to optimize workloads, resources, energy, and network traffic between the cloud and the edge in the face of the rapid varying conditions; 2. The CloudSkin platform will also help users to achieve “stack identicality” across the Cloud-edge continuum; and 3. CloudSkin will also contribute to prepare the needed infrastructure to integrate the new virtualized execution abstractions into the virtual resource continuum assisted by the AI/ML-based orchestration plane in the CloudSkin platform.

The main goal is to design an Extreme near-data platform to enable consumption, mining and processing of distributed and federated data without needing to master the logistics of data access across heterogeneous data locations and pools. We go beyond traditional passive or bulk data ingested from storage systems towards next generation near-data processing platforms both in the Cloud and in the Edge. In our platform, Extreme Data includes both metadata and trustworthy data connectors enabling advanced data management operations like data discovery, mining, and filtering from heterogeneous data sources.

virTUos starts from the experience of developing digitally supported university teaching - largely insular solutions without anchoring in curricula and regulations - and innovates them according to the following guidelines for action: 1)Hybrid teaching and learning is a pillar of a collaborative university development strategy of the TUD, which is advanced in the fields of action Internships & Assessment Formats, Collaboration & Internationalization, and Competence Development & Open Teaching. 2)Novel teaching and learning scenarios are continuously developed by two agile innovation teams (mechanical engineering, medicine, linguistics, economics), which take into account didactic as well as digital competencies. An integration team coordinates scaling/transfer and develops the interlocking of centralized and decentralized support structures along the requirements for study program development and examination law interpretation in the HYBRID strategy. 3)Students are not only users, but also partners and designers in the above-mentioned teams and the Steering Committee. 4)In virTUos, concepts of Open Teaching as Open Educational Practices are developed in an interdisciplinary way and further developed/used across universities.

Minimally invasive surgeries using endoscopes allow to reduce the necessary follow-up care as well as the restrictions for patients after surgical procedures. However, due to the limited visibility and lack of haptic feedback, these surgeries can be more challenging for physicians. Therefore, the goal of this project is to support surgeons during surgery by displaying information in the camera image.

Multiple conditions of the middle ear are leading to conductive hearing loss, e.g. chronic otitis media, cholesteatoma and otosclerosis, which is regularly investigated by differential diagnosis of several procedures. Current diagnostics cover only single aspects of the pathologies: Otoscopy provides a visual impression and tympanometry the compliance of the tympanic membrane, while audiometry evaluates the overall hearing performance. Nevertheless, all of those tools lack a determination of the leading origin and site of the transmission loss. Accordingly, direct otologic diagnostics enabling the localisation and differentiation for targeted treatment and improved patient outcome are needed.

Colorectal cancer is the third most common cancer entity and the second most common cause of cancer death worldwide. Curative treatment approaches always involve surgical excision of the primary tumor and locoregional lymph nodes. About half of all colorectal cancers occur in the rectum, necessitating oncologically radical resection of the primary tumor within the mesorectal envelope. Narrow and deep pelvic anatomy and the risk of injury to adjacent anatomical structures such as ureters and pelvic nerves responsible for urogenital functions make total mesorectal excision (TME) a particularly complex surgical procedure. These challenges could potentially be overcome by an integration of smart assistance functions into robotic surgical systems.

The aim of the project "SurgOmics" is to improve decision support in hospitals before, during and after surgical procedures. During surgical treatment, a system is designed to support the decisions of the attending physicians by analyzing clinical data in real time and incorporating the latest study findings. Within the framework of SurgOmics, surgical processes are to be modeled with the help of informatic methods. In addition to the data collected during the operation, data before and after the operation will also be included.