Clinical Artificial Intelligence
The Clinical Artificial Intelligence Group at TU Dresden, led by Prof. Dr. Jakob Nikolas Kather, is an interdisciplinary team dedicated to advancing precision oncology through the development and application of deep learning models.
Our group, which has affiliations with NCT/UCC and the Else-Kröner-Fresenius Center of Digital Health, uses the latest advancements in artificial intelligence (AI) to transform unstructured data like images and text into structured, clinically relevant information.
Read more about the group on:
Contact
Prof. Dr. Jakob Nikolas Kather, MSc
Head of the Group
Clinical Artificial Intelligence
Phone: +49 (0) 351 458 7558
Email: ekfz(at)tu-dresden.de
www.kather.ai
Group
Our group has long-standing expertise in computer vision, specifically in digital histopathology and radiology image analysis. We employ weakly supervised prediction methods to directly extract clinically actionable insights from raw image data, bypassing manual intermediary steps. Utilizing self-supervised learning on extensive datasets, we develop foundation models to identify subtle patterns in images, which are subsequently fine-tuned on clinically relevant downstream tasks. Our approach, predominantly using vision transformers, has successfully predicted genetic alterations, responses to immunotherapy, and patient prognosis directly from routine histopathology slides or from radiology imaging data.
Our team also integrates large language models to process unstructured clinical text data. By embedding these models into the oncology research workflow, we extract structured data from radiology, pathology, and clinical reports. This approach not only unlocks the value of unstructured data but also minimizes inaccuracies, providing a reliable tool for interpreting clinical text records or oncology knowledge stored in clinical guidelines.
Looking ahead, we are focusing on multimodal data analysis, combining various data types like immunohistochemistry images, genomic data, and clinical text. This emerging research area aims to develop holistic clinical decision support tools that transcend single data types and clinical scenarios, enhancing real-world applicability in diverse medical settings.
Neidlinger P, El Nahhas OSM, Muti HS, Lenz T, Hoffmeister M, Brenner H, van Treeck M, Langer R, Dislich B, Behrens HM, Röcken C, Foersch S, Truhn D, Marra A, Saldanha OL, Kather JN. Benchmarking foundation models as feature extractors for weakly supervised computational pathology. Nat Biomed Eng. 2025 Oct 1. doi: 10.1038/s41551-025-01516-3. Epub ahead of print. PMID: 41034516.
Wiest IC, Wolf F, Leßmann ME, van Treeck M, Ferber D, Zhu J, Boehme H, Bressem KK, Ulrich H, Ebert MP, Kather JN. A software pipeline for medical information extraction with large language models, open source and suitable for oncology. NPJ Precis Oncol. 2025 Sep 17;9(1):313. doi: 10.1038/s41698-025-01103-4. PMID: 40962856; PMCID: PMC12443949.
Loeffler CML, Bando H, Sainath S, Muti HS, Jiang X, van Treeck M, Reitsam NG, Carrero ZI, Meneghetti AR, Nishikawa T, Misumi T, Mishima S, Kotani D, Taniguchi H, Takemasa I, Kato T, Oki E, Tanwei Y, Durgesh W, Foersch S, Brenner H, Hoffmeister M, Nakamura Y, Yoshino T, Kather JN. HIBRID: histology-based risk-stratification with deep learning and ctDNA in colorectal cancer. Nat Commun. 2025 Aug 14;16(1):7561. doi: 10.1038/s41467-025-62910-8. PMID: 40813777; PMCID: PMC12354865.
Veldhuizen GP, Lenz T, Cifci D, van Treeck M, Clusmann J, Chen Y, Schneider CV, Luedde T, de Leeuw PW, El-Armouche A, Truhn D, Kather JN. Deep learning can predict cardiovascular events from liver imaging. JHEP Rep. 2025 Apr 22;7(8):101427. doi: 10.1016/j.jhepr.2025.101427. PMID: 40671834; PMCID: PMC12260415.
Meneghetti AR, Hernández ML, Kühn JP, Löck S, Carrero ZI, Perez-Lopez R, Bressem KK, Brinker TJ, Pearson AT, Truhn D, Nebelung S, Kather JN. End-to-end prediction of clinical outcomes in head and neck squamous cell carcinoma with foundation model-based multiple instance learning. BMC Artif Intell. 2025;1(1):3. doi: 10.1186/s44398-025-00003-8. Epub 2025 Jun 24. PMID: 40605934; PMCID: PMC12212421.
Boehm KM, El Nahhas OSM, Marra A, Waters M, Jee J, Braunstein L, Schultz N, Selenica P, Wen HY, Weigelt B, Paul ED, Cekan P, Erber R, Loeffler CML, Guerini-Rocco E, Fusco N, Frascarelli C, Mane E, Munzone E, Dellapasqua S, Zagami P, Curigliano G, Razavi P, Reis-Filho JS, Pareja F, Chandarlapaty S, Shah SP, Kather JN. Multimodal histopathologic models stratify hormone receptor-positive early breast cancer. Nat Commun. 2025 Mar 2;16(1):2106. doi: 10.1038/s41467-025-57283-x. PMID: 40025017; PMCID: PMC11873197.
Gustav M, van Treeck M, Reitsam NG, Carrero ZI, Loeffler CML, Meneghetti AR, Märkl B, Boardman LA, French AJ, Goode EL, Gsur A, Brezina S, Gunter MJ, Murphy N, Hönscheid P, Sperling C, Foersch S, Steinfelder R, Harrison T, Peters U, Phipps A, Kather JN. Assessing Genotype-Phenotype Correlations with Deep Learning in Colorectal Cancer: A Multi-Centric Study. medRxiv [Preprint]. 2025 Feb 8:2025.02.04.25321660. doi: 10.1101/2025.02.04.25321660. Update in: Lancet Digit Health. 2025 Aug;7(8):100891. doi: 10.1016/j.landig.2025.100891. PMID: 39973981; PMCID: PMC11838662.
Clusmann J, Ferber D, Wiest IC, Schneider CV, Brinker TJ, Foersch S, Truhn D, Kather JN. Prompt injection attacks on vision language models in oncology. Nat Commun. 2025 Feb 1;16(1):1239. doi: 10.1038/s41467-024-55631-x. PMID: 39890777; PMCID: PMC11785991.
Loeffler CML, El Nahhas OSM, Muti HS, Carrero ZI, Seibel T, van Treeck M, Cifci D, Gustav M, Bretz K, Gaisa NT, Lehmann KV, Leary A, Selenica P, Reis-Filho JS, Ortiz-Bruechle N, Kather JN. Prediction of homologous recombination deficiency from routine histology with attention-based multiple instance learning in nine different tumor types. BMC Biol. 2024 Oct 8;22(1):225. doi: 10.1186/s12915-024-02022-9. PMID: 39379982; PMCID: PMC11462727.
Wiest IC, Ferber D, Zhu J, van Treeck M, Meyer SK, Juglan R, Carrero ZI, Paech D, Kleesiek J, Ebert MP, Truhn D, Kather JN. Privacy-preserving large language models for structured medical information retrieval. NPJ Digit Med. 2024 Sep 20;7(1):257. doi: 10.1038/s41746-024-01233-2. PMID: 39304709; PMCID: PMC11415382.
El Nahhas OSM, van Treeck M, Wölflein G, Unger M, Ligero M, Lenz T, Wagner SJ, Hewitt KJ, Khader F, Foersch S, Truhn D, Kather JN. From whole-slide image to biomarker prediction: end-to-end weakly supervised deep learning in computational pathology. Nat Protoc. 2025 Jan;20(1):293-316. doi: 10.1038/s41596-024-01047-2. Epub 2024 Sep 16. PMID: 39285224.
Gustav M, Reitsam NG, Carrero ZI, Loeffler CML, van Treeck M, Yuan T, West NP, Quirke P, Brinker TJ, Brenner H, Favre L, Märkl B, Stenzinger A, Brobeil A, Hoffmeister M, Calderaro J, Pujals A, Kather JN. Deep learning for dual detection of microsatellite instability and POLE mutations in colorectal cancer histopathology. NPJ Precis Oncol. 2024 May 23;8(1):115. doi: 10.1038/s41698-024-00592-z. PMID: 38783059; PMCID: PMC11116442.
Hewitt KJ, Löffler CML, Muti HS, Berghoff AS, Eisenlöffel C, van Treeck M, Carrero ZI, El Nahhas OSM, Veldhuizen GP, Weil S, Saldanha OL, Bejan L, Millner TO, Brandner S, Brückmann S, Kather JN. Direct image to subtype prediction for brain tumors using deep learning. Neurooncol Adv. 2023 Nov 1;5(1):vdad139. doi: 10.1093/noajnl/vdad139. PMID: 38106649; PMCID: PMC10724115.
Opportunities and Collaboration
We offer opportunities for Bachelor, Master, PhD, and MD theses, and frequently have openings for research software engineers. We actively seek collaboration with clinicians and clinical investigators within the NCT network, aiming to integrate deep learning methods in translational research and clinical trials. Our ultimate goal is to develop AI software prototypes for evaluation in clinical studies and as predictive models across the NCT network.