• Generating dermatopathology re...

Treffer: Generating dermatopathology reports from gigapixel whole slide images with HistoGPT.

Title:
Generating dermatopathology reports from gigapixel whole slide images with HistoGPT.
Authors:
Tran M; Helmholtz AI, Helmholtz Munich, Neuherberg, Germany.; School of Computation, Information and Technology, Technical University of Munich, Munich, Germany., Schmidle P; Department of Dermatology, Medical Center, University of Freiburg, Freiburg, Germany., Guo RR; Department of Laboratory Medicine and Pathology, Mayo Clinic, Jacksonville, FL, USA., Wagner SJ; Helmholtz AI, Helmholtz Munich, Neuherberg, Germany.; School of Computation, Information and Technology, Technical University of Munich, Munich, Germany., Koch V; School of Computation, Information and Technology, Technical University of Munich, Munich, Germany.; Institute of AI for Health, Helmholtz Munich, Neuherberg, Germany., Lupperger V; MLL Munich Leukemia Laboratory, Munich, Germany., Novotny B; Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, USA., Murphree DH; Digital Health, Artificial Intelligence and Innovations Program, Mayo Clinic, Rochester, MN, USA., Hardway HD; Digital Health, Artificial Intelligence and Innovations Program, Mayo Clinic, Rochester, MN, USA., D'Amato M; Computational Pathology Group, Radboud University Medical Center, Nijmegen, The Netherlands., Lefkes J; Computational Pathology Group, Radboud University Medical Center, Nijmegen, The Netherlands.; Oncode Institute, Utrecht, The Netherlands., Geijs DJ; Computational Pathology Group, Radboud University Medical Center, Nijmegen, The Netherlands.; Oncode Institute, Utrecht, The Netherlands., Feuchtinger A; Core Facility Pathology and Tissue Analytics, Helmholtz Munich, Neuherberg, Germany., Böhner A; Department of Dermatology and Allergy, Technical University of Munich, Munich, Germany., Kaczmarczyk R; Department of Dermatology and Allergy, Technical University of Munich, Munich, Germany., Biedermann T; Department of Dermatology and Allergy, Technical University of Munich, Munich, Germany., Amir AL; Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands., Mooyaart AL; Department of Pathology, Erasmus University Medical Center, Rotterdam, The Netherlands., Ciompi F; Computational Pathology Group, Radboud University Medical Center, Nijmegen, The Netherlands., Litjens G; Computational Pathology Group, Radboud University Medical Center, Nijmegen, The Netherlands.; Oncode Institute, Utrecht, The Netherlands., Wang C; Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, USA., Comfere NI; Digital Health, Artificial Intelligence and Innovations Program, Mayo Clinic, Rochester, MN, USA.; Department of Dermatology and Laboratory Medicine & Pathology, Mayo Clinic, Rochester, MN, USA., Eyerich K; Department of Dermatology, Medical Center, University of Freiburg, Freiburg, Germany. kilian.eyerich@uniklinik-freiburg.de., Braun SA; Dermatology Department, University Hospital Münster, Münster, Germany. stephanalexander.braun@ukmuenster.de.; Department of Dermatology, Medical Faculty, Heinrich-Heine University, Düsseldorf, Germany. stephanalexander.braun@ukmuenster.de., Marr C; Helmholtz AI, Helmholtz Munich, Neuherberg, Germany. carsten.marr@helmholtz-munich.de.; Institute of AI for Health, Helmholtz Munich, Neuherberg, Germany. carsten.marr@helmholtz-munich.de., Peng T; Helmholtz AI, Helmholtz Munich, Neuherberg, Germany. tingying.peng@helmholtz-munich.de.; School of Computation, Information and Technology, Technical University of Munich, Munich, Germany. tingying.peng@helmholtz-munich.de.
Source:
Nature communications [Nat Commun] 2025 May 27; Vol. 16 (1), pp. 4886. Date of Electronic Publication: 2025 May 27.
Publication Type:
Journal Article; Multicenter Study
Language:
English
Journal Info:
Publisher: Nature Pub. Group Country of Publication: England NLM ID: 101528555 Publication Model: Electronic Cited Medium: Internet ISSN: 2041-1723 (Electronic) Linking ISSN: 20411723 NLM ISO Abbreviation: Nat Commun Subsets: MEDLINE
Imprint Name(s):
Original Publication: [London] : Nature Pub. Group
MeSH Terms:
Skin Neoplasms*/pathology , Skin Neoplasms*/diagnosis , Image Processing, Computer-Assisted*/methods , Dermatology*/methods, Humans ; Natural Language Processing ; Artificial Intelligence
References:
Data Brief. 2021 Nov 19;39:107587. (PMID: 34877372)
Med Image Anal. 2022 Jan;75:102274. (PMID: 34731777)
Nat Med. 2024 Mar;30(3):863-874. (PMID: 38504017)
Med Image Anal. 2022 Oct;81:102559. (PMID: 35952419)
J Dtsch Dermatol Ges. 2023 May;21(5):445-453. (PMID: 36772926)
Nat Med. 2025 Mar;31(3):943-950. (PMID: 39779926)
Nat Genet. 2013 Oct;45(10):1113-20. (PMID: 24071849)
Med Image Anal. 2021 Jan;67:101854. (PMID: 33091742)
Nat Med. 2024 Mar;30(3):850-862. (PMID: 38504018)
Cancer Cell. 2023 Sep 11;41(9):1650-1661.e4. (PMID: 37652006)
IEEE Trans Med Imaging. 2022 Nov;41(11):3003-3015. (PMID: 35594209)
Brief Bioinform. 2022 Nov 19;23(6):. (PMID: 36156661)
Nat Med. 2018 Oct;24(10):1559-1567. (PMID: 30224757)
Nat Biomed Eng. 2017 Dec;1(12):925. (PMID: 31015707)
Nat Commun. 2025 May 27;16(1):4886. (PMID: 40419470)
Nature. 2024 Oct;634(8033):466-473. (PMID: 38866050)
Nat Med. 2021 May;27(5):775-784. (PMID: 33990804)
J Dtsch Dermatol Ges. 2019 Jan;17(1):94-103. (PMID: 30592557)
Med Image Anal. 2024 Apr;93:103063. (PMID: 38194735)
Nat Med. 2023 Sep;29(9):2307-2316. (PMID: 37592105)
Virchows Arch. 2021 Feb;478(2):335-341. (PMID: 32719890)
Bioinformatics. 2020 Feb 15;36(4):1234-1240. (PMID: 31501885)
J Pathol. 2021 Jun;254(2):147-158. (PMID: 33904171)
J Digit Imaging. 2021 Feb;34(1):105-115. (PMID: 33169211)
Grant Information:
866411 EC | EU Framework Programme for Research and Innovation H2020 | H2020 Priority Excellent Science | H2020 European Research Council (H2020 Excellent Science - European Research Council); 101113551 EC | EU Framework Programme for Research and Innovation H2020 | H2020 Priority Excellent Science | H2020 European Research Council (H2020 Excellent Science - European Research Council)
Entry Date(s):
Date Created: 20250526 Date Completed: 20250526 Latest Revision: 20250625
Update Code:
20250625
PubMed Central ID:
PMC12106639
DOI:
10.1038/s41467-025-60014-x
PMID:
40419470
Database:
MEDLINE

Weitere Informationen

Histopathology is the reference standard for diagnosing the presence and nature of many diseases, including cancer. However, analyzing tissue samples under a microscope and summarizing the findings in a comprehensive pathology report is time-consuming, labor-intensive, and non-standardized. To address this problem, we present HistoGPT, a vision language model that generates pathology reports from a patient's multiple full-resolution histology images. It is trained on 15,129 whole slide images from 6705 dermatology patients with corresponding pathology reports. The generated reports match the quality of human-written reports for common and homogeneous malignancies, as confirmed by natural language processing metrics and domain expert analysis. We evaluate HistoGPT in an international, multi-center clinical study and show that it can accurately predict tumor subtypes, tumor thickness, and tumor margins in a zero-shot fashion. Our model demonstrates the potential of artificial intelligence to assist pathologists in evaluating, reporting, and understanding routine dermatopathology cases.
(© 2025. The Author(s).)

Competing interests: M.T. is employed by Roche Diagnostics GmbH but conducted his research independently of his work at Roche Diagnostics GmbH as a guest scientist at Helmholtz Munich (Helmholtz Zentrum München—Deutsches Forschungszentrum für Gesundheit und Umwelt GmbH). The remaining authors declare no competing interests. Ethics: An interdisciplinary team of computer scientists, dermatologists, and pathologists from different institutions worked closely together. They shared their expertise and maintained the integrity of the scientific record throughout the study. Local researchers were involved in the research process to ensure that the study was locally relevant. Roles and responsibilities were agreed prior to the study and capacity-building plans were discussed. All research procedures were conducted in accordance with the Declaration of Helsinki. Ethics approval was granted by the Ethics Committee of the Technical University Munich (reference number 2024-98-S-CB) and the Ethics Committee of Westfalen-Lippe (reference number 2024-157-b-S).