• Improving the Annotation Proce...

Treffer: Improving the Annotation Process in Computational Pathology: A Pilot Study with Manual and Semi-automated Approaches on Consumer and Medical Grade Devices.

Title:
Improving the Annotation Process in Computational Pathology: A Pilot Study with Manual and Semi-automated Approaches on Consumer and Medical Grade Devices.
Authors:
Cazzaniga G; Department of Medicine and Surgery, Pathology, IRCCS Fondazione San Gerardo Dei Tintori, University of Milano-Bicocca, Via Pergolesi, 33, 20900, Monza, Italy., Del Carro F; Department of Medicine and Surgery, Pathology, IRCCS Fondazione San Gerardo Dei Tintori, University of Milano-Bicocca, Via Pergolesi, 33, 20900, Monza, Italy., Eccher A; Department of Medical and Surgical Sciences for Children and Adults, University of Modena and Reggio Emilia, University Hospital of Modena, Modena, Italy., Becker JU; Institute of Pathology, University Hospital of Cologne, Cologne, Germany., Gambaro G; Division of Nephrology, Department of Medicine, University of Verona, Verona, Italy., Rossi M; Division of Nephrology, Department of Medicine, University of Verona, Verona, Italy., Pieruzzi F; Clinical Nephrology, Fondazione IRCCS San Gerardo Dei Tintori, Monza, Italy.; School of Medicine and Surgery, University of Milano-Bicocca, Milan, Italy., Fraggetta F; Pathology Unit, Azienda Sanitaria Provinciale (ASP) Catania, 'Gravina' Hospital, Caltagirone, Italy., Pagni F; Department of Medicine and Surgery, Pathology, IRCCS Fondazione San Gerardo Dei Tintori, University of Milano-Bicocca, Via Pergolesi, 33, 20900, Monza, Italy., L'Imperio V; Department of Medicine and Surgery, Pathology, IRCCS Fondazione San Gerardo Dei Tintori, University of Milano-Bicocca, Via Pergolesi, 33, 20900, Monza, Italy. vincenzo.limperio@unimib.it.
Source:
Journal of imaging informatics in medicine [J Imaging Inform Med] 2025 Apr; Vol. 38 (2), pp. 1112-1119. Date of Electronic Publication: 2024 Sep 04.
Publication Type:
Journal Article
Language:
English
Journal Info:
Publisher: Springer Nature Country of Publication: Switzerland NLM ID: 9918663679206676 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 2948-2933 (Electronic) Linking ISSN: 29482925 NLM ISO Abbreviation: J Imaging Inform Med Subsets: MEDLINE
Imprint Name(s):
Original Publication: [Cham, Switzerland] : Springer Nature, [2024]-
MeSH Terms:
Artificial Intelligence* , Observer Variation* , Pathology*/methods, Humans ; Algorithms ; Kidney/pathology ; Kidney/diagnostic imaging ; Pilot Projects ; Reproducibility of Results
References:
Nat Biomed Eng. 2021 Jun;5(6):555-570. (PMID: 33649564)
Med Image Anal. 2023 Oct;89:102918. (PMID: 37595404)
J Pathol Clin Res. 2022 May;8(3):209-216. (PMID: 35174655)
Mod Pathol. 2022 Feb;35(2):152-164. (PMID: 34599281)
Pathology. 2023 Dec;55(7):1017-1020. (PMID: 37813761)
Nurse Educ Today. 2009 Apr;29(3):284-91. (PMID: 19084297)
Biomed Eng Online. 2022 May 25;21(1):33. (PMID: 35614504)
Comput Methods Programs Biomed. 2022 Jun;220:106828. (PMID: 35500506)
J Clin Pathol. 2024 Nov 21;:. (PMID: 38538076)
Mod Pathol. 2023 Nov;36(11):100297. (PMID: 37544362)
J Nephrol. 2024 Jan;37(1):221-229. (PMID: 36786977)
J Med Imaging (Bellingham). 2020 Mar;7(2):027501. (PMID: 32341938)
Proc IAPR Int Conf Pattern Recogn. 2021 Jan;2020:8727-8734. (PMID: 36745147)
J Nephrol. 2024 Jan;37(1):65-76. (PMID: 37768550)
J Nephrol. 2022 Sep;35(7):1801-1808. (PMID: 35441256)
Pathologica. 2023 Jun;115(3):127-136. (PMID: 37387439)
Med Image Anal. 2021 Jan;67:101859. (PMID: 33129150)
Lancet Oncol. 2019 May;20(5):e253-e261. (PMID: 31044723)
Diagnostics (Basel). 2022 Sep 26;12(10):. (PMID: 36292013)
J Nephrol. 2021 Jun;34(3):681-688. (PMID: 32683656)
Nat Biomed Eng. 2022 Dec;6(12):1420-1434. (PMID: 36217022)
Nat Med. 2019 Aug;25(8):1301-1309. (PMID: 31308507)
Front Med (Lausanne). 2019 Nov 14;6:255. (PMID: 31799253)
Gastric Cancer. 2023 Mar;26(2):264-274. (PMID: 36264524)
Semin Diagn Pathol. 2023 Mar;40(2):100-108. (PMID: 36882343)
Front Med. 2020 Aug;14(4):470-487. (PMID: 32728875)
Mod Pathol. 2023 Apr;36(4):100086. (PMID: 36788085)
J Pathol Inform. 2015 Feb 24;6:7. (PMID: 25774318)
J Pathol Clin Res. 2022 Mar;8(2):116-128. (PMID: 35014198)
Sci Rep. 2017 Dec 4;7(1):16878. (PMID: 29203879)
J Pathol Inform. 2020 Aug 11;11:23. (PMID: 33042602)
Comput Methods Programs Biomed. 2023 Dec;242:107814. (PMID: 37722311)
AMIA Annu Symp Proc. 2005;:460-4. (PMID: 16779082)
Comput Med Imaging Graph. 2023 Jan;103:102155. (PMID: 36525770)
Crit Rev Oncog. 2023;28(3):1-6. (PMID: 37968987)
Am J Pathol. 2024 Oct;194(10):1898-1912. (PMID: 39032601)
J Pathol Clin Res. 2021 Nov;7(6):542-547. (PMID: 34288586)
Rev Esp Patol. 2022 Jan-Mar;55(1):19-25. (PMID: 34980436)
Nat Commun. 2023 Mar 22;14(1):1572. (PMID: 36949078)
Diagnostics (Basel). 2022 Nov 15;12(11):. (PMID: 36428854)
Crit Rev Oncol Hematol. 2022 Oct;178:103776. (PMID: 35934262)
Grant Information:
DIPLOMAT - PNRR-MR1-2022-12375735 Next Generation EU - NRRP M6C2; Dipartimenti di Eccellenza 2023-2027 (l. 232/2016 Ministero dell'Istruzione, dell'Università e della Ricerca; art. 1 Ministero dell'Istruzione, dell'Università e della Ricerca; commi 314 - 337) Ministero dell'Istruzione, dell'Università e della Ricerca; GR-2021-12374235 Ministero dell'Università e della Ricerca
Contributed Indexing:
Keywords: Annotation; Artificial intelligence; Computational pathology; Digital pathology; Segment Anything Model
Entry Date(s):
Date Created: 20240904 Date Completed: 20250424 Latest Revision: 20250521
Update Code:
20250522
PubMed Central ID:
PMC11950598
DOI:
10.1007/s10278-024-01248-x
PMID:
39231887
Database:
MEDLINE

Weitere Informationen

The development of reliable artificial intelligence (AI) algorithms in pathology often depends on ground truth provided by annotation of whole slide images (WSI), a time-consuming and operator-dependent process. A comparative analysis of different annotation approaches is performed to streamline this process. Two pathologists annotated renal tissue using semi-automated (Segment Anything Model, SAM)) and manual devices (touchpad vs mouse). A comparison was conducted in terms of working time, reproducibility (overlap fraction), and precision (0 to 10 accuracy rated by two expert nephropathologists) among different methods and operators. The impact of different displays on mouse performance was evaluated. Annotations focused on three tissue compartments: tubules (57 annotations), glomeruli (53 annotations), and arteries (58 annotations). The semi-automatic approach was the fastest and had the least inter-observer variability, averaging 13.6 ± 0.2 min with a difference (Δ) of 2%, followed by the mouse (29.9 ± 10.2, Δ = 24%), and the touchpad (47.5 ± 19.6 min, Δ = 45%). The highest reproducibility in tubules and glomeruli was achieved with SAM (overlap values of 1 and 0.99 compared to 0.97 for the mouse and 0.94 and 0.93 for the touchpad), though SAM had lower reproducibility in arteries (overlap value of 0.89 compared to 0.94 for both the mouse and touchpad). No precision differences were observed between operators (p = 0.59). Using non-medical monitors increased annotation times by 6.1%. The future employment of semi-automated and AI-assisted approaches can significantly speed up the annotation process, improving the ground truth for AI tool development.
(© 2024. The Author(s).)

Declarations. Ethical Approval: Approval was obtained from the local ethics committee (PNRR-MR1-2022–12375735, 03/16/23). Conflict of Interest: The authors declare no competing interests.