• Knowledge distillation and tea...

Treffer: Knowledge distillation and teacher-student learning in medical imaging: Comprehensive overview, pivotal role, and future directions.

Title:
Knowledge distillation and teacher-student learning in medical imaging: Comprehensive overview, pivotal role, and future directions.
Authors:
Li X; College of Information Science and Engineering, Northeastern University, Shenyang 110819, China; Hebei Key Laboratory of Micro-Nano Precision Optical Sensing and Measurement Technology, Qinhuangdao, 066004, China., Li L; College of Information Science and Engineering, Northeastern University, Shenyang 110819, China; Hebei Key Laboratory of Micro-Nano Precision Optical Sensing and Measurement Technology, Qinhuangdao, 066004, China., Li M; Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China., Yan P; Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China., Feng T; College of Information Science and Engineering, Northeastern University, Shenyang 110819, China; Hebei Key Laboratory of Micro-Nano Precision Optical Sensing and Measurement Technology, Qinhuangdao, 066004, China., Luo H; Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China. Electronic address: hao.luo@hit.edu.cn., Zhao Y; College of Information Science and Engineering, Northeastern University, Shenyang 110819, China; Hebei Key Laboratory of Micro-Nano Precision Optical Sensing and Measurement Technology, Qinhuangdao, 066004, China; State Key Laboratory of Synthetical Automation for Process Industries, Shenyang, 110819, China. Electronic address: zhaoyong@ise.neu.edu.cn., Yin S; Department of Mechanical and Industrial Engineering, Faculty of Engineering, Norwegian University of Science and Technology, 7034, Trondheim, Norway. Electronic address: shen.yin@ntnu.no.
Source:
Medical image analysis [Med Image Anal] 2026 Jan; Vol. 107 (Pt A), pp. 103819. Date of Electronic Publication: 2025 Sep 25.
Publication Type:
Journal Article; Review
Language:
English
Journal Info:
Publisher: Elsevier Country of Publication: Netherlands NLM ID: 9713490 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1361-8423 (Electronic) Linking ISSN: 13618415 NLM ISO Abbreviation: Med Image Anal Subsets: MEDLINE
Imprint Name(s):
Publication: Amsterdam : Elsevier
Original Publication: London : Oxford University Press, [1996-
MeSH Terms:
Diagnostic Imaging*/methods , Image Processing, Computer-Assisted*/methods, Humans ; Machine Learning ; Natural Language Processing
Contributed Indexing:
Keywords: Knowledge distillation; Medical image analysis; Teacher–student structure
Entry Date(s):
Date Created: 20250930 Date Completed: 20251013 Latest Revision: 20251013
Update Code:
20251013
DOI:
10.1016/j.media.2025.103819
PMID:
41027250
Database:
MEDLINE

Weitere Informationen

Knowledge Distillation (KD) is a technique to transfer the knowledge from a complex model to a simplified model. It has been widely used in natural language processing and computer vision and has achieved advanced results. Recently, the research of KD in medical image analysis has grown rapidly. The definition of knowledge has been further expanded by combining with the medical field, and its role is not limited to simplifying the model. This paper attempts to comprehensively review the development and application of KD in the medical imaging field. Specifically, we first introduce the basic principles, explain the definition of knowledge and the classical teacher-student network framework. Then, the research progress in medical image classification, segmentation, detection, reconstruction, registration, radiology report generation, privacy protection and other application scenarios is presented. In particular, the introduction of application scenarios is based on the role of KD. We summarize eight main roles of KD techniques in medical image analysis, including model compression, semi-supervised method, weakly supervised method, class balancing, etc. The performance of these roles in all application scenarios is analyzed. Finally, we discuss the challenges in this field and propose potential solutions. KD is still in a rapid development stage in the medical imaging field, we give five potential development directions and research hotspots. A comprehensive literature list of this survey is available at https://github.com/XiangQA-Q/KD-in-MIA.
(Copyright © 2025. Published by Elsevier B.V.)

Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.