Treffer: MTMedFormer: multi-task vision transformer for medical imaging with federated learning.
Title:
MTMedFormer: multi-task vision transformer for medical imaging with federated learning.
Authors:
Nath A; Department of Computer Science and Engineering, Indian Institute of Technology Indore, Indore, 453552, Madhya Pradesh, India. ms2104101001@iiti.ac.in., Shukla S; Department of Computer Science and Engineering, Indian Institute of Technology Indore, Indore, 453552, Madhya Pradesh, India., Gupta P; Department of Computer Science and Engineering, Indian Institute of Technology Indore, Indore, 453552, Madhya Pradesh, India.
Source:
Medical & biological engineering & computing [Med Biol Eng Comput] 2025 Nov; Vol. 63 (11), pp. 3421-3434. Date of Electronic Publication: 2025 Jul 08.
Publication Type:
Journal Article
Language:
English
Journal Info:
Publisher: Springer Country of Publication: United States NLM ID: 7704869 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1741-0444 (Electronic) Linking ISSN: 01400118 NLM ISO Abbreviation: Med Biol Eng Comput Subsets: MEDLINE
Imprint Name(s):
Publication: New York, NY : Springer
Original Publication: Stevenage, Eng., Peregrinus.
Original Publication: Stevenage, Eng., Peregrinus.
MeSH Terms:
References:
Santos MK, Ferreira Júnior JR, Wada DT, Tenório APM, Nogueira-Barbosa MH, Marques PMdA (2019) Artificial intelligence, machine learning, computer-aided diagnosis, and radiomics: advances in imaging towards to precision medicine. Radiol Bras 52:387–396. (PMID: 10.1590/0100-3984.2019.0049320473337007049)
Raparthi M (2020) Deep learning for personalized medicine-enhancing precision health with AI. J Sci Technol 1(1):82–90.
Raparthi M, Dodda SB, Maruthi S (2021) AI-enhanced imaging analytics for precision diagnostics in cardiovascular health. Eur Econ Lett (EEL) 11(1).
Najjar R (2023) Redefining radiology: a review of artificial intelligence integration in medical imaging. Diagnostics 13(17):2760. (PMID: 10.3390/diagnostics131727603768530010487271)
Fang P, Feng R, Liu C, Wen R (2024) Boundary sample-based class-weighted semi-supervised learning for malignant tumor classification of medical imaging. Medical & Biological Engineering & Computing, 1–11.
Liu Q, Zhang G, Li K, Zhou F, Yu D (2022) SFOD-Trans: semi-supervised fine-grained object detection framework with transformer module. Med Biol Eng Comput 60(12):3555–3566. (PMID: 10.1007/s11517-022-02682-136251131)
Jian M, Wu R, Xu W, Zhi H, Tao C, Chen H, Li X (2024) VascuConNet: an enhanced connectivity network for vascular segmentation. Medical & Biological Engineering & Computing, 1–12.
Gurcan MN, Boucheron LE, Can A, Madabhushi A, Rajpoot NM, Yener B (2009) Histopathological image analysis: a review. IEEE Rev Biomed Eng 2:147–171. (PMID: 10.1109/RBME.2009.2034865206718042910932)
Zhao Y, Wang X, Che T, Bao G, Li S (2023) Multi-task deep learning for medical image computing and analysis: a review. Comput Biol Med 153:106496. (PMID: 10.1016/j.compbiomed.2022.10649636634599)
Park S, Kim G, Kim J, Kim B, Ye JC (2021) Federated split task-agnostic vision transformer for COVID-19 CXR diagnosis. Adv Neural Inf Process Syst 34:24617–24630.
Gao F, Yoon H, Wu T, Chu X (2020) A feature transfer enabled multi-task deep learning model on medical imaging. Expert Syst Appl 143:112957. (PMID: 10.1016/j.eswa.2019.112957)
Graham S, Vu QD, Jahanifar M, Raza SEA, Minhas F, Snead D, Rajpoot N (2023) One model is all you need: multi-task learning enables simultaneous histology image segmentation and classification. Med Image Anal 83:102685. (PMID: 10.1016/j.media.2022.10268536410209)
Zhou Y, Chen H, Li Y, Liu Q, Xu X, Wang S, Yap P-T, Shen D (2021) Multi-task learning for segmentation and classification of tumors in 3D automated breast ultrasound images. Med Image Anal 70:101918. (PMID: 10.1016/j.media.2020.10191833676100)
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et al (2020) An image is worth 16x16 words: transformers for image recognition at scale. In: International conference on learning representations.
Filipiuk M, Singh, V (2022) Comparing vision transformers and convolutional nets for safety critical systems. In: SafeAI@ AAAI.
Chakraborty C, Khosravi MR, Casalino G, Rodrigues JJ (2023) Guest editorial special issue on AIoMT-enabled federated learning-based computing for socially implemented IoMT systems: how will healthcare systems change? IEEE Trans Comput Soc Syst 10(4):1537–1539. (PMID: 10.1109/TCSS.2023.3293352)
Ghosh S, Ghosh SK (2023) FEEL: Federated learning framework for elderly healthcare using edge-IoMT. IEEE Trans Comput Soc Syst 10(4):1800–1809. (PMID: 10.1109/TCSS.2022.3233300)
Lim WYB, Luong NC, Hoang DT, Jiao Y, Liang Y-C, Yang Q, Niyato D, Miao C (2020) Federated learning in mobile edge networks: a comprehensive survey. IEEE Commun Surv Tutorials 22(3):2031–2063. (PMID: 10.1109/COMST.2020.2986024)
Gupta A, Misra S, Pathak N, Das D (2023) FedCare: federated learning for resource-constrained healthcare devices in IoMT system. IEEE Trans Comput Soc Syst 10(4):1587–1596. (PMID: 10.1109/TCSS.2022.3232192)
Sheller MJ, Edwards B, Reina GA, Martin J, Pati S, Kotrotsou A, Milchenko M, Xu W, Marcus D, Colen RR et al (2020) Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci Rep 10(1):1–12. (PMID: 10.1038/s41598-020-69250-1)
Dayan I, Roth HR, Zhong A, Harouni A, Gentili A, Abidin AZ, Liu A, Costa AB, Wood BJ, Tsai C-S et al (2021) Federated learning for predicting clinical outcomes in patients with COVID-19. Nat Med 27(10):1735–1743. (PMID: 10.1038/s41591-021-01506-3345266999157510)
Zhang M, Wang Y, Luo T (2020) Federated learning for arrhythmia detection of non-IID ECG. In: 2020 IEEE 6th International Conference on Computer and Communications (ICCC). IEEE, pp 1176–1180.
Karimireddy SP, Kale S, Mohri M, Reddi S, Stich S, Suresh AT (2020) Scaffold: stochastic controlled averaging for federated learning. In: International conference on machine learning. PMLR, pp 5132–5143.
Moreira IC, Amaral I, Domingues I, Cardoso A, Cardoso MJ, Cardoso JS (2012) Inbreast: toward a full-field digital mammographic database. Acad Radiol 19(2):236–248. (PMID: 10.1016/j.acra.2011.09.01422078258)
Akselrod-Ballin A, Karlinsky, L, Alpert S, Hasoul S, Ben-Ari R, Barkan E (2016) A region based convolutional network for tumor detection and classification in breast mammography. Deep learning and data labeling for medical applications, 197–205.
Chen H, Wang Y, Guo T, Xu C, Deng Y, Liu Z, Ma S, Xu C, Xu C, Gao W (2021) Pre-trained image processing transformer. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 12299–12310.
Kim B, Kim J, Ye JC (2022) Task-agnostic vision transformer for distributed learning of image processing. IEEE Trans Image Process 32:203–218. (PMID: 10.1109/TIP.2022.322689237015481)
McMahan B, Moore E, Ramage D, Hampson S, Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics. PMLR, pp 1273–1282.
Vo VT-T, Shin T-H, Yang H-J, Kang S-R, Kim S-H (2024) A comparison between centralized and asynchronous federated learning approaches for survival outcome prediction using clinical and pet data from non-small cell lung cancer patients. Comput Methods Programs Biomed 248:108104. (PMID: 10.1016/j.cmpb.2024.10810438457959)
Jochems A, Deist TM, El Naqa I, Kessler M, Mayo C, Reeves J, Jolly S, Matuszak M, Ten Haken R, Soest J et al (2017) Developing and validating a survival prediction model for NSCLC patients through distributed learning across 3 countries. Int J Radiat Oncol* Biol* Phys 99(2):344–352. (PMID: 10.1016/j.ijrobp.2017.04.02128871984)
Singh C, Mishra R, Gupta HP, Banga G (2022) A federated learning-based patient monitoring system in internet of medical things. IEEE Trans Comput Soc Syst 10(4):1622–1628. (PMID: 10.1109/TCSS.2022.3228965)
Can YS, Ersoy C (2021) Privacy-preserving federated deep learning for wearable IoT-based biomedical monitoring. ACM Trans Internet Technol (TOIT) 21(1):1–17. (PMID: 10.1145/3428152)
Li Y, Zhang K, Cao J, Timofte R, Magno M, Benini L, Van Goo L (2023) Localvit: Analyzing locality in vision transformers. In: IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 9598–9605.
Wang J, Huang Q, Tang F, Meng J, Su J, Song S (2022) Stepwise feature fusion: Local guides global. In: Medical image computing and computer assisted intervention–MICCAI 2022: 25th international conference, Singapore, September 18–22, 2022, Proceedings, Part III. Springer, pp 110–120.
Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) End-to-end object detection with transformers. In: Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16. Springer, pp 213–229.
Taghanaki SA, Zheng Y, Zhou SK, Georgescu B, Sharma P, Xu D, Comaniciu D, Hamarneh G (2019) Combo loss: handling input and output imbalance in multi-organ segmentation. Comput Med Imaging Graph 75:24–33. (PMID: 10.1016/j.compmedimag.2019.04.00531129477)
Guan H, Yap P-T, Bozoki A, Liu M (2024) Federated learning for medical image analysis: a survey. Pattern Recognition, 110424.
Wang H, Yurochkin M, Sun Y, Papailiopoulos D, Khazaeni Y (2020) Federated learning with matched averaging. In: International conference on learning representations.
Chen H-Y, Chao W-L (2021) FedBE: making Bayesian model ensemble applicable to federated learning. In: International conference on learning representations.
Eberl MM, Fox CH, Edge SB, Carter CA, Mahoney MC (2006) Bi-rads classification for management of abnormal mammograms. J Am Board Fam Med 19(2):161–164. (PMID: 10.3122/jabfm.19.2.16116513904)
Jadon S (2020) A survey of loss functions for semantic segmentation. In: 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB). IEEE, pp 1–7.
Henderson P, Ferrari V (2017) End-to-end training of object class detectors for mean average precision. In: Computer vision–ACCV 2016: 13th Asian conference on computer vision, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part V 13. Springer, pp 198–213.
Huang J, Ling CX (2005) Using AUC and accuracy in evaluating learning algorithms. IEEE Trans Knowl Data Eng 17(3):299–310. (PMID: 10.1109/TKDE.2005.50)
Loshchilov I, Hutter F (2019) Decoupled weight decay regularization. In: International conference on learning representations.
Liu W, Zeng P, Jiang J, Chen J, Chen L, Hu C, Jian W, Diao X, Wang X (2024) Improved PAA algorithm for breast mass detection in mammograms. Comput Methods Programs Biomed 251:108211. (PMID: 10.1016/j.cmpb.2024.10821138744058)
Shen T, Gou C, Wang J, Wang F-Y (2019) Simultaneous segmentation and classification of mass region from mammograms using a mixed-supervision guided deep model. IEEE Signal Process Lett 27:196–200. (PMID: 10.1109/LSP.2019.2963151)
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention. Springer, pp 234–241.
Dwork C (2006) Differential privacy. In: Automata, languages and programming: 33rd international colloquium, ICALP 2006, Venice, Italy, July 10-14, 2006, Proceedings, Part II 33. Springer, pp 1–12.
Abadi M, Chu A, Goodfellow I, McMahan HB, Mironov I, Talwar K, Zhang L (2016) Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. pp 308–318.
Raparthi M (2020) Deep learning for personalized medicine-enhancing precision health with AI. J Sci Technol 1(1):82–90.
Raparthi M, Dodda SB, Maruthi S (2021) AI-enhanced imaging analytics for precision diagnostics in cardiovascular health. Eur Econ Lett (EEL) 11(1).
Najjar R (2023) Redefining radiology: a review of artificial intelligence integration in medical imaging. Diagnostics 13(17):2760. (PMID: 10.3390/diagnostics131727603768530010487271)
Fang P, Feng R, Liu C, Wen R (2024) Boundary sample-based class-weighted semi-supervised learning for malignant tumor classification of medical imaging. Medical & Biological Engineering & Computing, 1–11.
Liu Q, Zhang G, Li K, Zhou F, Yu D (2022) SFOD-Trans: semi-supervised fine-grained object detection framework with transformer module. Med Biol Eng Comput 60(12):3555–3566. (PMID: 10.1007/s11517-022-02682-136251131)
Jian M, Wu R, Xu W, Zhi H, Tao C, Chen H, Li X (2024) VascuConNet: an enhanced connectivity network for vascular segmentation. Medical & Biological Engineering & Computing, 1–12.
Gurcan MN, Boucheron LE, Can A, Madabhushi A, Rajpoot NM, Yener B (2009) Histopathological image analysis: a review. IEEE Rev Biomed Eng 2:147–171. (PMID: 10.1109/RBME.2009.2034865206718042910932)
Zhao Y, Wang X, Che T, Bao G, Li S (2023) Multi-task deep learning for medical image computing and analysis: a review. Comput Biol Med 153:106496. (PMID: 10.1016/j.compbiomed.2022.10649636634599)
Park S, Kim G, Kim J, Kim B, Ye JC (2021) Federated split task-agnostic vision transformer for COVID-19 CXR diagnosis. Adv Neural Inf Process Syst 34:24617–24630.
Gao F, Yoon H, Wu T, Chu X (2020) A feature transfer enabled multi-task deep learning model on medical imaging. Expert Syst Appl 143:112957. (PMID: 10.1016/j.eswa.2019.112957)
Graham S, Vu QD, Jahanifar M, Raza SEA, Minhas F, Snead D, Rajpoot N (2023) One model is all you need: multi-task learning enables simultaneous histology image segmentation and classification. Med Image Anal 83:102685. (PMID: 10.1016/j.media.2022.10268536410209)
Zhou Y, Chen H, Li Y, Liu Q, Xu X, Wang S, Yap P-T, Shen D (2021) Multi-task learning for segmentation and classification of tumors in 3D automated breast ultrasound images. Med Image Anal 70:101918. (PMID: 10.1016/j.media.2020.10191833676100)
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et al (2020) An image is worth 16x16 words: transformers for image recognition at scale. In: International conference on learning representations.
Filipiuk M, Singh, V (2022) Comparing vision transformers and convolutional nets for safety critical systems. In: SafeAI@ AAAI.
Chakraborty C, Khosravi MR, Casalino G, Rodrigues JJ (2023) Guest editorial special issue on AIoMT-enabled federated learning-based computing for socially implemented IoMT systems: how will healthcare systems change? IEEE Trans Comput Soc Syst 10(4):1537–1539. (PMID: 10.1109/TCSS.2023.3293352)
Ghosh S, Ghosh SK (2023) FEEL: Federated learning framework for elderly healthcare using edge-IoMT. IEEE Trans Comput Soc Syst 10(4):1800–1809. (PMID: 10.1109/TCSS.2022.3233300)
Lim WYB, Luong NC, Hoang DT, Jiao Y, Liang Y-C, Yang Q, Niyato D, Miao C (2020) Federated learning in mobile edge networks: a comprehensive survey. IEEE Commun Surv Tutorials 22(3):2031–2063. (PMID: 10.1109/COMST.2020.2986024)
Gupta A, Misra S, Pathak N, Das D (2023) FedCare: federated learning for resource-constrained healthcare devices in IoMT system. IEEE Trans Comput Soc Syst 10(4):1587–1596. (PMID: 10.1109/TCSS.2022.3232192)
Sheller MJ, Edwards B, Reina GA, Martin J, Pati S, Kotrotsou A, Milchenko M, Xu W, Marcus D, Colen RR et al (2020) Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci Rep 10(1):1–12. (PMID: 10.1038/s41598-020-69250-1)
Dayan I, Roth HR, Zhong A, Harouni A, Gentili A, Abidin AZ, Liu A, Costa AB, Wood BJ, Tsai C-S et al (2021) Federated learning for predicting clinical outcomes in patients with COVID-19. Nat Med 27(10):1735–1743. (PMID: 10.1038/s41591-021-01506-3345266999157510)
Zhang M, Wang Y, Luo T (2020) Federated learning for arrhythmia detection of non-IID ECG. In: 2020 IEEE 6th International Conference on Computer and Communications (ICCC). IEEE, pp 1176–1180.
Karimireddy SP, Kale S, Mohri M, Reddi S, Stich S, Suresh AT (2020) Scaffold: stochastic controlled averaging for federated learning. In: International conference on machine learning. PMLR, pp 5132–5143.
Moreira IC, Amaral I, Domingues I, Cardoso A, Cardoso MJ, Cardoso JS (2012) Inbreast: toward a full-field digital mammographic database. Acad Radiol 19(2):236–248. (PMID: 10.1016/j.acra.2011.09.01422078258)
Akselrod-Ballin A, Karlinsky, L, Alpert S, Hasoul S, Ben-Ari R, Barkan E (2016) A region based convolutional network for tumor detection and classification in breast mammography. Deep learning and data labeling for medical applications, 197–205.
Chen H, Wang Y, Guo T, Xu C, Deng Y, Liu Z, Ma S, Xu C, Xu C, Gao W (2021) Pre-trained image processing transformer. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 12299–12310.
Kim B, Kim J, Ye JC (2022) Task-agnostic vision transformer for distributed learning of image processing. IEEE Trans Image Process 32:203–218. (PMID: 10.1109/TIP.2022.322689237015481)
McMahan B, Moore E, Ramage D, Hampson S, Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics. PMLR, pp 1273–1282.
Vo VT-T, Shin T-H, Yang H-J, Kang S-R, Kim S-H (2024) A comparison between centralized and asynchronous federated learning approaches for survival outcome prediction using clinical and pet data from non-small cell lung cancer patients. Comput Methods Programs Biomed 248:108104. (PMID: 10.1016/j.cmpb.2024.10810438457959)
Jochems A, Deist TM, El Naqa I, Kessler M, Mayo C, Reeves J, Jolly S, Matuszak M, Ten Haken R, Soest J et al (2017) Developing and validating a survival prediction model for NSCLC patients through distributed learning across 3 countries. Int J Radiat Oncol* Biol* Phys 99(2):344–352. (PMID: 10.1016/j.ijrobp.2017.04.02128871984)
Singh C, Mishra R, Gupta HP, Banga G (2022) A federated learning-based patient monitoring system in internet of medical things. IEEE Trans Comput Soc Syst 10(4):1622–1628. (PMID: 10.1109/TCSS.2022.3228965)
Can YS, Ersoy C (2021) Privacy-preserving federated deep learning for wearable IoT-based biomedical monitoring. ACM Trans Internet Technol (TOIT) 21(1):1–17. (PMID: 10.1145/3428152)
Li Y, Zhang K, Cao J, Timofte R, Magno M, Benini L, Van Goo L (2023) Localvit: Analyzing locality in vision transformers. In: IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 9598–9605.
Wang J, Huang Q, Tang F, Meng J, Su J, Song S (2022) Stepwise feature fusion: Local guides global. In: Medical image computing and computer assisted intervention–MICCAI 2022: 25th international conference, Singapore, September 18–22, 2022, Proceedings, Part III. Springer, pp 110–120.
Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) End-to-end object detection with transformers. In: Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16. Springer, pp 213–229.
Taghanaki SA, Zheng Y, Zhou SK, Georgescu B, Sharma P, Xu D, Comaniciu D, Hamarneh G (2019) Combo loss: handling input and output imbalance in multi-organ segmentation. Comput Med Imaging Graph 75:24–33. (PMID: 10.1016/j.compmedimag.2019.04.00531129477)
Guan H, Yap P-T, Bozoki A, Liu M (2024) Federated learning for medical image analysis: a survey. Pattern Recognition, 110424.
Wang H, Yurochkin M, Sun Y, Papailiopoulos D, Khazaeni Y (2020) Federated learning with matched averaging. In: International conference on learning representations.
Chen H-Y, Chao W-L (2021) FedBE: making Bayesian model ensemble applicable to federated learning. In: International conference on learning representations.
Eberl MM, Fox CH, Edge SB, Carter CA, Mahoney MC (2006) Bi-rads classification for management of abnormal mammograms. J Am Board Fam Med 19(2):161–164. (PMID: 10.3122/jabfm.19.2.16116513904)
Jadon S (2020) A survey of loss functions for semantic segmentation. In: 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB). IEEE, pp 1–7.
Henderson P, Ferrari V (2017) End-to-end training of object class detectors for mean average precision. In: Computer vision–ACCV 2016: 13th Asian conference on computer vision, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part V 13. Springer, pp 198–213.
Huang J, Ling CX (2005) Using AUC and accuracy in evaluating learning algorithms. IEEE Trans Knowl Data Eng 17(3):299–310. (PMID: 10.1109/TKDE.2005.50)
Loshchilov I, Hutter F (2019) Decoupled weight decay regularization. In: International conference on learning representations.
Liu W, Zeng P, Jiang J, Chen J, Chen L, Hu C, Jian W, Diao X, Wang X (2024) Improved PAA algorithm for breast mass detection in mammograms. Comput Methods Programs Biomed 251:108211. (PMID: 10.1016/j.cmpb.2024.10821138744058)
Shen T, Gou C, Wang J, Wang F-Y (2019) Simultaneous segmentation and classification of mass region from mammograms using a mixed-supervision guided deep model. IEEE Signal Process Lett 27:196–200. (PMID: 10.1109/LSP.2019.2963151)
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention. Springer, pp 234–241.
Dwork C (2006) Differential privacy. In: Automata, languages and programming: 33rd international colloquium, ICALP 2006, Venice, Italy, July 10-14, 2006, Proceedings, Part II 33. Springer, pp 1–12.
Abadi M, Chu A, Goodfellow I, McMahan HB, Mironov I, Talwar K, Zhang L (2016) Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. pp 308–318.
Contributed Indexing:
Keywords: Bayesian federation; Federated learning; Medical imaging; Multi-task model; Vision transformer
Entry Date(s):
Date Created: 20250708 Date Completed: 20251120 Latest Revision: 20251120
Update Code:
20251121
DOI:
10.1007/s11517-025-03404-z
PMID:
40627238
Database:
MEDLINE
Weitere Informationen
Deep learning has revolutionized medical imaging, improving tasks like image segmentation, detection, and classification, often surpassing human accuracy. However, the training of effective diagnostic models is hindered by two major challenges: the need for large datasets for each task and privacy laws restricting the sharing of medical data. Multi-task learning (MTL) addresses the first challenge by enabling a single model to perform multiple tasks, though convolution-based MTL models struggle with contextualizing global features. Federated learning (FL) helps overcome the second challenge by allowing models to train collaboratively without sharing data, but traditional methods struggle to aggregate stable feature maps due to the permutation-invariant nature of neural networks. To tackle these issues, we propose MTMedFormer, a transformer-based multi-task medical imaging model. We leverage the transformers' ability to learn task-agnostic features using a shared encoder and utilize task-specific decoders for robust feature extraction. By combining MTL with a hybrid loss function, MTMedFormer learns distinct diagnostic tasks in a synergistic manner. Additionally, we introduce a novel Bayesian federation method for aggregating multi-task imaging models. Our results show that MTMedFormer outperforms traditional single-task and MTL models on mammogram and pneumonia datasets, while our Bayesian federation method surpasses traditional methods in image segmentation.
(© 2025. International Federation for Medical and Biological Engineering.)
Declarations. Conflict of interest: The authors declare no competing interests.