Treffer: A segmentation method for cardiac MRI that incorporates region constraint guidance and tubular structure awareness within a Siamese network architecture.
Original Publication: Lancaster, Pa., Published for the American Assn. of Physicists in Medicine by the American Institute of Physics.
Xu Y, Quan R, Xu W, Huang Y, Chen X, Liu F. Advances in medical image segmentation: a comprehensive review of traditional, deep learning and hybrid approaches. Bioengineering. 2024;11(10):1034. doi:10.3390/bioengineering11101034.
Tran PV, A fully convolutional neural network for cardiac segmentation in short‐axis mri, arXiv preprint arXiv:1604.00494 (2016).
Khened M, Kollerathu VA, Krishnamurthi G. Fully convolutional multi‐scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers[J]. Medical image analysis. 2019;51:21‐45. doi:10.1016/j.media.2018.10.004.
Baumgartner CF, Koch LM, Pollefeys M, et al. An exploration of 2D and 3D deep learning techniques for cardiac MR image segmentation. Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges: 8th International Workshop, STACOM 2017, Held in Conjunction with MICCAI 2017, Quebec City, Canada, September 10–14, 2017, Revised Selected Papers 8. Springer International Publishing, 2018: 111‐119.
Ronneberger O, Fischer P, Brox T, U‐net: convolutional networks for biomedical image segmentation, in: Medical image computing and computer‐assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5–9, 2015, proceedings, part III 18, Springer, 2015, pp. 234‐241.
Zhang Z, Wang M, Convolutional neural network with convolutional block attention module for finger vein recognition[J]. arXiv preprint arXiv:2202.06673, 2022.
Ye Y, Chen Y, Wang R, et al. Image segmentation using improved U‐Net model and convolutional block attention module based on cardiac magnetic resonance imaging[J]. J Rad Res Appl Sci. 2024;17(1):100816.
Das N, Das S. Attention‐UNet architectures with pretrained backbones for multi‐class cardiac MR image segmentation. Current Problems in Cardiology. 2024;49(1):102129. doi:10.1016/j.cpcardiol.2023.102129.
Wang Z, Chen Y, Mu N, et al. HDF‐SegNet: dynamic integration of hierarchical information for cardiac magnetic resonance image segmentation. Pacific Rim International Conference on Artificial Intelligence. Springer, 2024: 192‐203.
Kumar R, Gupta M, Agarwal A, et al. CBAR‐UNet: a novel methodology for segmentation of cardiac magnetic resonance images using block attention‐based deep residual neural network[J]. Multimedia Tools and Applications. 2024;83:85047‐85063.
Cui H, Yuwen C, Jiang L, et al. Multiscale attention guided U‐Net architecture for cardiac segmentation in short‐axis MRI images[J]. Computer Methods and Programs in Biomedicine. 2021;206:106142. doi:10.1016/j.cmpb.2021.106142.
Wang Z, Peng Y, Li D, et al. MMNet: a multi‐scale deep learning network for the left ventricular segmentation of cardiac MRI images. Applied Intelligence. 2022;52(5):5225‐5240. doi:10.1007/s10489‐021‐02720‐9.
Wang T, Xiong J, Xu X, et al. Msu‐net: multiscale statistical u‐net for real‐time 3d cardiac mri video segmentation. Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II 22. Springer, 2019: 614‐622.
Dong S, Pan Z, Fu Y, et al. DeU‐Net 2.0: enhanced deformable U‐Net for 3D cardiac cine MRI segmentation[J]. Med Image Anal. 2022;78:102389. doi:10.1016/j.media.2022.102389.
Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in neural information processing systems. 2017;30.
Chen J, Lu Y, Yu Q, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. arXiv preprint arXiv:2102.04306, 2021.
Cao H, WANG Y, CHEN J, et al. Swin‐unet: unet‐like pure transformer for medical image segmentation. European conference on computer vision. Springer, 2022: 205‐218.
Zhou HY, Guo J, Zhang Y, et al. nnFormer: volumetric medical image segmentation via a 3D transformer[J]. IEEE transactions on image processing. 2023;32:4036‐4045. doi:10.1109/TIP.2023.3293771.
Rahman MM, Munir M, MARCULESCU R, Emcad: efficient multi‐scale convolutional attention decoding for medical image segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 11769‐11779.
Bertinetto L, Valmadre J, Henriques JF, et al. Fully‐convolutional siamese networks for object tracking. Computer vision–ECCV 2016 workshops: Amsterdam, the Netherlands, October 8–10 and 15–16, 2016, proceedings, part II 14. Springer, 2016: 850‐865.
Li B, Yan J, Wu W, et al. High performance visual tracking with siamese region proposal network. Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 8971‐8980.
Hu YT, Huang JB, SCHWING AG, Videomatch: matching based video object segmentation. Proceedings of the European conference on computer vision (ECCV). 2018: 54‐70.
Lu X, Wang W, Shen J, et al. Zero‐shot video object segmentation with co‐attention siamese networks. IEEE transactions on pattern analysis and machine intelligence. 2020;44(4):2228‐2242.
Kwon D, Ahn J, Kim J, et al. Siamese U‐Net with healthy template for accurate segmentation of intracranial hemorrhage. Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part III 22. Springer, 2019: 848‐855.
Al Chanti D, Duque VG, Crouzier M, et al. IFSS‐Net: interactive few‐shot siamese network for faster muscle segmentation and propagation in volumetric ultrasound[J]. IEEE transactions on medical imaging. 2021;40(10):2615‐2628.
NI B, LIU Z, CAI X, et al. Segmentation of ultrasound image sequences by combing a novel deep siamese network with a deformable contour model. Neural Computing and Applications. 2023;35(20):14535‐14549. doi:10.1007/s00521‐022‐07054‐2.
Li C, Zheng Z, Wu D. Shape‐Aware adversarial learning for scribble‐supervised medical image segmentation with a MaskMix Siamese network: a case study of cardiac MRI segmentation. Bioengineering. 2024;11(11):1146. doi:10.3390/bioengineering11111146.
Pham TV, Vu TN, Le HMQ, et al. CapNet: an automatic attention‐based with mixer model for cardiovascular magnetic resonance image segmentation[J]. Journal of Imaging Informatics in Medicine. 2024;38:94‐123.
QI Y, HE Y, QI X, et al. Dynamic snake convolution based on topological geometric constraints for tubular structure segmentation. Proceedings of the IEEE/CVF international conference on computer vision. 2023: 6070‐6079.
Campello VM, Gkontra P, Izquierdo C, et al. Multi‐centre, multi‐vendor and multi‐disease cardiac segmentation: the M&Ms challenge[J]. IEEE Transactions on Medical Imaging. 2021;40(12):3543‐3554. doi:10.1109/TMI.2021.3090082.
Martín‐Isla C, Campello VM, Izquierdo C, et al. Deep learning segmentation of the right ventricle in cardiac MRI: the M&Ms challenge[J]. IEEE Journal of Biomedical and Health Informatics. 2023;27(7):3302‐3313.
Weitere Informationen
Background: Cardiac magnetic resonance imaging (CMRI) is a non-invasive medical examination method that provides a comprehensive evaluation of the anatomy, function, blood flow, and histology for cardiovascular diseases. Accurately segmentation of the left and right ventricles and myocardium from CMR images can significantly aid doctors in diagnosing cardiovascular diseases. However, due to the variable shapes of the right ventricle, narrow and tubular myocardial structures, and unclear boundaries caused by small grayscale differences between cardiac substructures, CMR image segmentation remains a challenging task.
Purpose: To address these challenges, a dual-branch network that combines a region-constrained Siamese encoder for ventricle segmentation and dynamic snake convolutions to enhance the detection of tubular myocardium structures. The fused outputs enable precise segmentation of cardiac substructures.
Methods: A multi-target segmentation network named RCSiamTANet which integrates a region constraint guided Siamese network and a tubular structure ware is proposed. The network consists of two sub-networks, i.e., the left-right ventricle segmentation sub-network and the myocardium segmentation sub-network. The encoder of the left right ventricle segmentation sub-network contains two parallel branches, in which the original image slice encoder uses CNN to extract the original image features, while the region constraint slice encoder leverages the Siamese network to focus on the target region, avoiding redundant feature extraction. The information of the adjacent slice region constraint provided by the Siamese network and the features of the original slices are extracted at the same time to capture more local details for the joint segmentation of the left and right ventricles. The myocardium segmentation sub-network employs dynamic snake convolutions to capture the topological information of the myocardium's tubular structure, improving the perception of the slender tubular structure. Finally, the outputs of the two sub-networks are fused to achieve accurate segmentation of the cardiac substructures.
Results: Compared to state-of-art models such as FCN, UNet, TransUNet, SwinUNet, and EMCAD, RCSiamTANet achieves significance improvements in mean Dice scores on the ACDC dataset, with increases of 4.47%, 3.81%, 2.56%, 4.56%, and 0.84%, respectively. On the M&Ms dataset, RCSiamTANet improves mean Dice scores by 3.44%, 3.49%, 2.76%, 3.37%, and 1.21%, respectively. On the M&Ms-2 dataset, improvements in mean Dice scores are 2.33%, 0.40%, 1.10%, 0.33%, and 0.33%, respectively.
Conclusions: The proposed RCSiamTANet significantly enhances the segmentation performance of complex cardiac structural regions and tubular structures, effectively improving both accuracy and generalization.
(© 2025 American Association of Physicists in Medicine.)