Automated Multimodal Fusion with PDE Preprocessing and Learnable Convolutional Pools
Abstract
This research paper introduces a novel automated multimodal and Multifocus fusion framework tailored for
medical imaging applications. The proposed approach leverages advanced deep learning techniques, incorporating Partial
Differential Equation (PDE) preprocessing and learnable convolutional pools. The algorithm accommodates diverse
medical modalities, such as MRI, CT, visual, infrared, and multi-focus images. Through modality-specific preprocessing,
modified convolutional layers, and adaptive pooling, the model intelligently fuses information from various sources,
enhancing the overall imaging quality. Experimental evaluations demonstrate the effectiveness of the proposed method in
generating high-quality multimodal medical images, showcasing its potential for improving diagnostic accuracy and clinical
decision-making.
Full Text:
PDFReferences
REFERENCES
Yu Zhang, Yu Liu, Peng Sun, Han Yan, Xiaolin Zhao, and Li Zhang. 2020. IFCNN: A general image fusion framework based on convolutional neural network. Information Fusion 54: 99–118. https://doi.org/10.1016/j.inffus.2019.07.011
Jameel Ahmed Bhutto, Lianfang Tian, Qiliang Du, Zhengzheng Sun, Lubin Yu, and Muhammad Faizan Tahir. 2022. CT and MRI Medical Image Fusion Using Noise-Removal and Contrast Enhancement Scheme with Convolutional Neural Network. Entropy 24, 3: 393. https://doi.org/10.3390/e24030393
Ahmed Sabeeh Yousif, Zaid Omar, and Usman Ullah Sheikh. 2022. An improved approach for medical image fusion using sparse representation and Siamese convolutional neural network. Biomedical Signal Processing and Control 72: 103357. https://doi.org/10.1016/j.bspc.2021.103357
Han Xu and Jiayi Ma. 2021. EMFusion: An unsupervised enhanced medical image fusion network. Information Fusion 76: 177–186. https://doi.org/10.1016/j.inffus.2021.06.001
Zeyu Wang, Xiongfei Li, Haoran Duan, Yanchi Su, Xiaoli Zhang, and Xinjiang Guan. 2021. Medical image fusion based on convolutional neural networks and non-subsampled contourlet transform. Expert Systems with Applications 171: 114574. https://doi.org/10.1016/j.eswa.2021.114574
Kunpeng Wang, Mingyao Zheng, Hongyan Wei, Guanqiu Qi, and Yuanyuan Li. 2020. Multi-Modality Medical Image Fusion Using Convolutional Neural Network and Contrast Pyramid. Sensors 20, 8: 2169. https://doi.org/10.3390/s20082169
Hao Zhang, Han Xu, Xin Tian, Junjun Jiang, and Jiayi Ma. 2021. Image fusion meets deep learning: A survey and perspective. Information Fusion 76: 323–336. https://doi.org/10.1016/j.inffus.2021.06.008
Gargi J Trivedi and Rajesh Sanghvi. 2022. Medical Image Fusion Using CNN with Automated Pooling. Indian Journal Of Science And Technology 15, 42: 2267–2274. https://doi.org/10.17485/ijst/v15i42.1812
D. Sunderlin Shibu and S. Suja Priyadharsini. 2021. Multi scale decomposition based medical image fusion using convolutional neural network and sparse representation. Biomedical Signal Processing and Control 69: 102789. https://doi.org/10.1016/j.bspc.2021.102789
Yi Li, Junli Zhao, Zhihan Lv, and Jinhua Li. 2021. Medical image fusion method by deep learning. International Journal of Cognitive Computing in Engineering 2: 21–29. https://doi.org/10.1016/j.ijcce.2020.12.004
Velmurugan Subbiah Parvathy, Sivakumar Pothiraj, and Jenyfal Sampson. 2020. A novel approach in multimodality medical image fusion using optimal shearlet and deep learning. International Journal of Imaging Systems and Technology 30, 4: 847–859. https://doi.org/10.1002/ima.22436
Yi Li, Junli Zhao, Zhihan Lv, and Zhenkuan Pan. 2021. Multimodal Medical Supervised Image Fusion Method by CNN. Frontiers in Neuroscience 15. https://doi.org/10.3389/fnins.2021.638976
Jiaheng Xie, Bin Zhang, Jian Ma, Daniel Zeng, and Jenny Lo-Ciganic. 2021. Readmission Prediction for Patients with Heterogeneous Medical History: A Trajectory-Based Deep Learning Approach. ACM Transactions on Management Information Systems 13, 2: 1–27. https://doi.org/10.1145/3468780
Aixia Guo, Michael Pasque, Francis Loh, Douglas L. Mann, and Philip R. O. Payne. 2020. Heart Failure Diagnosis, Readmission, and Mortality Prediction Using Machine Learning and Artificial Intelligence Models. Current Epidemiology Reports 7, 4: 212–219. https://doi.org/10.1007/s40471-020-00259-w
Bobak J. Mortazavi, Nicholas S. Downing, Emily M. Bucholz, Kumar Dharmarajan, Ajay Manhapra, Shu-Xia Li, Sahand N. Negahban, and Harlan M. Krumholz. 2016. Analysis of Machine Learning Techniques for Heart Failure Readmissions. Circulation: Cardiovascular Quality and Outcomes 9, 6: 629–640. https://doi.org/10.1161/circoutcomes.116.003039
Boshu Ru, Xi Tan, Yu Liu, Kartik Kannapur, Dheepan Ramanan, Garin Kessler, Dominik Lautsch, and Gregg Fonarow. 2023. Comparison of Machine Learning Algorithms for Predicting Hospital Readmissions and Worsening Heart Failure Events in Patients With Heart Failure With Reduced Ejection Fraction: Modeling Study. JMIR Formative Research 7: e41775. https://doi.org/10.2196/41775
Dibaba Adeba Debal and Tilahun Melak Sitote. 2022. Chronic kidney disease prediction using machine learning techniques. Journal of Big Data 9, 1. https://doi.org/10.1186/s40537-022-00657-5
Amadou Wurry Jallow, Adama N. S. Bah, Karamo Bah, Chien-Yeh Hsu, and Kuo-Chung Chu. 2022. Machine Learning Approach for Chronic Kidney Disease Risk Prediction Combining Conventional Risk Factors and Novel Metabolic Indices. Applied Sciences 12, 23: 12001. https://doi.org/10.3390/app122312001
Hasnain Iftikhar, Murad Khan, Zardad Khan, Faridoon Khan, Huda M Alshanbari, and Zubair Ahmad. 2023. A Comparative Analysis of Machine Learning Models: A Case Study in Predicting Chronic Kidney Disease. Sustainability 15, 3: 2754. https://doi.org/10.3390/su15032754
Qiong Bai, Chunyan Su, Wen Tang, and Yike Li. 2022. Machine learning to predict end stage kidney disease in chronic kidney disease. Scientific Reports 12, 1. https://doi.org/10.1038/s41598-022-12316-z
Md. Ariful Islam, Md. Ziaul Hasan Majumder, and Md. Alomgeer Hussein. 2023. Chronic kidney disease prediction based on machine learning algorithms. Journal of Pathology Informatics 14: 100189. https://doi.org/10.1016/j.jpi.2023.100189
Aristidis G. Vrahatis, Konstantina Skolariki, Marios G. Krokidis, Konstantinos Lazaros, Themis P. Exarchos, and Panagiotis Vlamos. 2023. Revolutionizing the Early Detection of Alzheimer’s Disease through Non-Invasive Biomarkers: The Role of Artificial Intelligence and Deep Learning. Sensors 23, 9: 4184. https://doi.org/10.3390/s23094184
1Ruoxuan Cui and Manhua Liu. 2019. RNN-based longitudinal analysis for diagnosis of Alzheimer’s disease. Computerized Medical Imaging and Graphics 73: 1–10. https://doi.org/10.1016/j.compmedimag.2019.01.005
Ali Ezzati and Richard B. Lipton. 2020. Machine Learning Predictive Models Can Improve Efficacy of Clinical Trials for Alzheimer’s Disease. Journal of Alzheimer’s Disease 74, 1: 55–63. https://doi.org/10.3233/jad-190822
Sayantan Kumar, Inez Oh, Suzanne Schindler, Albert M Lai, Philip R O Payne, and Aditi Gupta. 2021. Machine learning for modeling the progression of Alzheimer disease dementia using clinical data: a systematic literature review. JAMIA Open 4, 3. https://doi.org/10.1093/jamiaopen/ooab052
JungHo Kong, Doyeon Ha, Juhun Lee, Inhae Kim, Minhyuk Park, Sin-Hyeog Im, Kunyoo Shin, and Sanguk Kim. 2022. Network-based machine learning approach to predict immunotherapy response in cancer patients. Nature Communications 13, 1. https://doi.org/10.1038/s41467-022-31535-6
Yujie You, Xin Lai, Yi Pan, Huiru Zheng, Julio Vera, Suran Liu, Senyi Deng, and Le Zhang. 2022. Artificial intelligence in cancer target identification and drug discovery. Signal Transduction and Targeted Therapy 7, 1. https://doi.org/10.1038/s41392-022-00994-0
Vesna Cuplov and Nicolas André. 2020. Machine Learning Approach to Forecast Chemotherapy-Induced Haematological Toxicities in Patients with Rhabdomyosarcoma. Cancers 12, 7: 1944. https://doi.org/10.3390/cancers12071944
Raihan Rafique, S.M. Riazul Islam, and Julhash U. Kazi. 2021. Machine learning in the prediction of cancer therapy. Computational and Structural Biotechnology Journal 19: 4003–4017. https://doi.org/10.1016/j.csbj.2021.07.003
Yu Liu, Xun Chen, Juan Cheng, Hu Peng, and Zengfu Wang. 2018. Infrared and visible image fusion with convolutional neural networks. International Journal of Wavelets, Multiresolution and Information Processing 16, 03: 1850018. https://doi.org/10.1142/s0219691318500182
Yongzhi Long, Haitao Jia, Yida Zhong, Yadong Jiang, and Yuming Jia. 2021. RXDNFuse: A aggregated residual dense network for infrared and visible image fusion. Information Fusion 69: 128–141. https://doi.org/10.1016/j.inffus.2020.11.009
Jiayi Ma, Wei Yu, Pengwei Liang, Chang Li, and Junjun Jiang. 2019. FusionGAN: A generative adversarial network for infrared and visible image fusion. Information Fusion 48: 11–26. https://doi.org/10.1016/j.inffus.2018.09.004
Alexander Toet. 2017. The TNO Multiband Image Data Collection. Data in Brief 15: 249–251. https://doi.org/10.1016/j.dib.2017.09.038
The Whole Brain Atlas. The Whole Brain Atlas. Retrieved from https://www.med.harvard.edu/aanlib/
Refbacks
- There are currently no refbacks.
------------------------------------------------------------------------------------------------------------------------
The ADBU Journal of Engineering Technology (AJET)" ISSN:2348-7305
This journal is published under the terms of the Creative Commons Attribution (CC-BY) (http://creativecommons.org/licenses/)
Number of Visitors to this Journal: