Recent Advances in Deep Multimodal Fusion in Comput-er-Aided Diagnosis Systems: A literature Review

Abstract

Deep learning-based approaches have demonstrated great results; in handling the complexities of multimodal data and learning informative repre-sentations from heterogeneous modalities, these multimodal fusion techniques have attracted considerable attention for their role in the integration of infor-mation from different data modalities. In computer aided diagnosis (CAD) sys-tems, the mixture of different information extracted from heterogeneous modal-ities, like medical images, clinical data, genetic data, or textual reports, can pro-vide a more comprehensive and reliable assessment of diseases or conditions. This review article examines advances in deep multimodal fusion using hetero geneous neural networks for medical computer-aided-diagnosis (CAD) systems. We present an overview of the main methodologies and architectures used to combine information from numerous modalities. This review defines the vari ous challenges of the fusion approaches, including early fusion, late fusion, and hybrid fusion. Additionally, we discuss the advantages and limitations of differ ent neural network architectures used for multimodal data fusion. Finally, we mention future research directions and open challenges in this area, paving the way for further advances in deep multimodal fusion using heterogeneous neural networks.