Acta Informatica Pragensia 2022, 11(3), 423-457 | DOI: 10.18267/j.aip.2028093

Comprehensive Review of Multimodal Medical Data Analysis: Open Issues and Future Research Directions

Shashank Shetty ORCID...1,2, Ananthanarayana V S ORCID...1, Ajit Mahale ORCID...3
1 Department of Information Technology, National Institute of Technology Karnataka, Mangalore-575025, Karnataka, India
2 Department of Computer Science and Engineering, Nitte Mahalinga Adyanthaya Memorial Institute of Technology (NMAMIT), NITTE (Deemed to be University), Udupi-574110, India
3 Department of Radiology, Kasturba Medical College, Mangalore, Manipal Academy of Higher Education, Manipal-575001, India

Over the past few decades, the enormous expansion of medical data has led to searching for ways of data analysis in smart healthcare systems. Acquisition of data from pictures, archives, communication systems, electronic health records, online documents, radiology reports and clinical records of different styles with specific numerical information has given rise to the concept of multimodality and the need for machine learning and deep learning techniques in the analysis of the healthcare system. Medical data play a vital role in medical education and diagnosis; determining dependency between distinct modalities is essential. This paper gives a gist of current radiology medical data analysis techniques and their various approaches and frameworks for representation and classification. A brief outline of the existing medical multimodal data processing work is presented. The main objective of this study is to spot gaps in the surveyed area and list future tasks and challenges in radiology. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (or PRISMA) guidelines were incorporated in this study for effective article search and to investigate several relevant scientific publications. The systematic review was carried out on multimodal medical data analysis and highlighted advantages, limitations and strategies. The inherent benefit of multimodality in the medical domain powered with artificial intelligence has a significant impact on the performance of the disease diagnosis frameworks.

Keywords: AI; Big data analysis; Clinical recommendation system; Multimodality; Structured and unstructured healthcare data; Data extraction; Data classification; Data visualization.

Received: September 14, 2022; Revised: December 18, 2022; Accepted: December 24, 2022; Published: December 26, 2022  Show citation

ACS AIP APA ASA Harvard Chicago Chicago Notes IEEE ISO690 MLA NLM Turabian Vancouver
Shetty, S., S, A.V., & Mahale, A. (2022). Comprehensive Review of Multimodal Medical Data Analysis: Open Issues and Future Research Directions. Acta Informatica Pragensia, 11(3), 423-457. doi: 10.18267/j.aip.202
Download citation
  • BibTeX (.bib)
  • Bookends (.ris)
  • EasyBib (.ris)
  • EndNote (.enw)
  • EndNote 8 (.xml)
  • ISI WoS (.isi)
  • Medlars (.medlars)
  • Mendeley (.ris)
  • MODS (.xml)
  • Papers (.ris)
  • RefWorks (.txt)
  • RefManager (.ris)
  • RIS (.ris)
  • MS Word (.xml)
  • Zotero (.ris)
    • Open full article

    References

    1. Abdulkareem, M. B. (2018). Design and Development of Multimodal Medical Image Fusion using Discrete Wavelet Transform. In 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT). IEEE. https://doi.org/10.1109/icicct.2018.8472997 Go to original source...
    2. Abubaker, M., & Babayigit, B. (2022). Detection of Cardiovascular Diseases in ECG Images Using Machine Learning and Deep Learning Methods. IEEE Transactions on Artificial Intelligence, (in press). https://doi.org/10.1109/tai.2022.3159505 Go to original source...
    3. Abujaber, A. A., Nashwan, A. J., & Fadlalla, A. (2022). Harnessing machine learning to support evidence-based medicine: A pragmatic reconciliation framework. Intelligence-Based Medicine, 6, 100048. https://doi.org/10.1016/j.ibmed.2022.100048 Go to original source...
    4. Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022). Multimodal biomedical AI. Nature Medicine, 28(9), 1773-1784. https://doi.org/10.1038/s41591-022-01981-2 Go to original source...
    5. Adali, T., Levin-Schwartz, Y., & Calhoun, V. D. (2015). Multimodal Data Fusion Using Source Separation: Application to Medical Imaging. Proceedings of the IEEE, 103(9), 1494-1506. https://doi.org/10.1109/jproc.2015.2461601 Go to original source...
    6. Akcakaya, M., Nam, S., Hu, P., Moghari, M. H., Ngo, L. H., Tarokh, V., Manning, W. J., & Nezafat, R. (2011). Compressed Sensing With Wavelet Domain Dependencies for Coronary MRI: A Retrospective Study. IEEE Transactions on Medical Imaging, 30(5), 1090-1099. https://doi.org/10.1109/tmi.2010.2089519 Go to original source...
    7. Alfarghaly, O., Khaled, R., Elkorany, A., Helal, M., & Fahmy, A. (2021). Automated radiology report generation using conditioned transformers. Informatics in Medicine Unlocked, 24, 100557. https://doi.org/10.1016/j.imu.2021.100557 Go to original source...
    8. Almasri, M. M., & Alajlan, A. M. (2022). Artificial Intelligence-Based Multimodal Medical Image Fusion Using Hybrid S2 Optimal CNN. Electronics, 11(14), 2124. https://doi.org/10.3390/electronics11142124 Go to original source...
    9. Alqahtani, A., Ullah Khan, H., Alsubai, S., Sha, M., Almadhor, A., Iqbal, T., & Abbas, S. (2022). An efficient approach for textual data classification using deep learning. Frontiers in Computational Neuroscience, 16. https://doi.org/10.3389/fncom.2022.992296 Go to original source...
    10. Alodadi, M. (2017). Radiology Clinical Notes Mining Using Weighted Association Rules. In 2017 IEEE International Conference on Healthcare Informatics (ICHI). IEEE. https://doi.org/10.1109/ichi.2017.23 Go to original source...
    11. Amal, S., Safarnejad, L., Omiye, J. A., Ghanzouri, I., Cabot, J. H., & Ross, E. G. (2022). Use of Multi-Modal Data and Machine Learning to Improve Cardiovascular Disease Care. Frontiers in Cardiovascular Medicine, 9. https://doi.org/10.3389/fcvm.2022.840262 Go to original source...
    12. An, J., Lu, X., & Duan, H. (2008). Integrated Visualization of Multi-Modal Electronic Health Record Data. In 2008 2nd International Conference on Bioinformatics and Biomedical Engineering. IEEE. https://doi.org/10.1109/icbbe.2008.156 Go to original source...
    13. Atta-Ur-Rahman, Nasir, M. U., Gollapalli, M., Zubair, M., Saleem, M. A., Mehmood, S., Khan, M. A., & Mosavi, A. (2022). Advance Genome Disorder Prediction Model Empowered With Deep Learning. IEEE Access, 10, 70317-70328. https://doi.org/10.1109/access.2022.3186998 Go to original source...
    14. Aydin, F., Zhang, M., & Haffari, G. (2019). Medical Multimodal Classifiers Under Scarce Data Condition. arXiv. https://doi.org/10.48550/arXiv.1902.08888 Go to original source...
    15. Barillot, C., Lemoine, D., Le Briquer, L., Lachmann, F., & Gibaud, B. (1993). Data fusion in medical imaging: merging multimodal and multipatient images, identification of structures and 3D display aspects. European Journal of Radiology, 17(1), 22-27. https://doi.org/10.1016/0720-048x(93)90024-h Go to original source...
    16. Batko, K., & ¦lêzak, A. (2022). The use of Big Data Analytics in healthcare. Journal of Big Data, 9(1). https://doi.org/10.1186/s40537-021-00553-4 Go to original source...
    17. Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022). A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets. The Visual Computer, 38, 2939-2970. https://doi.org/10.1007/s00371-021-02166-7 Go to original source...
    18. Ben Said, A., Mohamed, A., Elfouly, T., Harras, K., & Wang, Z. J. (2017). Multimodal Deep Learning Approach for Joint EEG-EMG Data Compression and Classification. In 2017 IEEE Wireless Communications and Networking Conference (WCNC). IEEE. https://doi.org/10.1109/WCNC.2017.7925709 Go to original source...
    19. Bernal, E. A., Yang, X., Li, Q., Kumar, J., Madhvanath, S., Ramesh, P., & Bala, R. (2018). Deep Temporal Multimodal Fusion for Medical Procedure Monitoring Using Wearable Sensors. IEEE Transactions on Multimedia, 20(1), 107-118. https://doi.org/10.1109/tmm.2017.2726187 Go to original source...
    20. Berner, E. S. (2010). Clinical Decision Support Systems: Theory and Practice. Springer. https://doi.org/10.1007/978-0-387-38319-4 Go to original source...
    21. Bleyer, W. A. (1997). The U.S. pediatric cancer clinical trials programmes: International implications and the way forward. European Journal of Cancer, 33(9), 1439-1447. https://doi.org/10.1016/s0959-8049(97)00249-9 Go to original source...
    22. Bloch, I. (1996). Some aspects of Dempster-Shafer evidence theory for classification of multi-modality medical images taking partial volume effect into account. Pattern Recognition Letters, 17(8), 905-919. https://doi.org/10.1016/0167-8655(96)00039-6 Go to original source...
    23. Bu, S., Wang, L., Han, P., Liu, Z., & Li, K. (2017). 3D shape recognition and retrieval based on multi-modality deep learning. Neurocomputing, 259, 183-193. https://doi.org/10.1016/j.neucom.2016.06.088 Go to original source...
    24. Bodile, A., & Kshirsagar, M. (2015). Text mining in radiology reports by statistical machine translation approach. In 2015 Global Conference on Communication Technologies (GCCT). IEEE. https://doi.org/10.1109/gcct.2015.7342797 Go to original source...
    25. Cai, Q., Wang, H., Li, Z., & Liu, X. (2019). A Survey on Multimodal Data-Driven Smart Healthcare Systems: Approaches and Applications. IEEE Access, 7, 133583-133599. https://doi.org/10.1109/ACCESS.2019.2941419 Go to original source...
    26. Carvalho, R., Pedrosa, J., & Nedelcu, T. (2021). Multimodal Multi-tasking for Skin Lesion Classification Using Deep Neural Networks. In Advances in Visual Computing, (pp. 27-38). Springer. https://doi.org/10.1007/978-3-030-90439-5_3 Go to original source...
    27. Castiglione, A., Vijayakumar, P., Nappi, M., Sadiq, S., & Umer, M. (2021). COVID-19: Automatic Detection of the Novel Coronavirus Disease from CT Images Using an Optimized Convolutional Neural Network. IEEE Transactions on Industrial Informatics, 17(9), 6480-6488. https://doi.org/10.1109/tii.2021.3057524 Go to original source...
    28. Chandra, B. S., Sastry, C. S., & Jana, S. (2019). Robust Heartbeat Detection From Multimodal Data via CNN-Based Generalizable Information Fusion. IEEE Transactions on Biomedical Engineering, 66(3), 710-717. https://doi.org/10.1109/tbme.2018.2854899 Go to original source...
    29. Chaudhury, S., Dey, L., Verma, I., & Hassan, E. (2017). Mining Multimodal Data. Pattern Recognition and Big Data, (pp. 581-604). World Scientific. https://doi.org/10.1142/9789813144552_0017 Go to original source...
    30. Chen, R., Ho, J. C., & Lin, J.-M. S. (2020). Extracting medication information from unstructured public health data: a demonstration on data from population-based and tertiary-based samples. BMC Medical Research Methodology, 20(1). https://doi.org/10.1186/s12874-020-01131-7 Go to original source...
    31. Chen, M., Hao, Y., Hwang, K., Wang, L., & Wang, L. (2017). Disease Prediction by Machine Learning Over Big Data From Healthcare Communities. IEEE Access, 5, 8869-8879. https://doi.org/10.1109/access.2017.2694446 Go to original source...
    32. Chien, S.-C., Chen, Y.-L., Chien, C.-H., Chin, Y.-P., Yoon, C. H., Chen, C.-Y., Yang, H.-C., & Li, Y.-C. J. (2022). Alerts in Clinical Decision Support Systems (CDSS): A Bibliometric Review and Content Analysis. Healthcare (Basel, Switzerland), 10(4), 601. https://doi.org/10.3390/healthcare10040601 Go to original source...
    33. Chun-An Chou, Kampa, K., Mehta, S. H., Tungaraza, R. F., Chaovalitwongse, W. A., & Grabowski, T. J. (2014). Voxel Selection Framework in Multi-Voxel Pattern Analysis of fMRI Data for Prediction of Neural Response to Visual Stimuli. IEEE Transactions on Medical Imaging, 33(4), 925-934. https://doi.org/10.1109/tmi.2014.2298856 Go to original source...
    34. Comito, C., Falcone, D., & Forestiero, A. (2022). AI-Driven Clinical Decision Support: Enhancing Disease Diagnosis Exploiting Patients Similarity. IEEE Access, 10, 6878-6888. https://doi.org/10.1109/access.2022.3142100 Go to original source...
    35. Cooper, J. W., Ebadollahi, S., & Eide E. (2009). A Thin Client Interface to a High Performance Multi-modal Image Analytics System. In 2009 42nd Hawaii International Conference on System Sciences. IEEE. https://doi.org/10.1109/hicss.2009.45 Go to original source...
    36. Danzhou L., Hua, K. A., & Sugaya, K. (2008). A Generic Framework for Internet-Based Interactive Applications of High-Resolution 3-D Medical Image Data. IEEE Transactions on Information Technology in Biomedicine, 12(5), 618-626. https://doi.org/10.1109/titb.2008.923770 Go to original source...
    37. Dash, S., Shakyawar, S. K., Sharma, M., & Kaushik, S. (2019). Big data in healthcare: management, analysis and future prospects. Journal of Big Data, 6(1). https://doi.org/10.1186/s40537-019-0217-0 Go to original source...
    38. Denzil Bosco, T., Narayanan, A. L., & Veeramuthu, A. (2017). A new multimodal brain tumour image fusion for pre-processing of medical image database. In 2017 International Conference on Communication and Signal Processing (ICCSP). IEEE. https://doi.org/10.1109/iccsp.2017.8286687 Go to original source...
    39. Deshpande, P., Rasin, A., Brown, E., Furst, J., Raicu, D. S., Montner, S. M., & Armato, S. G. (2018). Big Data Integration Case Study for Radiology Data Sources. In 2018 IEEE Life Sciences Conference (LSC). IEEE. https://doi.org/10.1109/lsc.2018.8572185 Go to original source...
    40. Dramburg, S., Marchante Fernández, M., Potapova, E., & Matricardi, P. M. (2020). The Potential of Clinical Decision Support Systems for Prevention, Diagnosis, and Monitoring of Allergic Diseases. Frontiers in Immunology, 11. https://doi.org/10.3389/fimmu.2020.02116 Go to original source...
    41. Drotar, P., Mekyska, J., Rektorova, I., Masarova, L., Smekal, Z., & Faundez-Zanuy, M. (2015). Decision support framework for Parkinson's disease based on novel handwriting markers. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 1-1. https://doi.org/10.1109/tnsre.2014.2359997 Go to original source...
    42. Faris, H., Habib, M., Faris, M., Elayan, H., & Alomari, A. (2021). An intelligent multimodal medical diagnosis system based on patients' medical questions and structured symptoms for telemedicine. Informatics in Medicine Unlocked, 23, 100513. https://doi.org/10.1016/j.imu.2021.100513 Go to original source...
    43. Fazli, S., Dahne, S., Samek, W., Bieszmann, F., & Muller, K.-R. (2015). Learning From More Than One Data Source: Data Fusion Techniques for Sensorimotor Rhythm-Based Brain-Computer Interfaces. Proceedings of the IEEE, 103(6), 891-906. https://doi.org/10.1109/jproc.2015.2413993 Go to original source...
    44. Fei, Z., Ryeznik, Y., Sverdlov, A., Tan, C. W., & Wong, W. K. (2022). An overview of healthcare data analytics with applications to the COVID-19 pandemic. IEEE Transactions on Big Data, 8(6), 1463-1480. https://doi.org/10.1109/tbdata.2021.3103458 Go to original source...
    45. Fleury, A., Vacher, M., & Noury, N. (2010). SVM-Based Multimodal Classification of Activities of Daily Living in Health Smart Homes: Sensors, Algorithms, and First Experimental Results. IEEE Transactions on Information Technology in Biomedicine, 14(2), 274-283. https://doi.org/10.1109/titb.2009.2037317 Go to original source...
    46. Friedlin, J., Mahoui, M., Jones, J., & Jamieson, P. (2011). Knowledge Discovery and Data Mining of Free Text Radiology Reports. In 2011 IEEE First International Conference on Healthcare Informatics, Imaging and Systems Biology. IEEE. https://doi.org/10.1109/hisb.2011.31 Go to original source...
    47. Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005). Effects of Computerized Clinical Decision Support Systems on Practitioner Performance and Patient Outcomes. JAMA, 293(10), 1223. https://doi.org/10.1001/jama.293.10.1223 Go to original source...
    48. Gehrmann, S., Dernoncourt, F., Li, Y., Carlson, E. T., Wu, J. T., Welt, J., Foote, J., Moseley, E. T., Grant, D. W., Tyler, P. D., & Celi, L. A. (2018). Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives. PLOS ONE, 13(2), e0192360. https://doi.org/10.1371/journal.pone.0192360 Go to original source...
    49. Giannakos, M. N., Sharma, K., Pappas, I. O., Kostakos, V., & Velloso, E. (2019). Multimodal data as a means to understand the learning experience. International Journal of Information Management, 48, 108-119. https://doi.org/10.1016/j.ijinfomgt.2019.02.003 Go to original source...
    50. Gong, T., Tan, C. L., Leong, T. Y., Lee, C. K., Pang, B. C., Lim, C. C. T., Tian, Q., Tang, S., & Zhang, Z. (2008). Text Mining in Radiology Reports. In 2008 Eighth IEEE International Conference on Data Mining. IEEE. https://doi.org/10.1109/icdm.2008.150 Go to original source...
    51. Hak, F., Guimarães, T., & Santos, M. (2022). Towards effective clinical decision support systems: A systematic review. PLOS ONE, 17(8), e0272846. https://doi.org/10.1371/journal.pone.0272846 Go to original source...
    52. Hamidinekoo, A., Pieciak, T., Afzali, M., Akanyeti, O., & Yuan, Y. (2021). Glioma Classification Using Multimodal Radiology and Histology Data. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, (pp. 508-518). Springer. https://doi.org/10.1007/978-3-030-72087-2_45 Go to original source...
    53. Harada, T., Miyagami, T., Kunitomo, K., & Shimizu, T. (2021). Clinical Decision Support Systems for Diagnosis in Primary Care: A Scoping Review. International Journal of Environmental Research and Public Health, 18(16), 8435. https://doi.org/10.3390/ijerph18168435 Go to original source...
    54. Haque, M. E., & Tozal, M. E. (2022). Identifying Health Insurance Claim Frauds Using Mixture of Clinical Concepts. IEEE Transactions on Services Computing, 15(4), 2356-2367. https://doi.org/10.1109/tsc.2021.3051165 Go to original source...
    55. Heidarivincheh, F., McConville, R., Morgan, C., McNaney, R., Masullo, A., Mirmehdi, M., Whone, A. L., & Craddock, I. (2021). Multimodal Classification of Parkinson's Disease in Home Environments with Resiliency to Missing Modalities. Sensors, 21(12), 4133. https://doi.org/10.3390/s21124133 Go to original source...
    56. Hilmizen, N., Bustamam, A., & Sarwinda, D. (2020). The multimodal deep learning for diagnosing covid-19 pneumonia from chest ct-scan and x-ray images. In 2020 3rd International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), (pp. 26-31). IEEE. https://doi.org/10.1109/ISRITI51436.2020.9315478 Go to original source...
    57. Hossain, Md. J., Wadud, Md. A. H., Rahman, A., Ferdous, J., Alam, M. S., Amir - Ul - Haque Bhuiyan, T. M., & Mridha, M. F. (2021). A Secured Patient's Online Data Monitoring through Blockchain: An Intelligent way to Store Lifetime Medical Records. In 2021 International Conference on Science & Contemporary Technologies (ICSCT). IEEE. https://doi.org/10.1109/icsct53883.2021.9642537 Go to original source...
    58. Huang, S.-C., Pareek, A., Seyyedi, S., Banerjee, I., & Lungren, M. P. (2020a). Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. Npj Digital Medicine, 3(1). https://doi.org/10.1038/s41746-020-00341-z Go to original source...
    59. Huang, S.-C., Pareek, A., Zamanian, R., Banerjee, I., & Lungren, M. P. (2020b). Multimodal fusion with deep neural networks for leveraging CT imaging and electronic health record: a case-study in pulmonary embolism detection. Scientific Reports, 10(1), 22147. https://doi.org/10.1038/s41598-020-78888-w Go to original source...
    60. Iakovidis, D., & Christos, S. (2012). A semantic model for multimodal data mining in healthcare information systems. Studies in health technology and informatics, 180, 574-578. Go to original source...
    61. Institute of Medicine. (2000). To Err Is Human. National Academies Press. https://doi.org/10.17226/9728 Go to original source...
    62. Jacenkow, G., O'Neil, A. Q., & Tsaftaris, S. A. (2022). Indication as Prior Knowledge for Multimodal Disease Classification in Chest Radiographs with Transformers. In 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI). IEEE. https://doi.org/10.1109/isbi52829.2022.9761567 Go to original source...
    63. Jacobs, A. (2009). The pathologies of big data. Communications of the ACM, 52(8), 36. https://doi.org/10.1145/1536616.1536632 Go to original source...
    64. Jindal, A., Dua, A., Kumar, N., Das, A. K., Vasilakos, A. V., & Rodrigues, J. J. P. C. (2018). Providing Healthcare-as-a-Service Using Fuzzy Rule Based Big Data Analytics in Cloud Computing. IEEE Journal of Biomedical and Health Informatics, 22(5), 1605-1618. https://doi.org/10.1109/jbhi.2018.2799198 Go to original source...
    65. Jiang, Y., Li, W., Hossain, M. S., Chen, M., Alelaiwi, A., & Al-Hammadi, M. (2020). A snapshot research and implementation of multimodal information fusion for data-driven emotion recognition. Information Fusion, 53, 209-221. https://doi.org/10.1016/j.inffus.2019.06.019 Go to original source...
    66. Joseph, N., Bernadin, S., Hodges, D., & Sekhar, P. (2021). A Case Study on using Unstructured Data Analysis Methods to identify local Covid-19 Hotspots. In SoutheastCon 2021. IEEE. https://doi.org/10.1109/southeastcon45413.2021.9401921 Go to original source...
    67. Joshi, A., Papanastassiou, A., Vives, K. P., Spencer, D. D., Staib, L. H., & Papademetris, X. (2010). Light-sensitive visualization of multimodal data for neurosurgical applications. In 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. IEEE. https://doi.org/10.1109/isbi.2010.5490128 Go to original source...
    68. Karahan, E., Rojas-Lopez, P. A., Bringas-Vega, M. L., Valdes-Hernandez, P. A., & Valdes-Sosa, P. A. (2015). Tensor Analysis and Fusion of Multimodal Brain Images. Proceedings of the IEEE, 103(9), 1531-1559. https://doi.org/10.1109/jproc.2015.2455028 Go to original source...
    69. Khoo, V. S., Dearnaley, D. P., Finnigan, D. J., Padhani, A., Tanner, S. F., & Leach, M. O. (1997). Magnetic resonance imaging (MRI): considerations and applications in radiotherapy treatment planning. Radiotherapy and Oncology, 42(1), 1-15. https://doi.org/10.1016/s0167-8140(96)01866-x Go to original source...
    70. Kline, A., Wang, H., Li, Y., Dennis, S., Hutch, M., Xu, Z., Wang, F., Cheng, F., & Luo, Y. (2022). Multimodal machine learning in precision health: A scoping review. Npj Digital Medicine, 5(1), 1-14. https://doi.org/10.1038/s41746-022-00712-8 Go to original source...
    71. Kozlovszky, M., Batbayar, K., Garaguly, Z., & Karozkai, K. (2016). Multimodal biophysical data visualization for patient monitoring. In 2016 IEEE 11th International Symposium on Applied Computational Intelligence and Informatics (SACI). IEEE. https://doi.org/10.1109/saci.2016.7507411 Go to original source...
    72. Kumar, S., & Singh, M. (2019). Big data analytics for healthcare industry: impact, applications, and tools. Big Data Mining and Analytics, 2(1), 48-57. https://doi.org/10.26599/bdma.2018.9020031 Go to original source...
    73. Kurniawan, H., & Pechenizkiy, M. (2014). Towards the Stress Analytics Framework: Managing, Mining, and Visualizing Multi-modal Data for Stress Awareness. In 2014 IEEE 27th International Symposium on Computer-Based Medical Systems. IEEE. https://doi.org/10.1109/cbms.2014.129 Go to original source...
    74. Lahat, D., Adali, T., & Jutten, C. (2015). Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects. Proceedings of the IEEE, 103(9), 1449-1477. https://doi.org/10.1109/jproc.2015.2460697 Go to original source...
    75. Lahat, D., Adaly, T., & Jutten, C. (2014). Challenges in multimodal data fusion. In 2014 22nd European Signal Processing Conference (EUSIPCO), (pp. 101-105). IEEE.
    76. Lawonn, K., Smit, N. N., Bühler, K., & Preim, B. (2017). A Survey on Multimodal Medical Data Visualization. Computer Graphics Forum, 37(1), 413-438. https://doi.org/10.1111/cgf.13306 Go to original source...
    77. Levin, D., Aladl, U., Germano, G., & Slomka, P. (2005). Techniques for efficient, real-time, 3D visualization of multi-modality cardiac data using consumer graphics hardware. Computerized Medical Imaging and Graphics, 29(6), 463-475. https://doi.org/10.1016/j.compmedimag.2005.02.007 Go to original source...
    78. Li, H., & Fan, Y. (2019). Early Prediction Of Alzheimer's Disease Dementia Based On Baseline Hippocampal MRI and 1-Year Follow-Up Cognitive Measures Using Deep Recurrent Neural Networks. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). https://doi.org/10.1109/isbi.2019.8759397 Go to original source...
    79. Lin, S., Han, Z., Li, D., Zeng, J., Yang, X., Liu, X., & Liu, F. (2020). Integrating model- and data-driven methods for synchronous adaptive multi-band image fusion. Information Fusion, 54, 145-160. https://doi.org/10.1016/j.inffus.2019.07.009 Go to original source...
    80. Liu, Y., Yan, B., Zhang, R., Liu, K., Jeon, G., & Yang, X. (2022a). Multi-Scale Mixed Attention Network for CT and MRI Image Fusion. Entropy, 24(6), 843. https://doi.org/10.3390/e24060843 Go to original source...
    81. Liu, W., Zhang, X., Lv, H., Li, J., Liu, Y., Yang, Z., Weng, X., Lin, Y., Song, H., & Wang, Z. (2022b). Using a classification model for determining the value of liver radiological reports of patients with colorectal cancer. Frontiers in Oncology, 12. https://doi.org/10.3389/fonc.2022.913806 Go to original source...
    82. Lopez, K., Fodeh, S. J., Allam, A., Brandt, C. A., & Krauthammer, M. (2020). Reducing Annotation Burden Through Multimodal Learning. Frontiers in Big Data, 3, Article 19. https://doi.org/10.3389/fdata.2020.00019 Go to original source...
    83. Lv, Z., & Qiao, L. (2020). Analysis of healthcare big data. Future Generation Computer Systems, 109, 103-110. https://doi.org/10.1016/j.future.2020.03.039 Go to original source...
    84. Lynch, C. (2008). How do your data grow? Nature, 455(7209), 28-29. https://doi.org/10.1038/455028a Go to original source...
    85. Maghsoodi, A., Sevenster, M., Scholtes, J., & Nalbantov, G. (2012). Sentence-based classification of free-text breast cancer radiology reports. In 2012 25th IEEE International Symposium on Computer-Based Medical Systems (CBMS). IEEE. https://doi.org/10.1109/cbms.2012.6266374 Go to original source...
    86. Manssour, I. H., Furuie, S. S., Nedel, L. P., & Freitas, C. M. D. S. (2000). A multimodal visualization framework for medical data. In Proceedings 13th Brazilian Symposium on Computer Graphics and Image Processing. IEEE. https://doi.org/10.1109/sibgra.2000.895844 Go to original source...
    87. Nasution, B. I., Nugraha, Y., Prasetya, N. L., Aminanto, M. E., Sulasikin, A., Kanggrawan, J. I., & Suherman, A. L. (2022). COVID-19 Mortality Risk Factors Using Survival Analysis: A Case Study of Jakarta, Indonesia. IEEE Transactions on Computational Social Systems, 1-10. https://doi.org/10.1109/tcss.2022.3163325 Go to original source...
    88. Mealha, O. E., Rocha, C., Santos, B. S., Pereira, A. S., Pedroso de Lima, J. J., Botelho, M. F., & Ribeiro, M. J. (1994). 3D registration and integrated visualization of multimodal clinical data. In Proceedings of 16th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE. https://doi.org/10.1109/iembs.1994.411883 Go to original source...
    89. Megalooikonomou, V., & Kontos, D. (2007). Medical Data Fusion for Telemedicine. IEEE Engineering in Medicine and Biology Magazine, 26(5), 36-42. https://doi.org/10.1109/emb.2007.901790 Go to original source...
    90. Meng, T., Lin, L., Shyu, M.-L., & Chen, S.-C. (2010). Histology Image Classification Using Supervised Classification and Multimodal Fusion. In 2010 IEEE International Symposium on Multimedia. IEEE. https://doi.org/10.1109/ism.2010.29 Go to original source...
    91. Mieloszyk, R. J., Rosenbaum, J. I., Bhargava, P., & Hall, C. S. (2017). Predictive modeling to identify scheduled radiology appointments resulting in non-attendance in a hospital setting. In 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE. https://doi.org/10.1109/embc.2017.8037394 Go to original source...
    92. Min Chen, Shu-Ching Chen, Mei-Ling Shyu, & Wickramaratna, K. (2006). Semantic event detection via multimodal data mining. IEEE Signal Processing Magazine, 23(2), 38-46. https://doi.org/10.1109/msp.2006.1621447 Go to original source...
    93. Miron-Shatz, T., Lau, A. Y. S., Paton, C., & Hansen, M. M. (2014). Big Data in Science and Healthcare: A Review of Recent Literature and Perspectives. Yearbook of Medical Informatics, 23(01), 21-26. https://doi.org/10.15265/iy-2014-0004 Go to original source...
    94. Mohd, T. K., Carvalho, J., & Javaid, A. Y. (2017). Multi-modal data fusion of Voice and EMG data for robotic control. In 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON). IEEE. https://doi.org/10.1109/uemcon.2017.8249063 Go to original source...
    95. Mohsen, F., Ali, H., El Hajj, N., & Shah, Z. (2022). Artificial intelligence-based methods for fusion of electronic health records and imaging data. Scientific Reports, 12(1), 17981. https://doi.org/10.1038/s41598-022-22514-4 Go to original source...
    96. Momoki, Y., Ichinose, A., Shigeto, Y., Honda, U., Nakamura, K., & Matsumoto, Y. (2022). Characterization of Pulmonary Nodules in Computed Tomography Images Based on Pseudo-Labeling Using Radiology Reports. IEEE Transactions on Circuits and Systems for Video Technology, 32(5), 2582-2591. https://doi.org/10.1109/tcsvt.2021.3073021 Go to original source...
    97. Muzic, R. F., Chi-Hsien Chen, & Nelson, A. D. (1998). A method to correct for scatter, spillover, and partial volume effects in region of interest analysis in PET. IEEE Transactions on Medical Imaging, 17(2), 202-213. https://doi.org/10.1109/42.700732 Go to original source...
    98. Nischitha, & Padmavathi, N. B. (2017). Fusion of multimodal abdominal cancerous images and classification using support vector machine. In 2017 International Conference on Intelligent Sustainable Systems (ICISS). IEEE. https://doi.org/10.1109/iss1.2017.8389411 Go to original source...
    99. Niu, S., Yin, Q., Song, Y., Guo, Y., & Yang, X. (2021). Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Records. In 2021 IEEE International Conference on Data Mining (ICDM). IEEE. https://doi.org/10.1109/icdm51629.2021.00056 Go to original source...
    100. Nunes, J., Mealha, O., Santos, B. S., Pereira, A. S., Botelho, M. F., & Lima, J. P. (1996). Three different approaches in visualization of multimodal clinical data. In Proceedings of 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE. https://doi.org/10.1109/iembs.1996.651906 Go to original source...
    101. Nunes, N., Martins, B., André da Silva, N., Leite, F., & J. Silva, M. (2019). A Multi-modal Deep Learning Method for Classifying Chest Radiology Exams. In Progress in Artificial Intelligence, (pp. 323-335). Springer. https://doi.org/10.1007/978-3-030-30241-2_28 Go to original source...
    102. ONC. (2016). Office-based physician electronic health record adoption. Office of the National Coordinator for Health Information Technology. https://dashboard.healthit.gov/quickstats/pages/physician-ehr-adoption-trends.php
    103. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., & McGuinness, L. A. (2021). The PRISMA 2020 statement: an Updated Guideline for Reporting Systematic Reviews. British Medical Journal, 372(71), n71. https://doi.org/10.1136/bmj.n71 Go to original source...
    104. Pai, M. M. M., Ganiga, R., Pai, R. M., & Sinha, R. K. (2021). Standard electronic health record (EHR) framework for Indian healthcare system. Health Services and Outcomes Research Methodology, 21, 339-362. https://doi.org/10.1007/s10742-020-00238-0 Go to original source...
    105. Panayides, A. S., Amini, A., Filipovic, N. D., Sharma, A., Tsaftaris, S. A., Young, A., Foran, D., Do, N., Golemati, S., Kurc, T., Huang, K., Nikita, K. S., Veasey, B. P., Zervakis, M., Saltz, J. H., & Pattichis, C. S. (2020). AI in Medical Imaging Informatics: Current Challenges and Future Directions. IEEE Journal of Biomedical and Health Informatics, 24(7), 1837-1857. https://doi.org/10.1109/JBHI.2020.2991043 Go to original source...
    106. Petersen, S. (2022). HITECH (Health Information Technology for Economic and Clinical Health) Act of 2009. https://www.techtarget.com/searchhealthit/definition/HITECH-Act
    107. Pinho, E., & Costa, C. (2016). Extensible Architecture for Multimodal Information Retrieval in Medical Imaging Archives. In 2016 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS). IEEE. https://doi.org/10.1109/sitis.2016.58 Go to original source...
    108. Philipsen, R. H. H. M., Maduskar, P., Hogeweg, L., Melendez, J., Sanchez, C. I., & van Ginneken, B. (2015). Localized Energy-Based Normalization of Medical Images: Application to Chest Radiography. IEEE Transactions on Medical Imaging, 34(9), 1965-1975. https://doi.org/10.1109/tmi.2015.2418031 Go to original source...
    109. Polikar, R., Tilley, C., Hillis, B., & Clark, C. M. (2010). Multimodal EEG, MRI and PET data fusion for Alzheimer's disease diagnosis. In 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. IEEE. https://doi.org/10.1109/iembs.2010.5627621 Go to original source...
    110. Purwar, S., Tripathi, R. K., Ranjan, R., & Saxena, R. (2020). Detection of microcytic hypochromia using cbc and blood film features extracted from convolution neural network by different classifiers. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-019-07927-0 Go to original source...
    111. Qu, R., & Xiao, Z. (2022). An Attentive Multi-Modal CNN for Brain Tumor Radiogenomic Classification. Information, 13(3), 124. https://doi.org/10.3390/info13030124 Go to original source...
    112. Raghupathi, W., & Raghupathi, V. (2014). Big data analytics in healthcare: promise and potential. Health Information Science and Systems, 2(1). https://doi.org/10.1186/2047-2501-2-3 Go to original source...
    113. Ramirez-Alonso, G., Prieto-Ordaz, O., Lopez-Santillan, R., & Montes-Y-Gomez, M. (2022). Medical Report Generation through Radiology Images: An Overview. IEEE Latin America Transactions, 20(6), 986-999. https://doi.org/10.1109/tla.2022.9757742 Go to original source...
    114. Ramesh, T., & Santhi, V. (2020). Exploring big data analytics in health care. International Journal of Intelligent Networks, 1, 135-140. https://doi.org/10.1016/j.ijin.2020.11.003 Go to original source...
    115. Ramey, J., & Robinson, C. (1991). Video-based task analysis: a tool for understanding your audience (radiology application). In IPCC 91 Proceedings the Engineered Communication. IEEE. https://doi.org/10.1109/ipcc.1991.172736 Go to original source...
    116. Razzak, M. I., Imran, M., & Xu, G. (2020). Big data analytics for preventive medicine. Neural Computing and Applications, 32(9), 4417-4451. https://doi.org/10.1007/s00521-019-04095-y Go to original source...
    117. Reda, I., Khalil, A., Elmogy, M., Abou El-Fetouh, A., Shalaby, A., Abou El-Ghar, M., Elmaghraby, A., Ghazal, M., & El-Baz, A. (2018). Deep Learning Role in Early Diagnosis of Prostate Cancer. Technology in Cancer Research & Treatment, 17, 153303461877553. https://doi.org/10.1177/1533034618775530 Go to original source...
    118. Rehman, A., Naz, S., & Razzak, I. (2022). Leveraging big data analytics in healthcare enhancement: trends, challenges and opportunities. Multimedia Systems, 28(4), 1-33. https://doi.org/10.1007/s00530-020-00736-8 Go to original source...
    119. Ribeiro, R. T., Marinho, R. T., & Sanches, J. M. (2013). Classification and Staging of Chronic Liver Disease From Multimodal Data. IEEE Transactions on Biomedical Engineering, 60(5), 1336-1344. https://doi.org/10.1109/tbme.2012.2235438 Go to original source...
    120. Ros Maryana Sinaga, A. S., & Putra, R. E. (2022). Predictive Analytic Healthcare Sector Using Classification Machine Learning Algorithm. In 2022 International Symposium on Information Technology and Digital Innovation (ISITDI). IEEE. https://doi.org/10.1109/isitdi55734.2022.9944492 Go to original source...
    121. Rosenzweig, S., Scholand, N., Holme, H. C. M., & Uecker, M. (2020). Cardiac and Respiratory Self-Gating in Radial MRI Using an Adapted Singular Spectrum Analysis (SSA-FARY). IEEE Transactions on Medical Imaging, 39(10), 3029-3041. https://doi.org/10.1109/tmi.2020.2985994 Go to original source...
    122. Saxena, A., Singh Tomar, S., Jain, G., & Gupta, R. (2021). Deep learning based Diagnosis of diseases using Image Classification. In 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence). IEEE. https://doi.org/10.1109/confluence51648.2021.9377154 Go to original source...
    123. Schiff, G. D. et al. (2009). Diagnostic Error in Medicine: Analysis of 583 physician-reported errors. Archives of Internal Medicine, 169(20), 1881-1887. https://doi.org/10.1001/archinternmed.2009.333 Go to original source...
    124. Setio, A. A. A., Ciompi, F., Litjens, G., Gerke, P., Jacobs, C., van Riel, S. J., Wille, M. M. W., Naqibullah, M., Sanchez, C. I., & van Ginneken, B. (2016). Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks. IEEE Transactions on Medical Imaging, 35(5), 1160-1169. https://doi.org/10.1109/tmi.2016.2536809 Go to original source...
    125. Shachor, Y., Greenspan, H., & Goldberger, J. (2020). A mixture of views network with applications to multi-view medical imaging. Neurocomputing, 374, 1-9. https://doi.org/10.1016/j.neucom.2019.09.027 Go to original source...
    126. Shetty, S., Ananthanarayana, V. S., & Mahale, A. (2020). Medical Knowledge-Based Deep Learning Framework for Disease Prediction on Unstructured Radiology Free-Text Reports Under Low Data Condition. In Proceedings of the 21st EANN (Engineering Applications of Neural Networks) 2020 Conference, (pp. 352-364). Springer. https://doi.org/10.1007/978-3-030-48791-1_27 Go to original source...
    127. Shetty, S., S., A. V., & Mahale, A. (2022). MS-CheXNet: An Explainable and Lightweight Multi-Scale Dilated Network with Depthwise Separable Convolution for Prediction of Pulmonary Abnormalities in Chest Radiographs. Mathematics, 10(19), 3646. https://doi.org/10.3390/math10193646 Go to original source...
    128. Shin, H.-C., Le Lu, Kim, L., Seff, A., Yao, J., & Summers, R. M. (2015). Interleaved text/image Deep Mining on a large-scale radiology database. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. https://doi.org/10.1109/cvpr.2015.7298712 Go to original source...
    129. Sirshar, M., Paracha, M. F. K., Akram, M. U., Alghamdi, N. S., Zaidi, S. Z. Y., & Fatima, T. (2022). Attention based automated radiology report generation using CNN and LSTM. PLOS ONE, 17(1), e0262209. https://doi.org/10.1371/journal.pone.0262209 Go to original source...
    130. Skowron, A., Wang, H., Wojna, A., & Bazan, J. (2005). A Hierarchical Approach to Multimodal Classification. In Lecture Notes in Computer Science, (pp. 119-127). Springer. https://doi.org/10.1007/11548706_13 Go to original source...
    131. Soenksen, L. R., Ma, Y., Zeng, C., Boussioux, L., Villalobos Carballo, K., Na, L., Wiberg, H. M., Li, M. L., Fuentes, I., & Bertsimas, D. (2022). Integrated multimodal artificial intelligence framework for healthcare applications. Npj Digital Medicine, 5(1). https://doi.org/10.1038/s41746-022-00689-4 Go to original source...
    132. Song, J., Zheng, J., Li, P., Lu, X., Zhu, G., & Shen, P. (2021). An Effective Multimodal Image Fusion Method Using MRI and PET for Alzheimer's Disease Diagnosis. Frontiers in Digital Health, 3. https://doi.org/10.3389/fdgth.2021.637386 Go to original source...
    133. Sreejith, S., Khanna Nehemiah, H., & Kannan, A. (2022). A clinical decision support system for polycystic ovarian syndrome using red deer algorithm and random forest classifier. Healthcare Analytics, 2, 100102. https://doi.org/10.1016/j.health.2022.100102 Go to original source...
    134. Srivastava, N., & Salakhutdinov, R. (2014). Multimodal Learning with Deep Boltzmann Machines. The Journal of Machine Learning Research, 15(1), 2949-2980.
    135. Sultanum, N., Naeem, F., Brudno, M., & Chevalier, F. (2023). ChartWalk: Navigating large collections of text notes in electronic health records for clinical chart review. IEEE Transactions on Visualization and Computer Graphics, 29(1), 1244-1254. https://doi.org/10.1109/tvcg.2022.3209444 Go to original source...
    136. Sundararaman, A., Valady Ramanathan, S., & Thati, R. (2018). Novel Approach to Predict Hospital Readmissions Using Feature Selection from Unstructured Data with Class Imbalance. Big Data Research, 13, 65-75. https://doi.org/10.1016/j.bdr.2018.05.004 Go to original source...
    137. Sutton, R., Pincock, D., Baumgart, D., Sadowski, D., Fedorak, R., & Kroeker, K. (2020). An overview of clinical decision support systems: Benefits, risks, and strategies for success. NPJ Digital Medicine, 3(1), 1-10. https://doi.org/10.1038/s41746-020-0221-y Go to original source...
    138. Syeda-Mahmood, T., Wang, F., Beymer, D., Amir, A., Richmond, M., & Hashmi, S. N. (2007). AALIM: Multimodal mining for cardiac decision support. In 2007 Computers in Cardiology, (pp. 209-212). IEEE. https://doi.org/10.1109/cic.2007.4745458 Go to original source...
    139. Tabassum, S., Abedin, N., Rahman, M. M., Rahman, M. M., Ahmed, M. T., Islam, R., & Ahmed, A. (2022). An online cursive handwritten medical words recognition system for busy doctors in developing countries for ensuring efficient healthcare service delivery. Scientific Reports, 12(1), 3601. https://doi.org/10.1038/s41598-022-07571-z Go to original source...
    140. Taimoor, N., & Rehman, S. (2022). Reliable and Resilient AI and IoT-Based Personalised Healthcare Services: A Survey. IEEE Access, 10, 535-563. https://doi.org/10.1109/access.2021.3137364 Go to original source...
    141. Turkeli, S., Akkoca Gazioglu, B. S., Kurt, K. K., Atay, H. T., & Gorur, Y. (2017). Mining similar radiology reports using BoW and Fuzzy C-means clustering. In 2017 International Artificial Intelligence and Data Processing Symposium (IDAP). IEEE. https://doi.org/10.1109/idap.2017.8090213 Go to original source...
    142. Valova, I., & Kosugi, Y. (1997). MR brain image classification by multimodal perceptron tree neural network. Neural Networks for Signal Processing VII. In Proceedings of the 1997 IEEE Signal Processing Society Workshop. IEEE. https://doi.org/10.1109/nnsp.1997.622398 Go to original source...
    143. van der Putten, N., Vinke, R., Citroen, M., Cornet, R., van Mulligen, E., & den Boer, A. (1995). Integrating medical images, biosignals, and alphanumeric data in a cardiological department. Computers in Cardiology, 1995. https://doi.org/10.1109/cic.1995.482748 Go to original source...
    144. van der Voort, S. R., Gahrmann, R., van den Bent, M. J., Vincent, A. J. P. E., Niessen, W. J., Smits, M., & Klein, S. (2017). Radiogenomic classification of the 1p/19q status in presumed low-grade gliomas. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE. https://doi.org/10.1109/isbi.2017.7950601 Go to original source...
    145. Wang, F., Syeda-Mahmood, T., & Beymer, D. (2009). Information Extraction from Multimodal ECG Documents. In 2009 10th International Conference on Document Analysis and Recognition. IEEE. https://doi.org/10.1109/icdar.2009.189 Go to original source...
    146. Wang, X., Wang, Y., Gao, C., Lin, K., & Li, Y. (2018). Automatic Diagnosis With Efficient Medical Case Searching Based on Evolving Graphs. IEEE Access, 6, 53307-53318. https://doi.org/10.1109/access.2018.2871769 Go to original source...
    147. Wang, M., Zhang, D., Huang, J., Yap, P.-T., Shen, D., & Liu, M. (2020). Identifying Autism Spectrum Disorder With Multi-Site fMRI via Low-Rank Domain Adaptation. IEEE Transactions on Medical Imaging, 39(3), 644-655. https://doi.org/10.1109/tmi.2019.2933160 Go to original source...
    148. Weijie Chen, Metz, C. E., Giger, M. L., & Drukker, K. (2010). A Novel Hybrid Linear/Nonlinear Classifier for Two-Class Classification: Theory, Algorithm, and Applications. IEEE Transactions on Medical Imaging, 29(2), 428-441. https://doi.org/10.1109/tmi.2009.2033596 Go to original source...
    149. Weibel, N., Ashfaq, S., Calvitti, A., Hollan, J., & Agha, Z. (2013). Multimodal Data Analysis and Visualization to Study the Usage of Electronic Health Records. In Proceedings of the ICTs for Improving Patients Rehabilitation Research Techniques. EUDL. https://doi.org/10.4108/icst.pervasivehealth.2013.252025 Go to original source...
    150. White, T. (2012). Hadoop: The Definitive Guide. O'Reilly Media.
    151. Wood, C., Cross, M., & Phuong La. (1998). Multimodal information retrieval, extraction and generation for use in the health domain. In Proceedings of the Second International Conference, Knowledge-Based Intelligent Electronic Systems. IEEE. https://doi.org/10.1109/kes.1998.725988 Go to original source...
    152. Wu, H., & He, L. (2015). Combining visual and textual features for medical image modality classification with ℓp-norm multiple kernel learning. Neurocomputing, 147, 387-394. https://doi.org/10.1016/j.neucom.2014.06.046 Go to original source...
    153. Wu, N., Phang, J., Park, J., Shen, Y., Huang, Z., Zorin, M., Jastrzebski, S., Fevry, T., Katsnelson, J., Kim, E., Wolfson, S., Parikh, U., Gaddam, S., Lin, L. L. Y., Ho, K., Weinstein, J. D., Reig, B., Gao, Y., Toth, H., & Pysarenko, K. (2020). Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening. IEEE Transactions on Medical Imaging, 39(4), 1184-1194. https://doi.org/10.1109/tmi.2019.2945514 Go to original source...
    154. Wu, M., Zhong, X., Peng, Q., Xu, M., Huang, S., Yuan, J., Ma, J., & Tan, T. (2019). Prediction of molecular subtypes of breast cancer using BI-RADS features based on a "white box" machine learning approach in a multi-modal imaging setting. European Journal of Radiology, 114, 175-184. https://doi.org/10.1016/j.ejrad.2019.03.015 Go to original source...
    155. Xi, X., Xu, H., Shi, H., Zhang, C., Ding, H. Y., Zhang, G., Tang, Y., & Yin, Y. (2017). Robust texture analysis of multi-modal images using Local Structure Preserving Ranklet and multi-task learning for breast tumor diagnosis. Neurocomputing, 259, 210-218. https://doi.org/10.1016/j.neucom.2016.06.082 Go to original source...
    156. Yi, S., Zhang, G., Qian, C., Lu, Y., Zhong, H., & He, J. (2022). A Multimodal Classification Architecture for the Severity Diagnosis of Glaucoma Based on Deep Learning. Frontiers in Neuroscience, 16, 939472. https://doi.org/10.3389/fnins.2022.939472 Go to original source...
    157. Zhang, D., & Shen, D. (2011). Semi-supervised multimodal classification of alzheimer's disease. In 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro (pp. 1628-1631). IEEE. https://doi.org/10.1109/isbi.2011.5872715 Go to original source...
    158. Zhang, Y.-D., Dong, Z., Wang, S.-H., Yu, X., Yao, X., Zhou, Q., Hu, H., Li, M., Jiménez-Mesa, C., Ramirez, J., Martinez, F. J., & Gorriz, J. M. (2020). Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation. Information Fusion, 64, 149-187. https://doi.org/10.1016/j.inffus.2020.07.006 Go to original source...
    159. Zhou, X., Li, Y., & Liang, W. (2021). CNN-RNN Based Intelligent Recommendation for Online Medical Pre-Diagnosis Support. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 18(3), 912-921. https://doi.org/10.1109/tcbb.2020.2994780 Go to original source...
    160. Zhu, T., Li, L., Yang, J., Zhao, S., Liu, H., & Qian, J. (2022). Multimodal Sentiment Analysis With Image-Text Interaction Network. IEEE Transactions on Multimedia, (in press). https://doi.org/10.1109/tmm.2022.3160060 Go to original source...
    161. Zwettler, G., & Backfrieder, W. (2015). Evolution strategy classification utilizing meta features and domain-specific statistical a priori models for fully-automated and entire segmentation of medical datasets in 3D radiology. In 2015 International Conference on Computing and Communications Technologies (ICCCT). IEEE. https://doi.org/10.1109/iccct2.2015.7292712 Go to original source...

    This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits use, distribution, and reproduction in any medium, provided the original publication is properly cited. No use, distribution or reproduction is permitted which does not comply with these terms.


    Return to the content