Medical image super-resolution reconstruction: A comprehensive investigation of Generative Adversarial Networks

Research Article
Open access

Medical image super-resolution reconstruction: A comprehensive investigation of Generative Adversarial Networks

Yuntian Bao 1*
  • 1 Capital University of Economics and Business    
  • *corresponding author 32021230040@cueb.edu.cn
Published on 25 March 2024 | https://doi.org/10.54254/2755-2721/51/20241148
ACE Vol.51
ISSN (Print): 2755-273X
ISSN (Online): 2755-2721
ISBN (Print): 978-1-83558-347-0
ISBN (Online): 978-1-83558-348-7

Abstract

Medical images play a crucial role in modern healthcare diagnostics and treatment. However, many medical images suffer from limitations in resolution, potentially impeding a comprehensive understanding of a patient's condition by healthcare professionals. This comprehensive review delves into the applications of Generative Adversarial Networks (GANs) in medical image super-resolution reconstruction to address this challenge. In the Methods section, this paper first focused on the direction of medical image classification, including cell classification of histopathological images and synthetic data enhancement using GANs to improve liver lesion classification. Subsequently, this paper focused on the direction of medical image segmentation, looking into the use of Structure-Corrected Adversarial Networks (SCAN) for organ segmentation in chest radiographs and Deep Adversarial Networks for biomedical image segmentation using unannotated images. In the Applications and Discussion section, this paper thoroughly examined the current progress of GANs in telemedicine diagnosis and disease state generation and prediction. This paper emphasized the significant potential of GAN technology in telemedicine while outlining the current constraints and challenges. Furthermore, this paper highlighted the prospects of GANs in medical image super-resolution reconstruction and how they affect the discipline of medical imaging. This comprehensive review consolidates the latest research findings on GANs in medical image super-resolution reconstruction, underscoring their importance in the realm of healthcare. By critically analysing existing literature, this paper provides valuable insights for medical image analysts are researchers while inspiring future research directions and innovations.

Keywords:

Medical Imaging, Super Resolution, Machine Learning

Bao,Y. (2024). Medical image super-resolution reconstruction: A comprehensive investigation of Generative Adversarial Networks. Applied and Computational Engineering,51,14-19.
Export citation

1. Introduction

A crucial part of the medical industry's ongoing development is played by medical imaging. It is utilized for more than just clinical disease screening, diagnosis, and therapy advice and effect evaluation, but also an indispensable tool in the field of medicine. However, conventional diagnostic imaging has some drawbacks, such as over-reliance on the experience and level of the doctor who reads the film, low repeatability, high subjectivity, and insufficient quantitative analysis. Therefore, there is an urgent need to introduce more advanced intelligent technologies to improve the accuracy and efficiency of doctors' diagnosis [1]. Against this background, super-resolution image reconstruction techniques are emerging as a powerful tool for solving medical image quality problems. The field of medical imaging has embraced the research direction of super-resolution image reconstruction, which involves transforming low-resolution images into high-resolution images using image processing methods. It not only improves the visual quality of images, but also facilitates in extracting more accurate medical information, thus providing a more reliable basis for clinical medical decision-making. However, traditional super-resolution methods face many challenges in medical image processing, such as noise, image distortion, and the complex structure unique to medical images.

And the rise of Generative Adversarial Networks (GAN) has brought new possibilities for super-resolution reconstruction of medical images [2]. GAN is a network architecture consisting of a generator and a discriminator, and through the adversarial training process between the two, was first presented by Ian Goodfellow et al. in 2014, it is able to generate more realistic and higher resolution images. The introduction of this new technology will also bring more expansion for the application of medical imaging, so as to better protect and serve the health of patients. For instance, in Magnetic Resonance Imaging (MRI), high-resolution images can reveal more detailed tissue structures, helping to detect the location and extent of lesions [3]. Similarly, super-resolution reconstruction of Computed Tomography (CT) images can provide more accurate anatomical information to support surgical planning and radiotherapy treatment. In addition, super-resolution image reconstruction can enhance the visualization of medical images, making it easier for physicians to observe minute lesions or abnormalities. However, the field of medical imaging often suffers from less data, missing data and uneven categorization [4]. The objective of this study is to discuss recent developments in deep learning methods research in the area of medical imaging, as well as the use of such approaches in this sector.

The rest of this chapter is organized as follows. First, in Section 2, This study will introduce the relevant methods, basic concepts and application scenarios of GAN super-resolution in different areas such as classification and segmentation. It will also summarize and review the existing results, and explore in detail the application cases of generative adversarial networks in medical image super-resolution reconstruction, including the super-resolution effect and improvement of different types of medical images. Then, in Section 3, I will discuss the applications and future perspectives of GAN super-resolution and summarize the results of the case study to investigate its future directions, i.e., telemedicine, case image generation prediction, and other future trends in the field of medicine.

2. Method

In this section, different methods applied in the field of medical image super-resolution reconstruction will be presented in detail. This review will mainly categorize the classification super-resolution, segmentation super-resolution and other related methods, and discuss their basic concepts, application ideas and the role of GAN in them, respectively.

2.1. Medical Image Classification Direction

In this subsection, this paper will focus on the classification super-resolution method. This approach aims to achieve image super-resolution reconstruction by classifying medical images. The GAN-based classification super-resolution method provides an innovative idea to reconstruct high-resolution images from low-resolution images by generating generators and discriminators in adversarial networks.

2.1.1. Cell classification of histopathological images

In 2018, Hu et al. [5] presented an innovative study employing GAN for cellular classification of histopathology images. Their goal was to achieve robust cell-level visual representation learning in an unsupervised environment. In the field of pathology images, they fused two different types of GAN techniques to achieve superior image classification.

The core model of the study consists of three main components: generator G, discriminator D, and auxiliary network Q. The authors skillfully combine the stability of gradient penalized generative adversarial networks (Wasserstein GAN-gradient penalty, WGAN-GP) and the interpretability of mutually informative generative adversarial networks (mutual information GAN, InfoGAN) to achieve synergistic enhancement of model performance. During the training process, the generator and the discriminator are confronted with each other to allow the discriminator to learn the features of the image. Subsequently, the authors pass the learning parameters of the discriminator to the auxiliary network Q. The output of the auxiliary network Q is constrained by a specifically designed loss function to enable it to learn the intrinsic associations between images and categories. As a result, after training is completed, the auxiliary network Q becomes a classifier that outputs its corresponding categorization information after inputting cell images.

2.1.2. Enhancing synthetic data with GAN to improve liver lesion classification

Frid-Adar et al. [6] proposed a synthetic data enhancement method for medical images based on GAN. Their approach first extends the training dataset using traditional data enhancement techniques and then introduces GAN techniques for synthetic data enhancement to further increase the amount and diversity of data. This method was validated on a limited dataset on computed tomography (CT) images.

2.2. Medical Image Segmentation Direction

Medical image segmentation divides an image based on texture, color, and other features, thus slicing the image into regions with different characteristics and extracting the parts of interest. This process is crucial in image processing and analysis, and not only constitutes the basis for image feature extraction, parameter selection, and target recognition, but is also a prerequisite for the realization of these tasks [7]. However, traditional medical image segmentation models usually require a large number of labeled images for training, and obtaining labeled medical images is often quite challenging. Typically, only a limited number of labeled images can be obtained, while the vast majority of images are unlabeled.

2.2.1. Deep Adversarial Networks are used to segment biomedical images using unannotated images

A Deep Adversarial Network (DAN) model for biomedical image segmentation was creatively introduce in 2017. Its goal was to produce consistent and accurate segmentation of both tagged and unlabeled pictures. Segmentation Network (SN) and Evaluation Network (EN) are the two main components of the model, which are used for image segmentation and evaluation of segmentation quality, respectively, to achieve a higher level of segmentation performance.

During the training of the model, the research provides the EN module a special task of assigning different scores to the segmentation results of unlabeled and labeled images for differentiation purposes. At the same time, they also guided the SN module to produce segmentation results for unlabeled images to make it challenging for EN to distinguish these results from those of labeled images. Through gradual adversarial training, the EN module continuously "judges" the segmentation results of unlabeled images, thus forcing the SN module to generate more accurate segmentation results on unlabeled and unseen samples. Experiments demonstrated that this innovative DAN model can effectively obtain more accurate segmentation results from unlabeled image data.

2.2.2. Adversarial Network for Precise Organ Segmentation in Chest Radiography Using Structural Correction

In 2018, a new GAN-based structure-corrected adversarial network (SCAN) was proposed and applied to the segmentation task of chest X-ray images. The core idea of SCAN is to introduce the inherent structure of human physiology into the convolutional segmentation network, and through adversarial training, under the guidance of the discriminative network, the segmentation network is able to realize a more realistic approximation of the actual segmentation, so as to achieve the effect of mimicking the real-life scene. The uniqueness of this method is that it can instantly obtain highly accurate and realistic segmentation results without the need for pre-training models, reducing the dependence on the number of samples. In addition, the method breaks through the limitations of the current leading methods in an innovative way by being applicable to chest X-ray image segmentation not only under different disease characteristics, but also from different patient groups. This comprehensive applicability makes SCAN promising for a wide range of applications in the field of medical image segmentation.

3. Applications and discussion

In this section, this review will delve into the practical applications of the methods discussed in the previous sections and comprehensively discuss their potential impact on the field of medical image analysis.

3.1. Telemedicine diagnosis

Telemedicine diagnostics has made significant progress in recent years, driven by advanced technologies. Integration of deep learning techniques, such as GAN, has great potential to extend high-quality healthcare to less developed areas and improve accessibility to patients far from healthcare facilities.

3.1.1. Current progress

Telemedicine diagnostics, a fusion of technology and medical necessity, sees notable advancements. The integration of GAN-based techniques for medical image analysis markedly enhances image quality and diagnostic precision. Researchers and practitioners leverage GANs to tackle issues concerning image quality, artifact reduction, and diagnostic accuracy. In recent research, Wang et al. [8] explored telemedicine's application in Parkinson's patient care, while He et al. [9] delved into glaucoma diagnosis and management through telemedicine. Both studies emphasize telemedicine's advantages, including convenient access to specialists, time efficiency, and patient convenience. These strides signify the promising trajectory of telemedicine and its integration with GAN-based methodologies.

3.1.2. Constraints and challenges

However, applying GAN-based methods in telemedicine diagnostics faces several challenges. Concerns include data privacy, network connectivity, and regulatory compliance. In the United States, only about 59% of adults aged 65 and older have home broadband access due to limited technological development and a digital divide [10]. Globally, an UN-backed report [11] indicates that 54.8% of households have Internet access, often at speeds as low as 256kbps, leaving 3.7 billion people, mainly in developing regions, without Internet access. Moreover, the absence of healthcare regulations and standardized rules for mHealth systems can lead to issues like identity theft and eavesdropping due to patient data breaches [12]. Ensuring the clinical accuracy and representativeness of generated medical images remains a significant concern. Addressing these issues is essential for establishing trust and reliability in GAN-assisted remote diagnosis.

3.1.3. Future prospects

Considering the current state and challenges of telemedicine, this study envisions a transformative role for GAN in telemedicine diagnosis. As edge computing advances and real-time image analysis improves, faster and more accurate diagnoses become feasible. Collaborative efforts among technicians, healthcare providers, and policymakers will be vital to establish regulatory frameworks and ensure ethical GAN-based telediagnostics implementation. This partnership will address technical, regulatory, and ethical challenges, fostering GAN's widespread adoption in telemedicine. This collaboration safeguards patient rights, enhances healthcare quality, and drives innovation. Upgrading hardware systems, including network speed and firewalls, is essential for telemedicine's further expansion.

3.2. Condition generation and prediction

GANs in healthcare are applied to generate virtual medical image samples, aiding understanding of image changes in various diseases and predicting disease progression. This assists clinical decisions with accuracy.

3.2.1. Current progress

GANs in medical imaging offer innovative disease generation and prediction. GANs create virtual medical image samples, mirroring disease-induced image changes, aiding comprehensive disease understanding. Moreover, GANs predict patient condition trends by analyzing image features and clinical data, building predictive models. Recent research, by Liu et al. [13], presents MiSrc-GAN, a novel medical image generation model. MiSrc-GAN comprises three core components: an asymptotic generator for accuracy, a multi-scale discriminator, and an adversarial sample pair construction module. This integration enhances the discriminator's robustness while optimally learning joint probability distributions of original and generated images.

3.2.2. Constraints and challenges

In disease generation and prediction, challenges persist. Generative adversarial networks demand abundant high-quality medical image data for training, ensuring faithful virtual sample generation. However, obtaining and annotating medical image data is costly and time-consuming, potentially limiting data availability. Disease complexities yield case differences, affecting GAN-generated virtual sample accuracy. Additionally, existing models haven't yet comprehensively integrated domain-specific information in terms of feature representation, necessitating deeper domain feature learning. While the introduction of the adversarial sample technique partly mitigates the lack of labeled medical image data, there's room for image generation quality enhancement.

3.2.3. Future prospects

In the future, as medical imaging data accumulates and technology advances, the application of generative adversarial networks in this field will become more refined and precise. With a growing volume of data, GAN-generated virtual samples will better capture the diversity of various disease states. Moreover, through the integration of clinical data and multimodal information [14, 15], prediction models will enhance accuracy, offering physicians earlier intervention and treatment strategies. Finally, effectively harnessing more unlabeled data, including knowledge transfer across diverse data domains, emerges as a prominent focus for future research.

4. Conclusion

This paper presents a comprehensive review of super-resolution reconstruction techniques for medical images, especially Generative Adversarial Network-based methods. Through an exploration of medical image classification, segmentation, and the application of GANs in this domain, the potential of GAN techniques in enhancing medical image quality and diagnostic accuracy becomes evident. Within the discussion and applications section, two critical domains are examined: telemedicine diagnostics and disease generation and prediction. Notable attention is placed on the role of GANs in telemedicine, accompanied by the presentation of challenges, including data privacy, network connectivity, and regulatory compliance. The promise of disease generation and prediction is also highlighted, alongside the mention of limitations like data access and model complexity. Future efforts are dedicated to overcoming technical and data-related constraints, fostering innovation and progress in medical image processing. The objective is to advance medical image quality, expedite diagnostic procedures, and yield more favorable outcomes in telemedicine and disease prediction, further deepening our grasp and application of GAN technology.


References

[1]. Shi J Wang L L Wang S S Chen Y X Wang Q Wei D M & Shen D G 2020 Applications of deep learning in medical imaging: a survey Journal of Image and Graphics 25(10) pp 1953-1981

[2]. Goodfellow I Pouget-Abadie J Mirza M Xu B Warde-Farley D Ozair S & Bengio Y 2014 Generative adversarial nets Advances in neural information processing systems p 27

[3]. Liu F D Du F & Cheng H Q 2018 Application and research progress of texture analysis based on MRI images Chinese Journal of Medical Computer Imaging 24(5) pp 426-429

[4]. Chen F Y Gao J Y Zhao D Wang H Z Song H & Su Q H 2021 Deep learning and Biomedical image Analysis 2020 Review

[5]. Hu B Tang Y Eric I Chang C Fan Y Lai M & Xu Y 2018 Unsupervised learning for cell-level visual representation in histopathology images with generative adversarial networks IEEE journal of biomedical and health informatics 23(3) pp 1316-1328

[6]. Frid-Adar M Klang E Amitai M Goldberger J & Greenspan H 2018 April Synthetic data augmentation using GAN for improved liver lesion classification In 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018) (pp 289-293) IEEE

[7]. Tang S Y Xing J F & Yang M 2017 A new method of medical image segmentation based on BP neural network Computer Science (S1) pp 240-243

[8]. Wang M J & Zhan T T 2023 Application of telemedicine in patients with Parkinson's disease Advances in Clinical Medicine 13 p 7960

[9]. He M & Tang L 2023 Application of telemedicine in glaucoma diagnosis and treatment management Journal of Otolaryngology Shandong University 37(4) pp 187-195

[10]. Larson D N Schneider R B & Simuni T 2021 A new era: the growth of video-based visits for remote management of persons with Parkinson’s disease Journal of Parkinson's disease 11(s1) S27-S34

[11]. Shalash A Spindler M & Cubo E 2021 Global perspective on telemedicine for Parkinson’s disease Journal of Parkinson's Disease 11(s1) S11-S18

[12]. Katusiime J & Pinkwart N 2019 A review of privacy and usability issues in mobile health systems: Role of external factors Health informatics journal 25(3) pp 935-950

[13]. Liu S P Zhao H M Hong J M Wu X H Xu F B OuYang J & Xiong J B 2023 Robust Conditional Generation Adversarial network for medical image generation Acta Electronica Sinica 51(2) p 427

[14]. Qiu Y et al 2022 Pose-guided matching based on deep learning for assessing quality of action on rehabilitation training Biomedical Signal Processing and Control 72 103323

[15]. Khare Y et al 2021 Mmbert: Multimodal bert pretraining for improved medical vqa. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) pp 1033-1036 IEEE


Cite this article

Bao,Y. (2024). Medical image super-resolution reconstruction: A comprehensive investigation of Generative Adversarial Networks. Applied and Computational Engineering,51,14-19.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

Disclaimer/Publisher's Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

About volume

Volume title: Proceedings of the 4th International Conference on Signal Processing and Machine Learning

ISBN:978-1-83558-347-0(Print) / 978-1-83558-348-7(Online)
Editor:Marwan Omar
Conference website: https://www.confspml.org/
Conference date: 15 January 2024
Series: Applied and Computational Engineering
Volume number: Vol.51
ISSN:2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

References

[1]. Shi J Wang L L Wang S S Chen Y X Wang Q Wei D M & Shen D G 2020 Applications of deep learning in medical imaging: a survey Journal of Image and Graphics 25(10) pp 1953-1981

[2]. Goodfellow I Pouget-Abadie J Mirza M Xu B Warde-Farley D Ozair S & Bengio Y 2014 Generative adversarial nets Advances in neural information processing systems p 27

[3]. Liu F D Du F & Cheng H Q 2018 Application and research progress of texture analysis based on MRI images Chinese Journal of Medical Computer Imaging 24(5) pp 426-429

[4]. Chen F Y Gao J Y Zhao D Wang H Z Song H & Su Q H 2021 Deep learning and Biomedical image Analysis 2020 Review

[5]. Hu B Tang Y Eric I Chang C Fan Y Lai M & Xu Y 2018 Unsupervised learning for cell-level visual representation in histopathology images with generative adversarial networks IEEE journal of biomedical and health informatics 23(3) pp 1316-1328

[6]. Frid-Adar M Klang E Amitai M Goldberger J & Greenspan H 2018 April Synthetic data augmentation using GAN for improved liver lesion classification In 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018) (pp 289-293) IEEE

[7]. Tang S Y Xing J F & Yang M 2017 A new method of medical image segmentation based on BP neural network Computer Science (S1) pp 240-243

[8]. Wang M J & Zhan T T 2023 Application of telemedicine in patients with Parkinson's disease Advances in Clinical Medicine 13 p 7960

[9]. He M & Tang L 2023 Application of telemedicine in glaucoma diagnosis and treatment management Journal of Otolaryngology Shandong University 37(4) pp 187-195

[10]. Larson D N Schneider R B & Simuni T 2021 A new era: the growth of video-based visits for remote management of persons with Parkinson’s disease Journal of Parkinson's disease 11(s1) S27-S34

[11]. Shalash A Spindler M & Cubo E 2021 Global perspective on telemedicine for Parkinson’s disease Journal of Parkinson's Disease 11(s1) S11-S18

[12]. Katusiime J & Pinkwart N 2019 A review of privacy and usability issues in mobile health systems: Role of external factors Health informatics journal 25(3) pp 935-950

[13]. Liu S P Zhao H M Hong J M Wu X H Xu F B OuYang J & Xiong J B 2023 Robust Conditional Generation Adversarial network for medical image generation Acta Electronica Sinica 51(2) p 427

[14]. Qiu Y et al 2022 Pose-guided matching based on deep learning for assessing quality of action on rehabilitation training Biomedical Signal Processing and Control 72 103323

[15]. Khare Y et al 2021 Mmbert: Multimodal bert pretraining for improved medical vqa. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) pp 1033-1036 IEEE