Download pdf
Uncertainty-Aware High-Fidelity Anatomical MRI Synthesis using Deep Convolutional Network with Monte Carlo Dropout
- 1 Harrow International School Shenzhen Qianhai, Shenzhen, China
* Author to whom correspondence should be addressed.
Abstract
Multi-modality high-resolution MRI is beneficial for studying the brain structure and function in research and clinical settings. However, its acquisition is time-consuming, which reduces its feasibility for wider adoption especially for certain populations who cannot tolerate long scans. In this study, we propose a convolution neural network to obtain high-resolution T1-weighted MRI from lower-resolution T2-weighted input that can be acquired within a shorter scan time. By leveraging Monte Carlo dropout, our model not only produces high-fidelity anatomical T1-weighted image with higher accuracy compared to baseline model, but also generates uncertainty estimation similar to the actual error map. Our method is validated on the Human Connectome Project, and the experiments indicate our method has the potential to improve the robustness and reliability of deep learning image synthesis and accurately accelerate multi-modality MRI to benefit research and clinical practice.
Keywords
Deep Learning, Multi-contrast MRI, Convolutional Neural Network Image Synthesis, Monte Carlo Dropout
[1]. Fischl, B. (2012). FreeSurfer. Neuroimage, 62(2), 774-781.
[2]. Filippi, M., Paty, D. W., Kappos, L., Barkhof, F., Compston, D. A. S., Thompson, A. J., ... & Miller, D. H. (1995). Correlations between changes in disability and T2‐weighted brain MRI activity in multiple sclerosis: a follow‐up study. Neurology, 45(2), 255-260.
[3]. Brown, R. W., Cheng, Y. C. N., Haacke, E. M., Thompson, M. R., & Venkatesan, R. (2014). Magnetic resonance imaging: physical principles and sequence design. John Wiley & Sons.
[4]. Setsompop, K., Gagoski, B. A., Polimeni, J. R., Witzel, T., Wedeen, V. J., & Wald, L. L. (2012). Blipped‐controlled aliasing in parallel imaging for simultaneous multislice echo planar imaging with reduced g‐factor penalty. Magnetic resonance in medicine, 67(5), 1210-1224.
[5]. Bilgic, B., Gagoski, B. A., Cauley, S. F., Fan, A. P., Polimeni, J. R., Grant, P. E., ... & Setsompop, K. (2015). Wave‐CAIPI for highly accelerated 3D imaging. Magnetic resonance in medicine, 73(6), 2152-2162.
[6]. Lustig, M., Donoho, D. L., Santos, J. M., & Pauly, J. M. (2008). Compressed sensing MRI. IEEE signal processing magazine, 25(2), 72-82.
[7]. Emami, H., Dong, M., Nejad‐Davarani, S. P., & Glide‐Hurst, C. K. (2018). Generating synthetic CTs from magnetic resonance images using generative adversarial networks. Medical physics, 45(8), 3627-3636.
[8]. Choi, H., & Lee, D. S. (2018). Generation of structural MR images from amyloid PET: application to MR-less quantification. Journal of Nuclear Medicine, 59(7), 1111-1117.
[9]. Dai, X., Lei, Y., Fu, Y., Curran, W. J., Liu, T., Mao, H., & Yang, X. (2020). Multimodal MRI synthesis using unified generative adversarial networks. Medical physics, 47(12), 6343-6354.
[10]. Iglesias, J. E., Billot, B., Balbastre, Y., Tabari, A., Conklin, J., González, R. G., ... & Alzheimer’s Disease Neuroimaging Initiative. (2021). Joint super-resolution and synthesis of 1 mm isotropic MP-RAGE volumes from clinical MRI exams with scans of different orientation, resolution and contrast. Neuroimage, 237, 118206.
[11]. Yurt, M., Dar, S. U., Erdem, A., Erdem, E., Oguz, K. K., & Çukur, T. (2021). mustGAN: multi-stream generative adversarial networks for MR image synthesis. Medical image analysis, 70, 101944.
[12]. Cohen, J. P., Luck, M., & Honari, S. (2018). Distribution matching losses can hallucinate features in medical image translation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part I (pp. 529-536). Springer International Publishing.
[13]. Tanno, R., Worrall, D. E., Kaden, E., Ghosh, A., Grussu, F., Bizzi, A., ... & Alexander, D. C. (2021). Uncertainty modelling in deep learning for safer neuroimage enhancement: Demonstration in diffusion MRI. NeuroImage, 225, 117366.
[14]. Schlemper, J., Castro, D. C., Bai, W., Qin, C., Oktay, O., Duan, J., ... & Rueckert, D. (2018). Bayesian deep learning for accelerated MR image reconstruction. In Machine Learning for Medical Image Reconstruction: First International Workshop, MLMIR 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings 1 (pp. 64-71). Springer International Publishing.
[15]. Avci, M. Y., Li, Z., Fan, Q., Huang, S., Bilgic, B., & Tian, Q. (2021). Quantifying the uncertainty of neural networks using Monte Carlo dropout for deep learning based quantitative MRI. arXiv preprint arXiv:2112.01587.
[16]. Gal, Y., & Ghahramani, Z. (2016, June). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning (pp. 1050-1059). PMLR.
[17]. Dong, C., Loy, C. C., & Tang, X. (2016). Accelerating the super-resolution convolutional neural network. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14 (pp. 391-407). Springer International Publishing.
[18]. Chen, Y., Xie, Y., Zhou, Z., Shi, F., Christodoulou, A. G., & Li, D. (2018). Brain MRI super-resolution using 3D deep densely connected neural networks. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) (pp. 739-742). IEEE.
[19]. Dong, C., Loy, C. C., He, K., & Tang, X. (2014). Learning a deep convolutional network for image super-resolution. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part IV 13 (pp. 184-199). Springer International Publishing.
[20]. Kim, J., Lee, J. K., & Lee, K. M. (2016). Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1646-1654).
[21]. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., ... & Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1874-1883).
[22]. Li, Z., Fan, Q., Bilgic, B., Wang, G., Wu, W., Polimeni, J. R., ... & Tian, Q. (2023). Diffusion MRI data analysis assisted by deep learning synthesized anatomical images (DeepAnat). Medical Image Analysis, 86, 102744.
[23]. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 (pp. 234-241). Springer International Publishing.
[24]. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1), 1929-1958.
[25]. Stehling, M. K., Turner, R., & Mansfield, P. (1991). Echo-planar imaging: magnetic resonance imaging in a fraction of a second. Science, 254(5028), 43-50.
[26]. Schilling, K. G., Blaber, J., Hansen, C., Cai, L., Rogers, B., Anderson, A. W., ... & Landman, B. A. (2020). Distortion correction of diffusion weighted MRI without reverse phase-encoding scans or field-maps. PLoS One, 15(7), e0236418.
Cite this article
Xu,Z. (2024). Uncertainty-Aware High-Fidelity Anatomical MRI Synthesis using Deep Convolutional Network with Monte Carlo Dropout. Applied and Computational Engineering,100,62-71.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Volume title: Proceedings of the 5th International Conference on Signal Processing and Machine Learning
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).