Street view imagery: AI-based analysis method and application

Research Article
Open access

Street view imagery: AI-based analysis method and application

Haoyang Song 1*
  • 1 East China Normal University    
  • *corresponding author 10213903427@stu.ecnu.edu.cn
Published on 21 February 2024 | https://doi.org/10.54254/2755-2721/40/20230627
ACE Vol.40
ISSN (Print): 2755-273X
ISSN (Online): 2755-2721
ISBN (Print): 978-1-83558-305-0
ISBN (Online): 978-1-83558-306-7

Abstract

Street view imagery is an emerging form of geographic big data. It presents urban visual environments from the perspective of urban residents and also contains non-visual environment of cities, such as urban human activities and socio-economic development. However, traditional digital image processing has its limitations, and the continuous development of artificial intelligence, especially computer vision and deep learning, provides strong technical support for exploring the rich semantic information in street view imagery. This paper reviews the related research on street view imagery and its artificial intelligence analysis methods and applications. It outlines the acquisition, storage, and common data sources of street view imagery. Then it introduces computer vision, deep learning, and commonly used open-source datasets in street view imagery analysis. It also detailed three aspects of AI-based street view imagery applications, namely quantification of the physical space, urban perception, and spatial semantic speculation. Finally, issues like data acquisition, domain adaption and deep learning black box are discussed. The hotspots and prospects for the development of this research topic are also prospected.

Keywords:

Street View Imagery, Computer Vision, Deep Learning

Song,H. (2024). Street view imagery: AI-based analysis method and application. Applied and Computational Engineering,40,54-62.
Export citation

References

[1]. Li Y C, Peng L, Wu C W and Zhang J Z 2002 Street View Imagery (SVI) in the Built Environment: A Theoretical and Systematic Review. Buildings-Basel 12(8) 1167

[2]. Zhang F and Liu Y 2021 Street view imagery:Methods and applications based on artificial intelligence. Nrsb 25(5) 1043-1054

[3]. Zhou B L, Lapedriza A, Khosla A, Oliva A and Torralba A 2018 Places: A 10 Million Image Database for Scene Recognition. Tpami 40(6) 1452-1464

[4]. Seiferling I, Naik N, Ratti C and Proulx R 2017 Green streets - Quantifying and mapping urban trees with street-level imagery and computer vision. Landscape urban plan 165 93-101

[5]. Stubbing P, Peskett J, Rowe F and Arribas-Bel D 2019 A Hierarchical Urban Forest Index Using Street-Level Imagery and Deep Learning. Remote Sens-Basel 11(12) 1395

[6]. Gong F Y, Zeng Z C, Zhang F, Li X J, Ng E and Norford L K 2018 Mapping sky, tree, and building view factors of street canyons in a high-density urban environment. Build environ 134 155-167

[7]. Li X J and Ratti C 2019 Mapping the spatio-temporal distribution of solar radiation within street canyons of Boston using Google Street View panoramas and building height model. Landscape urban plan 191 103387

[8]. Li X J, Cai B Y, Qiu W S, Zhao J H and Ratti C 2019 A novel method for predicting and mapping the occurrence of sun glare using Google Street View. Transport res c-emer 106 132-144

[9]. Zhang F, Zhou B L, Liu L, Liu Y, Fung H H, Lin H and Ratti C 2018 Measuring human perceptions of a large-scale urban region using machine learning. Landscape urban plan 180 148-160

[10]. Amiruzzaman M, Curtis A, Zhao Y, Jamonnak S and Ye X Y 2021 Classifying crime places by neighborhood visual appearance and police geonarratives: a machine learning approach. Jcss 4(2) 813-837

[11]. Zhang F, Fan Z Y, Kang Y H, Hu Y J and Ratti C 2021 “Perception bias”: Deciphering a mismatch between urban crime and perception of safety. Landscape urban plan 207 104003

[12]. Nguyen Q C, Sajjadi M, McCullough M, Pham M, Nguyen T T, Yu W J, Meng H W, Wen M, Li F F, Smith K R, Brunisholz K and Tasdizen T 2018 Neighbourhood looking glass: 360 degrees automated characterisation of the built environment for neighbourhood effects research. Jech 72(3) 260-266

[13]. Nguyen Q C, Huang Y R, Kumar A, Duan H S, Keralis J M, Dwivedi P, Meng H W, Brunisholz K D, Jay J, Javanmardi M and Tasdizen T 2020 Using 164 Million Google Street View Images to Derive Built Environment Predictors of COVID-19 Cases. Int J Env Res Pub He 17(17) 6359

[14]. Ma R X, Wang W, Zhang F, Shim K and Ratti C 2019 Typeface Reveals Spatial Economical Patterns. Sci Rep-uk 9 15946

[15]. Gebru T, Krause J, Wang Y L, Chen D Y, Deng J, Aiden E L and Li F F 2017 Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States. P natl Acad Sci 114(50) 13108-13113

[16]. Joglekar S, Quercia D, Redi M, Aiello L M, Kauer T and Sastry N 2020 FaceLift: a transparent deep learning framework to beautify urban scenes. Roy Soc OpenSci 7(1) 190987


Cite this article

Song,H. (2024). Street view imagery: AI-based analysis method and application. Applied and Computational Engineering,40,54-62.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

Disclaimer/Publisher's Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

About volume

Volume title: Proceedings of the 2023 International Conference on Machine Learning and Automation

ISBN:978-1-83558-305-0(Print) / 978-1-83558-306-7(Online)
Editor:Mustafa İSTANBULLU
Conference website: https://2023.confmla.org/
Conference date: 18 October 2023
Series: Applied and Computational Engineering
Volume number: Vol.40
ISSN:2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

References

[1]. Li Y C, Peng L, Wu C W and Zhang J Z 2002 Street View Imagery (SVI) in the Built Environment: A Theoretical and Systematic Review. Buildings-Basel 12(8) 1167

[2]. Zhang F and Liu Y 2021 Street view imagery:Methods and applications based on artificial intelligence. Nrsb 25(5) 1043-1054

[3]. Zhou B L, Lapedriza A, Khosla A, Oliva A and Torralba A 2018 Places: A 10 Million Image Database for Scene Recognition. Tpami 40(6) 1452-1464

[4]. Seiferling I, Naik N, Ratti C and Proulx R 2017 Green streets - Quantifying and mapping urban trees with street-level imagery and computer vision. Landscape urban plan 165 93-101

[5]. Stubbing P, Peskett J, Rowe F and Arribas-Bel D 2019 A Hierarchical Urban Forest Index Using Street-Level Imagery and Deep Learning. Remote Sens-Basel 11(12) 1395

[6]. Gong F Y, Zeng Z C, Zhang F, Li X J, Ng E and Norford L K 2018 Mapping sky, tree, and building view factors of street canyons in a high-density urban environment. Build environ 134 155-167

[7]. Li X J and Ratti C 2019 Mapping the spatio-temporal distribution of solar radiation within street canyons of Boston using Google Street View panoramas and building height model. Landscape urban plan 191 103387

[8]. Li X J, Cai B Y, Qiu W S, Zhao J H and Ratti C 2019 A novel method for predicting and mapping the occurrence of sun glare using Google Street View. Transport res c-emer 106 132-144

[9]. Zhang F, Zhou B L, Liu L, Liu Y, Fung H H, Lin H and Ratti C 2018 Measuring human perceptions of a large-scale urban region using machine learning. Landscape urban plan 180 148-160

[10]. Amiruzzaman M, Curtis A, Zhao Y, Jamonnak S and Ye X Y 2021 Classifying crime places by neighborhood visual appearance and police geonarratives: a machine learning approach. Jcss 4(2) 813-837

[11]. Zhang F, Fan Z Y, Kang Y H, Hu Y J and Ratti C 2021 “Perception bias”: Deciphering a mismatch between urban crime and perception of safety. Landscape urban plan 207 104003

[12]. Nguyen Q C, Sajjadi M, McCullough M, Pham M, Nguyen T T, Yu W J, Meng H W, Wen M, Li F F, Smith K R, Brunisholz K and Tasdizen T 2018 Neighbourhood looking glass: 360 degrees automated characterisation of the built environment for neighbourhood effects research. Jech 72(3) 260-266

[13]. Nguyen Q C, Huang Y R, Kumar A, Duan H S, Keralis J M, Dwivedi P, Meng H W, Brunisholz K D, Jay J, Javanmardi M and Tasdizen T 2020 Using 164 Million Google Street View Images to Derive Built Environment Predictors of COVID-19 Cases. Int J Env Res Pub He 17(17) 6359

[14]. Ma R X, Wang W, Zhang F, Shim K and Ratti C 2019 Typeface Reveals Spatial Economical Patterns. Sci Rep-uk 9 15946

[15]. Gebru T, Krause J, Wang Y L, Chen D Y, Deng J, Aiden E L and Li F F 2017 Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States. P natl Acad Sci 114(50) 13108-13113

[16]. Joglekar S, Quercia D, Redi M, Aiello L M, Kauer T and Sastry N 2020 FaceLift: a transparent deep learning framework to beautify urban scenes. Roy Soc OpenSci 7(1) 190987