1. Introduction

1.1. Research background

As a highly concentrated place of human activities, urban space creates convenience and opportunities for social development but also faces more and more challenges of public security issues. With the increasing level of urbanization, community security has become one of the important indicators to measure the sustainability and livability of a city, and also an important factor for people to consider when traveling or moving. The existence of criminal activities not only affects the quality of life of residents and leads to various social problems, but also increases the cost of the government and related departments [1]. Understanding the distribution patterns and causes of urban crime and improving the security of the design area through planning methods have attracted more and more attention from scholars and urban planners. But the occurrence of urban crime is a complicated process that is frequently influenced by a variety of factors, including neighborhood socioeconomic position, the physical environment, government, and human psychology[2]. In these factors, the neighborhood socioeconomic status has been regarded as one of the important causes of criminal activities, claim by Becker first proposed in 1968, Becker thinks that criminal behavior is based on profit/cost analysis after rational decisions, the individual level of criminal expected return is under the influence of the social economic conditions, such as unemployment, income level, Education level, etc [3]. When acquiring the necessary social capital cannot be done so legally, some people will opt to do it in an illegal manner, which results in the prevalence of criminal behaviors.

Neighborhood socioeconomic status (NSES) is the actual or perceived relative position of a group in a specific social environment at the overall or ecological level, which reflects the social class and status of people in a certain area. It is generally represented by the combination of factors such as education level, population composition, employment status, and economic income [4]. Many studies have confirmed the close relationship between neighborhood socioeconomic factors and criminal activity. On the one hand, it has a long-term impact on the development of individuals living in disadvantaged neighborhoods. Changing neighborhoods can reduce individual hostility and the likelihood of engaging in criminal activity. People who live in underprivileged communities during youth are more likely to commit crimes later in life [5]. On the other hand, crime and violence rates are higher in underprivileged areas than in wealthy areas [6] .

However, research on the relationship between NSES and crime rates is still limited, mainly reflected in two aspects. First of all, existing research perspectives are mostly one-dimensional, such as unemployment and crime rates, immigration and violent murder and property crime, and lack of discussion on comprehensive NSES. Second, there is a lack of accurate analysis of spatially fine-grained NSES. One reason is that precise, high-quality data are still not available in many countries. Although census data are widely available, the long intervals between available data and the large size of the census area do not allow for fine and accurate analysis. With the development of big data and computer algorithms, some studies try to use open data combined with models to explain and predict socioeconomic conditions. However, previous studies based on remote sensing data and social media data are mainly from a single source and can only represent one aspect of socioeconomic attributes [7].

1.2. The likelihood of housing as a predictor of NSES

Housing has been widely recognized as an important physical manifestation of NSES. First of all, real estate occupies a considerable proportion of household wealth and social assets, and it has both use and investment value. It is an important indicator reflecting the social and economic status of residents [8]. Secondly, adjacent housing units, due to geographical proximity and regional regulations, often have similar housing prices, physical characteristics, and quality of life, and are simultaneously exposed to local social relationships, ecological risks, sanitation, and crime [9]. Finally, the similar geographical characteristics of housing also condense the living of groups with similar socioeconomic status, which further forms new housing attributes. Therefore, housing conveys rich socioeconomic information about the community and is a potential tool for analyzing NSES. Housing specialist advertising websites such as Craigslist, Beike, Zillow, and others have become the mainstream way to post, share and find properties to buy and rent. The popularity of these online housing ads presents an opportunity for NSES speculation because they contain a large amount of comprehensive information that is not available in traditional census data sets, and because they are fast to update, have wide coverage, and are publicly available. In this regard. Wang et al. successfully used housing advertising information to predict NSES in urban areas of Hangzhou, China, and achieved good results, but the feasibility of this approach in other regions has not been proven [8].

1.3. Objects

Existing studies have achieved limited results on the relationship between fine-grained NSES and crime rates, and the lack of fine-grained comprehensive NSES evaluation methods is a potential reason. From this perspective, this study uses data collected from Zillow and other public datasets to verify the feasibility of using advertising data combined with machine learning models to predict NSES in Brooklyn, USA, and analyzes the relationship between precinct fine-grained NSES and crime rates.

2. Background of study area

Brooklyn, the largest borough of New York city in the U.S. state of New York (Figure 1) [10]. Brooklyn has transformed into a booming hub of entrepreneurship, high-tech start-up enterprises, and postmodern art and design during the 2010s. However, it also experiences significant socioeconomic inequality (e.g., income and education inequality, deprivation) and a high crime rate, for example, one in five Brooklyn neighborhoods has a median income of over $100,000 per year, while one in five receives food stamp benefits [11]. The crime rate committed in Brooklyn in 2022 was the second highest in New York City, behind Manhattan [12]. Brooklyn's sociodemographic tendencies are generally becoming more complicated and diverse. The SES-related disparities in Brooklyn are similar to those in the majority of U.S. megacities that are urbanizing. As a result, it provides a useful example for measuring neighborhood SES and mapping it at the county level. All Brooklyn neighborhoods are the subject of our analysis; a more detailed neighborhood SES investigation might produce more results. The information gleaned from this case ought to serve as a helpful resource for legislators and urban planners both domestically and internationally.

3. Materials and methods

3.1. Data and preprocessing

The methodology process consists of five key steps. (1) Data collection and cleaning. (2) Using Naive Bayes classifier for semantic recognition of topics. (3) Quantifying the emotion of each topic using a lexical-based approach. (4) Find the best performing machine learning algorithm to draw NSES map (5) Analyze the inferred relationship between fine-grained SES and crime rate.

3.1.1. Housing advertisements data collection and cleaning. First, the study wrote a web crawler in Python and collected information about housing ads in the Brooklyn area on Zillow from January to September 2022. Zillow was selected for this research for two reasons. On the one hand, Zillow provides comprehensive and reliable housing information, including detailed descriptions and property characteristics. On the other hand, as an open data platform, Zillow has easy access to data, and the methods verified in this study are also easy to replicate and use in other regions. Three types of abnormal samples were removed to further clean up the data: (1) missing important data (such as physical features and text advertisements); (2) Duplicate samples; (3) The information is illogical (e.g. much lower or higher than average). The final dataset includes house prices for 5958 samples, 5 physical variables (total area, number of toilets, number of rooms, Walkscore, Bikescore), and at least 1 paragraph of text description.

3.1.2. Semantic analysis. The text description information extracted from the advertisement can be transformed into a series of features to infer SES. First, text descriptions need to be preprocessed in R to split long paragraphs into independent sentences without punctuation. text classification is used to process text, which is a level of natural language processing (NLP), and its purpose is to assign topics, categories, and so on to the specified text. The Naïve Bayes Classifier is then used to determine the topic involved in each description sentence. Naïve Bayes Classifier is an efficient supervised recognition technology that can identify the topic to which a test sample belongs. This study used 1000 identified topic items as test data for further manual labeling. For example, "walking distance to bus stop" is labeled "transportation convenience"; "The whole house is in move-in condition" is defined as "finished level". The remaining descriptive sentences were used as test data, and the Naïve Bayes model inferred the topic from the training data. Described in the end, all the houses were identified as eight themes: traffic convenience (time of arrival in public transportation or distance), decoration (indoor decoration), natural light (lighting conditions), the security level (security) of the residence, the number of private gardens, the number of the yard, garage and basement levels (the availability of basement and decoration).

3.1.3. Sentimental analysis. Word-based sentiment analysis was applied to quantify publisher descriptions into scores for each of the eight topics already defined. All sentences were scored according to the emotion contained in the words, using sentiment dictionaries and custom vocabulary. Two dictionaries are used, the English Positive Opinion Dictionary and the English Negative Opinion Dictionary (http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html). To ensure accuracy, words and idioms used in advertisements to express degree have also been added to the dictionary, such as "half-finished" and "well maintained". The words in the AD descriptions were rated for negative, positive, and neutral emotions. In addition, the sentiment scores of the words in each sentence are added as a score for the topic of the sentence.

3.1.4. Amenity accessibility data supplement. The information collected from housing advertisements will be used as input data for SES prediction. In the successful case of Hangzhou, Amenity accessibility information accounts for a high proportion in the prediction and is a necessary input data. Related studies also prove this view. Different socioeconomic status communities have significant inequalities in accessibility to some facilities, such as supermarkets, fast food restaurants, sports facilities, etc., and high socioeconomic status communities are more likely to stay away from unpopular facilities to some extent [13]. Therefore, the input of Amenity accessibility data helps improve the prediction's accuracy. However, insufficient POI accessibility information was extracted from the ads collected in Brooklyn, so this study supplemented relevant data from the Open Street Map(OSM). According to relevant literature, the longitude and latitude information of banking, education, entertainment, food, green facilities, medical treatment, shops, sports, and transportation facilities were collected from OSM and then summarized in Arcgis to count the number of various facilities within 500 meters of walking distance of each residential unit.

3.2. Actual SES indicators

The real SES assessment indicators are generally selected from the aspects of population, occupation, economic income, and education level, and adjusted according to the different focus of the research. In this study, eight neighborhood socioeconomic status variables (Table 1) were selected based on the study of Wang et al. [8] and the urban characteristics of New York, and the original variable data were obtained from the U.S. Census Bureau [10]. To make the SES prediction method more convenient and reduce the error and uncertainty caused by large and mixed data, after data standardization, the entropy weight method was used to further integrate all variables into a comprehensive SES index. Finally, the reliability of the integrated SES index is verified by comparing the results of the entropy weight method, coefficient of variation method, and principal component analysis.

Table 1.The information about selected SES indicators.

3.3. Prediction based on the machine learning algorithms

3.3.1. Rationale of the inference model. The principle of inferring community SES is to use the extracted housing features to establish regression models to make predictions. Machine learning models are used to solve pattern recognition and classification problems with no representation between inputs and outputs, as well as data mining and prediction problems. Its ability to handle nonlinear rational data has been proved in a variety of disciplines, and it is capable of processing high-dimensional data and extracting features at a faster training pace [14]. However, different machine learning algorithms operate on distinct principles, resulting in varying accuracies and estimates. To establish reliable input data, it is required to compare the performance of various machine learning algorithms. This study chose seven open-source machine learning models, and the research reasoning is as follows: (1) Enter all of the determined input data; (2) Separate reliable features by comparing the relative importance of input characteristics predicted by different machine learning techniques; and (3) Determine whether only reliable features can be used to forecast neighborhood SES.

3.3.2. Normalize features, Based on the previous steps, we obtain a series of input features (indicators) for the model, including housing price, number of bathrooms, number of bedrooms, number of private gardens, number of yards, decoration level, traffic convenience, medical facility accessibility, entertainment facility accessibility, shopping facility accessibility, food amenity accessibility, green infrastructure accessibility, bank accessibility, education facility accessibility, basement, and the living area of the house. Since the measurement of each housing feature (inference indicator) are not uniform, it is necessary to normalize them before using them to calculate comprehensive indicators (i.e., to convert the absolute values of indicators into relative values) using equation (1) and (2), to solve the problem of homogenization of the values of different qualitative indicators.

Assume that there are m indicators: X1, X2,…, Xn. Then for positive indicators:

\( x_{ij}^{ \prime }=\frac{x_{ij}^{ \prime }-min⁡({X_{1j}}, {X_{2j}},…,{X_{nj}})}{max{({X_{1j}},{X_{2j}},…,{X_{nj}})}-min⁡({X_{1j}},{X_{2j}},…,{X_{nj}})} \) (1)

For negative indicators:

\( x_{ij}^{ \prime }=\frac{max{({X_{1j}}, {X_{2j}},…,{X_{nj}})}-{X_{ij}}}{max{({X_{1j}},{X_{2j}},…,{X_{nj}})}-min⁡({X_{1j}},{X_{2j}},…,{X_{nj}})} \) (2)

3.3.3. Model operation and validation. In this paper, seven open source models are selected for training based on four basic types of machine learning algorithms(Table 3).To test the machine learning algorithm's accuracy, 80% of the dataset was utilized for training and 20% for validation. The process was performed 10 times. The specific performance indicators include the accuracy (Eq. (3)), the percentage of root mean squared error (%RMSE; Eq. (4)) and the percentage of mean absolute error (%MAE; Eq. (5)). The next stage is to use the chosen method to rank the relative relevance of each input feature, and then to filter out trustworthy signs for recalculation. Finally, we map the neighborhood SES in Brooklyn using the best machine learning approach.

\( Accuracy=\frac{\sum (1-|\frac{{y_{i,0}}-{y_{i,p}}}{{y_{i,0}}}|)}{n} \) (3)

\( \%RMSE=\frac{\sqrt[]{\frac{1}{n}\sum {({y_{i,0}}-{y_{i,p}})^{2}}}}{ {y_{0}}} \) (4)

\( \%MAE= \frac{1}{n}\sum \frac{|{y_{i,0}}-{y_{i,p}}|}{{y_{i,0}}} \) (5)

Where n is the total number of samples; \( {y_{i,0}} \) and \( {y_{i,p}} \) are actual and inferred SES; \( {y_{0}} \) is the mean actual SES.

3.4. Crime data

The study collected data on all crimes in Brooklyn since 2022 from the New York City Police Department's official website [15]. These data are CSV files available for public download, which contain a lot of information such as the location, time, and type of crime. This study first cleaned the data from two aspects: (1) delete abnormal and duplicated data (for example, the coordinates are outside the constituency); (2) screen out the crime types that are easy to occur in the neighborhood, a total of 11 categories (Table 2). After that, the total number of classified crimes within 500 meters of each housing sample was counted, and the crime rate was calculated per 10,000 people.

Table 2. Crime types.

4. Results and discussion

4.1. Input features extracted

The semantic analysis separated nine features from the Zillow dataset's housing descriptions: traffic convenience, decoration level, lighting condition, number of bathrooms and bedrooms; Living areas; Housing price; The number of the garden, yards, and basement. These topics, together with 11 other indicators, constitute the input data. Table 3 summarizes the statistical data of each indicator. It can be shown that food accessibility, green facility accessibility, Walk score, Bike score, living area, pricing, and other topics have high positive emotion levels and standard deviations in terms of maximum and average values. Besides, according to the normal distribution of the data, the skewness and kurtosis distribution of each topic are not uniform. These results imply that there are clear differences in these characteristics between communities.

Table 3. Statistical descriptions of each input feature.

4.2. The results of machine learning algorithms

The machine learning algorithms performance are summarized in Table 3. GBDT ranks first in terms of predictive accuracy and lowest error estimates. Nine features, including Shop accessibility, Price, Bike score, Walk score, green infrastructure, education accessibility, living area, sport accessibility, and food accessibility present relatively higher importance estimated by all the machine learning algorithms and are therefore labeled as “reliable features”(Figure 2). After retraining with only these features with high weight ratio, the accuracy of all algorithms is enhanced (Table 4).

Table 4. Model performances of the algorithms.

4.3. SES pattern within Brooklyn

Figure 3 depicts the GBDT-generated NSES distribution pattern (the best-performing machine learning algorithm). It can be seen that there is significant NSES inequality in Brooklyn. The Moran index analysis was carried out to test the accuracy of the results. The global Moran I index score was 0.93, the z was 46.51, and the p-value was 0, which confirmed the autocorrelation of SES distribution in Brooklyn. Local Moran I maps further reinforce this consistency, and, in particular, high and low groupings should be of concern as they indicate the presence of some areas of deprivation.

4.4. The association between criminal activity and neighborhood SES

The crime rate of each housing unit in Brooklyn is between 0.0005 and 0.159, which is quite different. To preliminarily explore whether the crime rate has a potential spatial distribution pattern, the natural break point classification method was used to classify the crime rate in Brooklyn from January to June 2022 into five grades: high, medium-high, medium-low and low (Figure 4). The natural break point classification method detects the classification interval based on the natural grouping inherent in the data, with the goal of minimizing intra-class difference and maximizing inter-class difference.

Figure 4 shows that the distribution of crime in Brooklyn has an obvious aggregation effect, and it highly coincides with the boundary of NSES distribution, with higher crime rates in the northeast and west. Figure 5 shows a crime hot spot map in Brooklyn categorized by crime severity. It can be seen that the distribution of crime in Brooklyn is concentrated in the northeast, and Violation type crime is the most common.

Figure 5. Crime rate heat map of Brooklyn by crime type.

The above analysis shows that the crime rate has autocorrelation in space, so the geographic weighted regression model in ArcGIS is used to fit various crime rates and NSES data. In this paper, the Gaussian kernel function and AICc bandwidth calculation method is selected to establish the GWR model. Figure 6 is the spatial distribution diagram of the standard residuals of the total crime rate. Figure 6 shows that the residuals of most statistical units are between -0.5 and 0.5, the standard residuals in the whole study area are minor, and those with absolute values greater than 2.5 are concentrated in some northern regions.

Table 5 displays the GWR model's diagnostic signs. R2 and R2 Adjusted measure how well the model fits the data; their values range from 0 to 1. The greater the value, the better the model fits the data. Table 5 shows that the model can explain 75.2% of the total crime rate variation, and the overall model has a good fitting effect. The R2 adjusted values of most variables were above 0.7. NSES had the best interpretation of Harassment and Burglary (0.878157 and 0.843320, respectively), and the worst interpretation of Sex crime (0.569421).

Table 5. Diagnostic index of the GWR model.

5. Discussion

5.1. Using online advertising data to predict the viability of NSES in Brooklyn

By modifying the model of Wang et al [8], this paper verifies the applicability of the machine learning method for predicting NSES based on public data such as online housing ads in Brooklyn, USA. The results show that fine-grained neighborhood socioeconomic status can be predicted fairly accurately by adding amenity accessibility information obtained from Open Street Map and Walkscore data combined with online housing advertising. By comparing different machine learning algorithms, we identified nine features that reliably infer NSES. These characteristics are Shop accessibility, Price, Bike score, Walk score, green infrastructure, education accessibility, living area, sport accessibility, and food accessibility. In addition, we observed that although Zillow's housing advertising information emphasized the number of basements, gardens, and yards of a home, they were not recognized as valid predictors or even given low weight. One possible explanation is that Brooklyn houses are similar in these characteristics, resulting in insignificant differences between neighborhoods. This study is of interest to countries that lack publicly available data or fine-grained data sets, meaning that they can monitor the distribution of neighborhood SES in their own countries at any time based on publicly available, dynamic data.

5.2. Differences in crime rates by socioeconomic status

Through geographic weighted model analysis, this paper finds that the use of neighborhood socioeconomic status can effectively explain the distribution of some crime types, that is, criminal activities usually occur in areas with poor NSES. Among them, Harassment and Burglary were the best explanations, with Sex crime the worst. Figure 7 shows the distribution of different types of crimes in different NSES areas. It can be found that the most common types of crimes in Brooklyn were assault, burglary, petit larceny, and harassment. With the increase of NSES, the rates of all kinds of crimes decreased. burglary was higher in the most desirable socioeconomic area, probably because of the greater benefit of burglary in this area. This study verifies the impact of NSES on the occurrence of crime and helps to improve local crime through planning.

5.3. Strengths, limitations and prospects

This article has helped advance the use of public data for social segregation research, establishing a fine-grained neighborhood socioeconomic status estimation method for Brooklyn, USA, that relies entirely on publicly accessible data and open codes, opening new opportunities for other cities around the world. At the same time, this paper also verified the correlation between neighborhood socioeconomic status and criminal activities. The incidence of criminal activities is more likely to occur in disadvantaged communities, which helps relevant departments to improve urban safety through planning. However, there are still some limitations. First, this method has not been tested in a large number of cities. Second, advertising data may overlook some residents, such as the poor.

6. Conclusion

Using data collected from Zillow and other public data sets, this paper verifies the feasibility of using advertising data combined with machine learning model to predict NSES in Brooklyn, USA, and achieves an accuracy of 0.82. Then, the geospatial weighted regression model is used to analyze the relationship between fine-grained NSES and crime rate. It is found that there is a significant correlation between NSES and crime rate, that is, the crime rate decreases with the rise of socioeconomic status. This paper also found that NSES has the best explanation for harassment and burglar.