1. Introduction

In 2D image processing, edge detection is a fundamental tool for object identification. Edges are characterized by the significant changes in image pixel brightness, or special pixels with discontinuities. Edges contain vital information of 2D images, as they define the boundaries between regions in 2D images, which contribute to segmentation and object recognition in 2D image processing and machine learning [1]. Based on direction, there are three types of common edges in digital images: horizontal edges, vertical edges, and diagonal edges. Generally, an edge detection method will detect two or three types of edges separately and treat them with mathematical processes for more accurate detection results. Fig.1 shows the edge detection results of hand-painted geometric patterns. The handwriting tracks of patterns, which are not obvious in the original image, have also been detected.

Figure 1. A example of edge detection in 2D image.

There are many edge detection methods that can be classified into three categories based on the derivatives they apply. The first category is gradient-based methods, also named as search-based methods. These methods calculate first-order derivatives or gradient magnitude for edge detection. The Robert cross operator, the Prewitt operator, and the Sobel operator are commonplace gradient-based methods. The Robert cross operator is first proposed by Lawrence Roberts in 1963. It is one of the simplest operators in edge detection, but its results are usually unreliable due to its small kernels. The Sobel operator is a discrete differentiation operator, named after Irwin Sobel and Gary Feldman at the Stanford Artificial Intelligent Laboratory in 1968 [2]. The Sobel operator places a proper estimation to the first-order derivatives, making it simpler compared with other edge detection operators. Therefore, Sobel operator is popular among artificial intelligent systems with limited computational ability. G. Chaple and R. D. Daruwala integrated Sobel operator-based image edge detection in low-cost field-programmable gate array (FPGA) devices of intelligent transport system, autonomous vehicles, and self-guided armaments etc. [3]. The Prewitt operator was developed by Judith M. S. Prewitt in 1970 [4]. The Prewitt operator shares many similarities with the Sobel operator. The Prewitt operator utilizes a compact filter to convolve the 2D image in horizontal and vertical directions. The Prewitt operator is also applied to edge detection with cost-efficient hardware. Nguyen et al. presented cost-efficient hardware co-simulation with Prewitt operator-based algorithm [5].

The second category is zero-crossing based methods. These methods look for pixels where a 2D image’s Laplacian value passes through zero, during which second-order derivatives are applied. The typical zero-crossing based method is Laplacian operator. Compared with gradient-based methods, the Laplacian operator has better edge localization. Bansal et al. used Laplacian operator and Open-CV for blur image detection [6].

The third category is optimal methods. These methods are optimized by several criteria for applications under noisy conditions. One of the most well-known optimal methods is the Canny operator. The Canny operator is an optimal method proposed by John Canny in 1986 [7]. The Canny operator has been optimized by three vital criteria: SNR criterion, localization precision criterion and single edge response criterion. As a result, the Canny operator has good edge detection performance under noisy conditions, and it has been widely applied in image processing. Compared with common edge detection algorithms, in most cases, the Canny algorithm has better performance [8-9]. Nevertheless, the traditional Canny operators’ edge detection accuracy can hardly meet higher and higher requirement of modern applications. Researchers tried many efforts to improve Canny operator. Deng et al. developed an improved Canny operator that can make selection of variance of Gaussian filtering for higher detection accuracy [10].

There are also some advanced edge detection methods, such as phased congruency-based methods, an edge detection approach in frequency domain. But these methods will lead to larger computational load, compared with the three categories of edge detection methods mentioned above.

The convolutional neural network (CNN) is a kind of biologically inspired artificial neural network for visual image analysis [11]. A convolutional neural network has three parts: an input layer, several hidden layers, and an output layer. CNN hidden layers contain convolutional layers, pooling layers, fully connected layers, and normalization layers. A convolutional layer is a kernel to extract certain features and reduce image dimensionality. A pooling layer is another kernel which can also reduce image dimensionality. Furthermore, a pooling layer can extract dominate features. A fully connected layer plays a role as classification. A normalization layer, which is optional, standardizes the data and accelerates training.

The convolutional neural network is one of the most popular topics in 2D image processing because of its efficiency in the object identification and prediction. During edge detection, the convolutional neural network can identify the pixels that belong to edges and predict edge positions, which are beneficial to detection accuracy.

This work tries to utilize convolutional neural network to enhance different edge detection methods for 2D images. An overview of the edge detection system is provided at first. Then the experiments are implemented and the results are shown, along with some discussion. Finally, some conclusions are synthesized.

2. Edge detection system

An edge detection system using convolutional neural network was developed, as shown in fig.2. Firstly, the designed convolutional neural network implemented image processing on the input original image (a classical image: Pepper.tiff), extracting important features that were useful for edge detection. Secondly, denoising or smoothing operations was applied as pre-processing step to enhance feature extraction, which was optional in the system. Finally, a simple edge detection operator (Canny, Laplacian, Sobel, or Prewitt) was utilized to output 2D image edge information.

Figure 2. An overview of the edge detection system.

2.1. Convolutional neural network

The convolutional neural network in the designed edge detection system had three-layer structure: two convolutional layers and one normalization layer, in which a 3x3 kernel and a 5x5 kernel were utilized. The main function of the convolutional neural network was to extract edge-related features in 2D images.

2.2. Pre-processing

The output of convolutional neural network might contain noise and inaccurate edge pixels [12]. As a result, denoising and smoothing operations should be implemented as a pre-processing step, granting a simpler and more reliable input for edge detection operators in next step.

2.3. Edge detection operator

There are many edge detection operators can be used in edge detection system. A comparison among four edge detection operators (Canny, Laplacian, Sobel, and Prewitt) was performed.

2.3.1. Canny operator. There are three complex steps in the Canny edge detection: Gaussian convolution smoothing, differentiation, and non-maximum suppression, accounting for Canny operator’s time consuming compared to gradient-based methods or zero-crossing based methods. Two thresholds are applied in a Canny operator. The lower and the higher thresholds are T1 and T2, respectively. The pixels with magnitudes above T2 are regarded as edge components, while the pixels, whose magnitudes are below T1, have no contribution to edge. As for pixels with magnitudes between T1 and T2, they are referred to as a part of edge only when they have direct connections with a magtitude-above-T2 pixel [13]. The Canny edge detection provides better edge detection, better localization and direct response [14-15]. For Canny operators in this work, T1 was set to 110 and T2 was set to 220 for all experiments.

2.3.2. Laplacian operator. The Laplacian operator, a zero-crossing based operator, relies on a certain edge classification rather than edge XY direction in 2D image edge detection. There are two kinds of edges: outward and inward edges, resulting in two kinds of Laplacian operators, as shown in Fig.3, positive Laplacian operators and negative Laplacian operators, respectively.

Figure 3. Two kinds of Laplacian Operators.

2.3.3. Sobel operator. The Sobel operator detects horizontal and vertical edges with Sobel Gx operator and Sobel Gy operator, receptively. The operators shown in Fig.4 are the standard Sobel Gx and Gy. According to practical requirement, more or less weight can be applied on Gx and Gy. For example, 2 and -2 can be changed to 5 and -5, respectively.

Figure 4. Two kinds of Sobel Operators.

2.3.4. Prewitt operator. The Prewitt operator is similar to the Sobel operator, but has more limitations due to its fixed coefficients. The Prewitt operator should have a sum equal to zero. Therefore, it is impossible to adjust this method to meet users’ requirements. Usually, Sobel and Prewitt operators are used to detect objects, but they are not recommended owing to their high noise sensitivity, resulting in compromised accuracy [16].

Figure 5. Two kinds of Prewitt Operators.

3. Materials and methods

3.1. Hardware and software

The experiments were implemented with Python language in software PyCham CE on MacOS Big Sur computer system. The samples used in the experiments was the classical image, Lena.tiff.

3.2. Experiment design

The Canny operator, Laplacian operator, Sobel operator, and Prewitt operator were tested in the designed edge detection system for the best performance. Serving as a control group, these four operators were also applied directly on original images for their results without enhancement of the convolutional neural network.

3.3. Evaluation method

Because of the ground truth reference’s absence, it was difficult to calculate accuracy of results directly. Therefore, relative frequencies of different results were considered as an evaluation parameter for different systems’ performances. The evaluation code was written with Python. Quality of edge detection plays a very important role in the focused area selection, image segmentation and object recognition [17].

4. Results and discussion

The detection results of the experiment on Lena.tiff are shown in Fig.6. The images in the first row are the control group’s edges without convolutional neural network (CNN). The images in the second row are edges detected by the designed edge detection system. In the calculation of the images in different groups, the same operators were used with identified parameters.

Figure 6. The results of the experiment on Lena.tiff.

Through the comparison of the edge results using same operators, the Canny operator had best performance either without or with convolutional neural network, as Canny edge detection’s results had the clearest edge of the woman and the least noise. Through line to line comparison, many edge details, such as the woman’s chin and the decoration on her hat, had been lost in Canny edge detection’s results without CNN while edges detected by Canny operators with CNN could preserve these important details. Laplacian edge detection’s results without CNN had similar problems: the information of the woman’s nose was missing while Laplacian edge detection’s results with CNN showed a better shape of the woman facial features. Sobel edge detection’s results without CNN contained too much noise, so the edge of the women was not clear while Sobel edge detection’s results with CNN got rid of this problem. Prewitt edge detection’s results without CNN was too sharp compared with Prewitt edge detection’s results with CNN. To sum up, the CNN could enhance edge detection performance.

The relative frequencies (R) between results of the control group and the designed edge detection system are shown in Tables 1 and 2, respectively. The relative frequencies can represent the relationship between different edges detected by different operators, which can serve as an alternative evaluation of edge detection qualities quantitatively at the absence of ground truth references. Higher relative frequencies means higher edge detection qualities.

Table 1. Relative frequencies between different edge detection operators’ results without CNN.

Table 2. Relative frequencies between different edge detection operators’ results with CNN.

The quantitative evaluation using relative frequencies in Table 1 shows the Canny operators have the best performance, and the Prewitt operators’ performance is little worse. The third best is the Laplacian operators’s performance. The Sobel operators have the worst performance. While in Table 2, the Laplacian operators had the best edge detection performance. Based on the comparison of these two tables, it can be concluded that the CNN increased relative frequencies dramatically of all operators, so the qualities of edges were improved.

5. Conclusion

This work developed an edge detection system using CNN. In this system, a CNN with two convolutional layers and one normalization layer were utilized for edge-related feature extraction. Then a pre-process was applied to denoise and smooth convoluted outputs. Finally pre-processed images were treated by edge detection operators (Canny, Laplacian, Sobel, and Prewitt). Compare with the results using a simple edge detection operator, the system with CNN showed a higher edge detection quality, according to edge detection results and their quantitative evaluation based on relative frequencies. What’s more, the designed edge detection system had easy-to-use structure that could be implemented and integrated in various 2D image processing applications.