1. Introduction

In the actual production process of Printed Circuit Boards (PCBs), issues such as non-standard manufacturing processes and an environment that does not meet production standards can lead to surface defects on PCBs, including misalignment, scratches, open circuits, short circuits, and contamination. These defects can impact the manufacturing process, increase production costs, and even affect a company's competitiveness.

To address these defects, optical inspection methods are commonly employed in the industry. Traditional manual optical inspection methods require significant labor and mental effort. In contrast, AOI (Automatic Optical Inspection) methods not only reduce the need for human labor but also improve inspection accuracy. With the integration of deep learning technology, AOI is advancing towards faster and more accurate detection.

However, PCB defect detection based on deep learning relies on data. During normal production, defective products account for a small portion compared to the normal ones, and the frequency of different types of defects varies significantly. This makes it challenging to obtain sufficient defect data. Additionally, most current single deep learning models can only detect one or a few types of defects, lacking comprehensive coverage and generalization. Moreover, image clarity requirements during the detection process are relatively high. Due to limitations in industrial camera capabilities, complete inspection images can only be obtained through stitching, and the quality of stitching directly affects the final detection results.

In summary, based on the review and summarization of recent literature in the field of PCB defect detection using deep learning in the past few years, we have identified the prevailing algorithms and approaches for addressing challenges related to data scarcity, low model generalization, and poor image stitching quality.

2. Difficulty in acquiring PCB defect data

2.1. Description of the problem

Deep learning-based defect detection models require extensive sample training to achieve satisfactory defect detection performance. On the one hand, data labeling is a highly labor and time-intensive task, and in practical production processes, there are relatively fewer defective PCBs compared to defect-free ones, making it challenging to obtain a sufficient amount of labeled data [1]. On the other hand, although there exist multiple datasets with fewer samples at present, they are difficult to directly merge into a larger dataset due to variations in the data collection equipment used, differences in the background images of PCB boards, varying lighting conditions during the collection process, and other factors [2].

2.2. Solution

To address the issue of insufficient defect datasets, there are currently two primary approaches. The first method involves data generation using Generative Adversarial Networks (GANs) or improved GAN networks. This approach aims to generate defect regions and defect-free backgrounds directly, thus increasing the quantity of data in the dataset. The second method is based on pre-trained models, employing transfer learning to utilize existing high-quality feature extraction networks for defect feature extraction. This reduces the reliance on training data and helps overcome the challenges associated with obtaining defect data.

Abu Ebayyeh et al. leveraged an enhanced neural network based on Generative Adversarial Networks (GANs), known as DCGAN (Deep Convolutional Generative Adversarial Network) [3] [4]. The distinctive and improved feature of the DCGAN architecture lies in the replacement of the original Multi-Layer Perceptron (MLP) structure in both the discriminator and generator networks with convolutional layers and transposed convolutional layers. Subsequently, DCGAN was trained using the MNIST dataset. The data generated by DCGAN was mixed with the original MNIST dataset to train the WaferCaps model [4]. As a result, the accuracy on the test set reached an impressive 91.41%, a substantial improvement compared to the accuracy of 78.2% achieved when training with only the mixed MNIST dataset. Wan et al. introduced a semi-supervised learning framework for defect detection called BA-SSL, which incorporates a data augmentation strategy [2]. The framework proposes a dynamic threshold strategy and a batch-adding strategy for unlabeled data. Ultimately, they achieved a detection accuracy of 98.4% on the DeepPCB dataset using only 50 labeled samples. Ghosh et al., performed transfer learning using a pre-trained Inception-V3 model [5]. They extracted the mid-level neural network of Inception-V3 as the front half of the defect detection network and an adaptation network as the back half of the detection network. By training the adaptation network using the output from the mid-level, they reconstructed the complete detection network. This method ultimately achieved a detection accuracy of 91.125%.

3. Low generalizability of PCB component defect detection

3.1. Description of the problem

Due to the limited availability of PCB component defect datasets at present, and the imbalance in data distribution and sample sizes for different defect types among various datasets, certain defect types exhibit a degree of ambiguity. Sankar et al. conducted a study where they ranked the top 5 defect types in terms of their occurrence frequency within a one-month period [6]. The defect type ranked fifth had an occurrence frequency that was less than 1% of the average occurrence frequency of the top-ranked defect type. Meanwhile, most current detection models can only recognize a single type of defect or multiple defects of the same kind [7]. This leads to a situation where some defect types are relatively ambiguous, making it difficult or challenging to obtain corresponding features and shapes for the respective defect categories. Consequently, this poses difficulties in accurately identifying various types of defects and results in lower generalization capability.

3.2. Solution

Ding et al. proposed a method based on multi-scale deep similarity measure. The main architecture of the model is a Siamese model, and a spatial pyramid pooling network is incorporated into the feature maps of the convolutional module to achieve the fusion of multi-scale feature vectors [8]. Finally, during the training process, discriminative feature embeddings and a similarity metric can be obtained by calculating the contrastive loss, thereby enabling image similarity measurement. The test results showed a significant improvement compared to existing networks. A semi-supervised learning method based on GANs, called the Tinynomaly approach, is proposed for defect detection [9]. This approach involves training an end-to-end network using defect-free samples during the training phase. In the final testing phase, it determines the type of unknown defects by calculating the spatial distance in the feature space between the samples generated by the network and real samples. Test results on the MNISTand CIFAR-10 datasets demonstrate that this method exhibits strong generalization capabilities and performs effectively in detecting tiny unknown defects on printed circuit boards (PCBs) [10][11]. Meanwhile, some tiny defects can be challenging for certain deep neural networks to extract accurate features. Zeng et al. proposed a multiscale feature fusion strategy called ABFPN for detecting small PCB defects [12]. This approach leverages different dilation rates to fully extract contextual information, followed by the utilization of skip connections and a balanced module to achieve feature fusion and enhancement. The model's detection performance on three public datasets surpasses that of seven other top-performing models. Yu et al. introduced an ES-Net network for small object detection that is applicable to various resource scenarios [13]. They addressed the issue of small target information loss by designing the "aggregated feature guidance module (AFGM)" and tackled the problem of differences in target size and detection field of view by designing the "dynamic scale-aware head (DSH)." These two aspects, respectively, address the challenges of easy information loss for small targets and the significant differences in target size and detection field of view that can lead to reduced detection performance. In addition, through image enhancement techniques such as rotation, cropping, resizing, brightness adjustment, and color enhancement applied to existing data, diverse images are generated to increase the model's generalization and robustness. This method has relatively low technical complexity and is widely adopted, and therefore, I won't delve into it further here.

4. Slow image stitching process with low quality

4.1. Description of the problem

Due to the precision requirements of detection algorithms for image accuracy, the field of view captured in a single shot by industrial cameras in industrial settings is often much smaller than the area of the PCB (Printed Circuit Board). Traditional automatic optical inspection systems with industrial cameras typically require multiple captures through the movement of a gimbal [14]. However, due to the limitations of gimbal movement accuracy and the impact of stitching algorithms, this approach can lead to issues with the quality of stitched images. Such issues can directly result in a reduction in the defect detection capability of neural networks. Another stitching method involves extracting features from the PCB through software and then performing image stitching by matching these features. This approach demands substantial computational power, leading to longer detection times. Nevertheless, compared to hardware-based detection methods, this method is more cost-effective.

Therefore, there is a need for a fast and moderately priced image stitching method to achieve efficient image stitching for defect detection.

4.2. Solution

Lin Deng proposed an improved ant colony algorithm using variable domain path search to achieve the shortest image-capturing path. Additionally, a clustering method conducted in the coverage domain was introduced to minimize the number of image-capturing windows required for all target electronic components [14]. Through practical testing, the overall image-capturing time for the window path was reduced by over 12%, and the image capture time for PCB images with window counts ranging from 16 to 83 was consistently under 30 seconds, demonstrating excellent overall performance. Huo et al. proposed a method for image stitching using convolutional neural networks [15]. This convolutional network enhances the similarity between images through a multi-level neural network in the reconstruction network. They defined two loss functions to minimize errors during the image stitching process and ultimately achieved the goal of smoothing the image by reducing pixel gradients. In addition to the two relatively traditional stitching methods mentioned above, Zhao et al. proposed a wide-field, high-resolution image stitching method based on a curved compound eye in [16]. They obtained a special arrangement of sub-eyes based on the relationship between the sub-eyes and the field of view, achieving high angular resolution in both the edge and central fields. Feature points were extracted using Scale-Invariant Feature Transform (SIFT) and then matched using the Random Sample Consensus (RANSAC) algorithm for image stitching.

5. Conclusion

This article reviews the challenges encountered by deep learning methods in PCB defect detection. It identifies several key issues, including the limited size of training datasets, imbalanced distribution of defect categories, low model generalization, single detection type, slow image stitching speed, and poor image quality. Based on existing literature, solutions to these problems have been explored.

To address the problem of small training datasets, approaches such as transfer learning, semi-supervised learning, or unsupervised learning have been utilized to reduce the data requirements. Additionally, improvements in Generative Adversarial Networks (GANs) have been employed to generate synthetic defect data. To enhance model generalization, techniques like feature fusion and enhancement have been applied. By calculating spatial vector distances, unknown category defect detection has been achieved. Semi-supervised learning methods have also been used to calculate spatial distances between feature vectors and real samples, thus improving the model's ability to recognize various defects and enhance its generalization. Regarding the issue of slow image stitching, deep learning techniques have been employed to minimize stitching errors through loss function minimization. Dynamic path planning has been introduced to improve image quality and speed. Furthermore, a novel method utilizing compound eyes to enhance image resolution and achieve a large field of view has been introduced.