1. Introduction

This study mainly explores the current state of unsupervised feature extraction methods, identifying their advantages and disadvantages in handling high-dimensional data. Specifically, this research aims to address the challenges of improving the interpretability, robustness to outliers, and computational efficiency of these methods. By studying information theory methods, sparse learning techniques, and strategies combining deep learning models (such as autoencoders and generative adversarial networks), it seeks to enhance the effectiveness of unsupervised feature extraction. This paper employs a comprehensive literature review and comparative analysis to evaluate the performance of these methods in various fields, such as image processing, gene expression analysis, text mining, and network security. This research is important because it closes gaps in the literature and offers ideas for new research. It makes recommendations for possible difficulties and projects future developments.

2. Current state of unsupervised feature extraction methods

2.1. Traditional methods

Unsupervised feature extraction dimensionality reduction is a method that extracts important features by reducing data dimensions without labeled data. Its main goal is to map high-dimensional data to a low-dimensional space while preserving the key structure and information of the data as much as possible. This method is particularly important when dealing with high-dimensional data, as high-dimensional data often contains a large amount of redundant and noisy features, increasing computational complexity and potentially affecting model performance. As shown in Table 1, this paper compares traditional unsupervised feature extraction methods in terms of interpretability, sensitivity to outliers, and computational complexity and analyzes their respective most suitable scenarios.

Table 1. Comparison of Traditional Unsupervised Feature Extraction Methods.

2.2. Problems with unsupervised feature selection

It can be seen that these methods may ignore the correlations between features when processing high-dimensional data, and the feature subsets obtained by these methods are often not optimal. This leads to problems such as poor interpretability of results, sensitivity to outliers, difficulty in handling multi-classification problems, and high computational resource requirements. In different application domains, improving existing algorithms and adopting unsupervised feature selection methods to obtain better classification results is a direction worth researching. In recent years, unsupervised feature selection algorithms have made significant progress in multiple aspects, especially in combining deep learning and practical applications[9].

3. Improvement strategies

3.1. Algorithm improvements

In recent years, researchers have proposed various improvement strategies, such as information theory-based methods, spectral similarity methods, biologically inspired methods, sparse learning methods, regularization methods, and adaptive neighborhood embedding methods.

(1)Information theory-based methods: These methods use concepts from information theory, such as entropy and mutual information, to evaluate the importance of features [10]. For example, SUD (Sequential Backward Selection Method for Unsupervised Data) uses entropy values based on distance similarity as indicators for correlation ranking and feature selection.

(2)Spectral similarity methods: These methods select features by analyzing the spectral characteristics of data, such as SPEC (SPECtrum decomposition) and USFSM (Unsupervised Spectral Feature Selection Method for mixed data), which use Laplacian operators to evaluate the importance of features and select them based on their variance and local preservation ability [11].

(3)Biologically inspired methods[12]: These methods draw inspiration from optimization strategies in the biological world, such as ant colony optimization algorithms. UFSACO (Unsupervised Feature Selection based on Ant Colony Optimization) uses genetic algorithms to prioritize features with high pheromone values and low similarity until a pre-specified stopping criterion is reached.

(4)Sparse learning methods[13]: These methods select features through sparse representation, for example, mR-SP (minimum-Redundancy SPectral feature selection), which combines SPEC ranking with minimum redundancy-maximum relevance criterion for feature selection.

(5)Regularization methods[14]: These methods introduce regularization terms to control model complexity and thus select important features, for example, RMR (Regularized Mutual Representation), which utilizes correlations between features to establish an unsupervised feature selection mathematical model constrained by the Frobenius norm, and designs a divide-and-conquer ridge regression optimization algorithm for rapid model optimization.

(6)Adaptive neighborhood embedding methods[15]: These methods determine the number of neighbors for each sample based on the distribution characteristics of the dataset itself, thereby constructing a sample similarity matrix. For example, ANEFS (Adaptive Neighborhood Embedding Based Unsupervised Feature Selection) determines the number of neighbors for each sample based on the distribution characteristics of the dataset itself, introduces an intermediate matrix mapping from high-dimensional space to low-dimensional space, and uses the Lagrange multiplier method to optimize the objective function for solving.

3.2. Combining with deep learning

Deep learning has made significant progress in its application to unsupervised feature selection. Models such as deep autoencoders and generative adversarial networks (GANs) can automatically extract high-level abstract features, further enhancing the effectiveness of unsupervised feature selection.

3.2.1. Application of deep autoencoders in unsupervised feature selection. Matrix Capsules with EM Routing: This is a deep autoencoder model based on matrix capsules that can automatically extract key features in an unsupervised environment while preserving the global and local structure of the data [16]. In the MNIST handwritten digit recognition task, this method achieved 99.2% accuracy, significantly outperforming traditional convolutional neural networks [17]. It performs particularly well in handling rotated and deformed digit images, demonstrating its advantage in preserving spatial relationships.

Deep Sparse Autoencoder with Feature Aggregation: This is a deep sparse autoencoder that combines feature aggregation, achieving unsupervised feature selection through sparsity constraints and feature aggregation strategies [18]. In processing high-dimensional gene expression data, this method achieved 85% feature dimensionality reduction on the TCGA cancer dataset while maintaining 91% classification accuracy, greatly improving computational efficiency and reducing storage requirements [19].

Enhanced Deep Sparse Autoencoders with Adaptive Clustering is an enhanced deep sparse autoencoder that improves unsupervised feature selection through adaptive clustering and sparsity constraints [20]. In a large-scale social media text analysis study, researchers applied the EDSAAC method to analyze millions of tweets about COVID-19 on Twitter [21]. The method first represented each tweet as a 5000-dimensional bag-of-words vector, then reduced it to 100 dimensions using EDSAAC. The adaptive clustering mechanism automatically identified 23 topic clusters, including "vaccine information," "social distancing measures," and "economic impact." Compared to traditional LDA topic models, EDSAAC not only reduced processing time from 48 hours to 6 hours but also improved topic coherence scores by 15%. EDSAAC performed exceptionally well in handling emerging topics and rare words, capturing emerging concepts like "vaccine hesitancy" often overlooked in traditional methods. Furthermore, by maintaining sparsity, the method effectively filtered noise information, improving the interpretability of topics and enabling public health agencies to understand and respond to public sentiment more quickly and accurately.

Deep Autoencoder-Based Unsupervised Feature Selection with Locality Preservation is an unsupervised feature selection method based on deep autoencoders that effectively selects important features in high-dimensional data by preserving the local structure of the data [22]. This method has demonstrated strong performance in image classification and gene data analysis. In a cross-species gene function prediction study, the DAUFSLP method was used to analyze gene expression data from humans, mice, and zebrafish [23].

3.2.2. Application of Generative Adversarial Networks (GANs) in unsupervised feature selection. Adversarial Learning-Based Unsupervised Feature Selection is a method based on adversarial learning for unsupervised feature selection. It uses GANs to automatically generate high-quality features, overcoming the limitations of traditional methods, such as redundant features and noise interference [24]. This approach is particularly suitable for image classification and text analysis tasks. ALUFS has demonstrated outstanding performance in large-scale image classification tasks. In a study involving one million natural scene images [25].

Self-Ensembling GAN for Unsupervised Feature Selection is a method that uses Self-Ensembling GAN for unsupervised feature selection in high-dimensional data [26]. This method improves feature selection accuracy and model generalization by using adaptive weights and multi-scale feature extraction mechanisms. SE-GAN has made groundbreaking progress in medical imaging analysis. In a pneumonia diagnosis study involving 10,000 chest X-rays [27], SE-GAN selected 100 of the most diagnostic features from the original 3,000 radiological features through its adaptive weighting mechanism. This not only increased diagnostic accuracy from 89% to 94% but also reduced diagnosis time by 60%. The multi-scale feature extraction capability of SE-GAN enables it to capture both local details and global structures, which is particularly important for identifying early lung lesions.

GAN-Based Unsupervised Feature Selection with Diversity Enhancement is a GAN-based unsupervised feature selection method that combines diversity enhancement strategies to improve feature selection accuracy and model robustness [28]. This method is particularly effective for addressing the issue of redundant features in image data. GUFDE has shown strong capabilities in high-resolution satellite image analysis. In a study monitoring global forest cover changes [29].

4. Conclusion

In conclusion, unsupervised feature extraction methods have demonstrated significant progress in handling high-dimensional and unlabeled data, particularly in fields such as image processing, gene expression data analysis, text mining, and network security. While traditional methods like Principal Component Analysis (PCA) and Independent Component Analysis (ICA) are effective, they often suffer from poor interpretability and sensitivity to outliers when dealing with complex data. In recent years, algorithms combining deep learning, such as deep autoencoders and generative adversarial networks (GANs), have significantly improved the effectiveness of feature selection. These methods overcome the limitations of traditional approaches by automatically extracting high-level abstract features.

In future research, several key directions in the field of unsupervised feature selection can enhance model performance. First and foremost, multimodal data processing is essential. By merging image, text, and audio data, for example, multimodal techniques can combine features from many data sources and optimize model performance in complicated contexts. Second, real-time processing capability is key; as data scales expand, there is a need to develop efficient algorithms and optimization strategies, such as designing parallel algorithms and utilizing distributed resources to accelerate processing and reduce overhead. Third, integration with other technologies, such as combining with reinforcement learning to optimize the selection process, and with transfer learning to improve performance in target domains, can make models more adaptive and flexible. In summary, with the advancement of technology and the expansion of application scenarios, unsupervised feature extraction methods are expected to achieve even more significant results in the future.