Exploring the Advancements and Challenges of Automated Machine Learning

Research Article
Open access

Exploring the Advancements and Challenges of Automated Machine Learning

Zhengyang Jin 1*
  • 1 University of Electronic Science and Technology of China    
  • *corresponding author 1625159433@qq.com
Published on 1 August 2023 | https://doi.org/10.54254/2755-2721/8/20230095
ACE Vol.8
ISSN (Print): 2755-273X
ISSN (Online): 2755-2721
ISBN (Print): 978-1-915371-63-8
ISBN (Online): 978-1-915371-64-5

Abstract

Automl,a rapidly growing field which is aiming to apply the machine to solve problems that human can’t easily deal with . This includes tasks such as feature selection, model selection, and hyperparameter tuning. One of the many advantages about Auto Ml is that it can greatly shorten the cost of researchs and resources cost by applying machine learning to a problem. This makes it accessible to a wider range of users, including those without a background in computer science or statistics.In spite of some advantages of AutoML, many challenges are waiting to be addressed. The main challenge is that it is often challenging to ensure that the models generated by AutoML are of high quality and generalize well to new data. Another challenge is that AutoML can be computationally expensive, which can make it infeasible for some problems. Overall, AutoML has the potential to revolutionize the way we apply machine learning to real-world problems, but it is important to be aware of its limitations and challenges.

Keywords:

feature selection, feature construction, model selection, hyperparameter tuning, ensemble learning, greedy algorithm, random search, grid search, organic search, model fusion

Jin,Z. (2023). Exploring the Advancements and Challenges of Automated Machine Learning. Applied and Computational Engineering,8,847-852.
Export citation

References

[1]. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot et al., “Mastering the game of Go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, p.484, 2016.

[2]. M. Feurer and F. Hutter, “Hyperparameter optimization,” 2018.[Online]. Available: https://www.ml4aad.org/wp-content/uploads/2018/09/chapter1-hpo.pdf.

[3]. G. Katz, E. C. R. Shin, and D. Song, “Explorekit: Automatic feature generation and selection,” in International Conference on Data Mining, 2016, pp. 979–984.

[4]. J. M. Kanter and K. Veeramachaneni, “Deep feature synthesis: Towards automating data science endeavors,” in IEEE International Conference on Data Science and Advanced Analytics, 2015, pp.1–10.

[5]. G. Katz, E. C. R. Shin, and D. Song, “Explorekit: Automatic feature generation and selection,” in International Conference on Data Mining, 2016, pp. 979–984.

[6]. J. M. Kanter and K. Veeramachaneni, “Deep feature synthesis: Towards automating data science endeavors,” in IEEE International Conference on Data Science and Advanced Analytics, 2015, pp.1–10.

[7]. B. Tran, B. Xue, and M. Zhang, “Genetic programming for feature construction and selection in classification on high-dimensional data,” Memetic Computing, vol. 8, no. 1, pp. 3–15, 2016

[8]. G. Katz, E. C. R. Shin, and D. Song, “Explorekit: Automatic feature generation and selection,” in International Conference on Data Mining, 2016, pp. 979–984.

[9]. K. Swersky, J. Snoek, and R. P. Adams, “Freeze-thaw bayesian optimization,” arXiv preprint arXiv:1406.3896, 2014.

[10]. T. Nickson, M. A. Osborne, S. Reece, and S. J. Roberts, “Automated machine learning on big data using stochastic algorithm tuning,” arXiv preprint arXiv:1407.7969, 2014.

[11]. C. Thornton, F. Hutter, H. Hoos, and K. Leyton-Brown, “AutoWEKA: Combined selection and hyperparameter optimization of classification algorithms,” in ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2013, pp.847–855.

[12]. M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter, “Efficient and robust automated machine learning,” in Advances in Neural Information Processing Systems,2015, pp. 2962–2970.

[13]. L. Kotthoff, C. Thornton, H. Hoos, F. Hutter, and K. LeytonBrown, “Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA,” Journal of Machine Learning Research, vol. 18, no. 1, pp. 826–830, 2017.

[14]. J. A. Tropp, “Greed is good: Algorithmic results for sparse approximation,” IEEE Transactions on Information theory, vol. 50,no. 10, pp. 2231–2242, 2004.

[15]. S. Huang, X. Li, Z. Cheng, Z. Zhang, and A. G. Hauptmann,“GNAS: A greedy neural architecture search method for multiattribute learning,” in ACM Multimedia, 2018, pp. 2049–2057.

[16]. K. Eggensperger, F. Hutter, H. H. Hoos, and K. Leyton-Brown,“Efficient benchmarking of hyperparameter optimizers via surrogates,” in AAAI Conference on Artificial Intelligence, 2015.

[17]. J. Bergstra and Y. Bengio, “Random search for hyper-parameter optimization,” Journal of Machine Learning Research, vol. 13, no. Feb, pp. 281–305, 2012.

[18]. Guo Y, Huang J, Dong Y et al. Guoym at SemEval-2020 task 8: Ensemble-based classification of visuo-lingual metaphor in memes. In Proceedings of the Fourteenth Workshop on Semantic Evaluation. Barcelona (online): International Committee for Computational Linguistics, pp. 1120–1125. URL https://aclanthology.org/2020.semeval-1.148.


Cite this article

Jin,Z. (2023). Exploring the Advancements and Challenges of Automated Machine Learning. Applied and Computational Engineering,8,847-852.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

Disclaimer/Publisher's Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

About volume

Volume title: Proceedings of the 2023 International Conference on Software Engineering and Machine Learning

ISBN:978-1-915371-63-8(Print) / 978-1-915371-64-5(Online)
Editor:Anil Fernando, Marwan Omar
Conference website: http://www.confseml.org
Conference date: 19 April 2023
Series: Applied and Computational Engineering
Volume number: Vol.8
ISSN:2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

References

[1]. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot et al., “Mastering the game of Go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, p.484, 2016.

[2]. M. Feurer and F. Hutter, “Hyperparameter optimization,” 2018.[Online]. Available: https://www.ml4aad.org/wp-content/uploads/2018/09/chapter1-hpo.pdf.

[3]. G. Katz, E. C. R. Shin, and D. Song, “Explorekit: Automatic feature generation and selection,” in International Conference on Data Mining, 2016, pp. 979–984.

[4]. J. M. Kanter and K. Veeramachaneni, “Deep feature synthesis: Towards automating data science endeavors,” in IEEE International Conference on Data Science and Advanced Analytics, 2015, pp.1–10.

[5]. G. Katz, E. C. R. Shin, and D. Song, “Explorekit: Automatic feature generation and selection,” in International Conference on Data Mining, 2016, pp. 979–984.

[6]. J. M. Kanter and K. Veeramachaneni, “Deep feature synthesis: Towards automating data science endeavors,” in IEEE International Conference on Data Science and Advanced Analytics, 2015, pp.1–10.

[7]. B. Tran, B. Xue, and M. Zhang, “Genetic programming for feature construction and selection in classification on high-dimensional data,” Memetic Computing, vol. 8, no. 1, pp. 3–15, 2016

[8]. G. Katz, E. C. R. Shin, and D. Song, “Explorekit: Automatic feature generation and selection,” in International Conference on Data Mining, 2016, pp. 979–984.

[9]. K. Swersky, J. Snoek, and R. P. Adams, “Freeze-thaw bayesian optimization,” arXiv preprint arXiv:1406.3896, 2014.

[10]. T. Nickson, M. A. Osborne, S. Reece, and S. J. Roberts, “Automated machine learning on big data using stochastic algorithm tuning,” arXiv preprint arXiv:1407.7969, 2014.

[11]. C. Thornton, F. Hutter, H. Hoos, and K. Leyton-Brown, “AutoWEKA: Combined selection and hyperparameter optimization of classification algorithms,” in ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2013, pp.847–855.

[12]. M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter, “Efficient and robust automated machine learning,” in Advances in Neural Information Processing Systems,2015, pp. 2962–2970.

[13]. L. Kotthoff, C. Thornton, H. Hoos, F. Hutter, and K. LeytonBrown, “Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA,” Journal of Machine Learning Research, vol. 18, no. 1, pp. 826–830, 2017.

[14]. J. A. Tropp, “Greed is good: Algorithmic results for sparse approximation,” IEEE Transactions on Information theory, vol. 50,no. 10, pp. 2231–2242, 2004.

[15]. S. Huang, X. Li, Z. Cheng, Z. Zhang, and A. G. Hauptmann,“GNAS: A greedy neural architecture search method for multiattribute learning,” in ACM Multimedia, 2018, pp. 2049–2057.

[16]. K. Eggensperger, F. Hutter, H. H. Hoos, and K. Leyton-Brown,“Efficient benchmarking of hyperparameter optimizers via surrogates,” in AAAI Conference on Artificial Intelligence, 2015.

[17]. J. Bergstra and Y. Bengio, “Random search for hyper-parameter optimization,” Journal of Machine Learning Research, vol. 13, no. Feb, pp. 281–305, 2012.

[18]. Guo Y, Huang J, Dong Y et al. Guoym at SemEval-2020 task 8: Ensemble-based classification of visuo-lingual metaphor in memes. In Proceedings of the Fourteenth Workshop on Semantic Evaluation. Barcelona (online): International Committee for Computational Linguistics, pp. 1120–1125. URL https://aclanthology.org/2020.semeval-1.148.