Spam/Ham email classification using BERT

Research Article
Open access

Spam/Ham email classification using BERT

Siwei Zhang 1*
  • 1 The Chinese University of Hong Kong    
  • *corresponding author siweizhang@link.cuhk.edu.cn
Published on 14 June 2023 | https://doi.org/10.54254/2755-2721/6/20230571
ACE Vol.6
ISSN (Print): 2755-273X
ISSN (Online): 2755-2721
ISBN (Print): 978-1-915371-59-1
ISBN (Online): 978-1-915371-60-7

Abstract

Email is a popular method for communicating with each other. However, as sending email is free of charge as long as an email server and a domain name are available, spam mail is becoming a critical problem in the email network. Conventionally, the industry uses spam filters based on rules and Bayesian inference to counteract spam mail, reaching an accuracy of 98.76%, which is far from satisfactory. Hence, to better protect email users from unsolicited messages containing advertisements, sensitive content, phishing content, and viruses, a new approach is proposed, in which email content is filtered by a spam detector using bidirectional encoder representations from transformers (BERT). BERT is a new language representation model published by Google that has achieved great success because of its powerful capabilities in understanding natural language. After the model is trained on a corpus from Kaggle, the spam detector equipped with the BERT model reaches a binary accuracy of 99.40% when classifying spam mail.

Keywords:

machine learning, NLP, BERT, spam detector.

Zhang,S. (2023). Spam/Ham email classification using BERT. Applied and Computational Engineering,6,1189-1195.
Export citation

References

[1]. Guo, Y., Mustafaoglu, Z., and Koundal, D 2022 Journal of Computational and Cognitive Engineering. Wang, F., Ko, R., and Mickens, J 2019 In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19) 615-630.

[2]. Gordon C, and Thomas L 2007 Online Supervised Spam Filter Evaluation. ACM Transactions on Information Systems 25 3 11 (Preprint https://doi.org/10.1145/1247715.1247717)

[3]. John G 2004 How to beat an adaptive spam filter MIT Spam Conference Cambridge

[4]. Cindy C, Annalee N 2011 Noncommercial Email Lists: Collateral Damage in the Fight against Spam Electronic Frontier Foundation: White Paper Electronic Frontier Foundation

[5]. Jacob D, Ming-Wei C, Kenton L, and Kristina T 2019 Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Minneapolis Association for Computational Linguistics 1 4171–4186 (Preprint https://doi.org/10.18653/v1/n19-1423)

[6]. Ashish V, Noam S, Niki P, Jakob U, Llion J, Aidan G, Lukasz K, and Illia P 2017 Attention is All You Need Advances in Neural Information Processing Systems 30 (Preprint https://doi.org/10.48550/arXiv.1706.03762)

[7]. The University of California, Berkeley Data 100 Fall 19, Project 2 2019 Kaggle

[8]. Turc I, Ming-Wei C, Kenton L, and Kristina T 2019 Well-Read Students Learn Better: On the Importance of Pre-Training Compact Models Preprint https://doi.org/10.48550/arXiv.1908.08962

[9]. Alec R, Karthik N, Tim S, and Ilya S 2020 Improving Language Understanding with Unsupervised Learning OpenAI

[10]. Martín Abadi et al. 2015 TensorFlow: Large-scale machine learning on heterogeneous systems Preprint https://doi.org/10.5281/zenodo.4724125

Cite this article

Zhang,S. (2023). Spam/Ham email classification using BERT. Applied and Computational Engineering,6,1189-1195.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

Disclaimer/Publisher's Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

About volume

Volume title: Proceedings of the 3rd International Conference on Signal Processing and Machine Learning

ISBN：978-1-915371-59-1(Print) / 978-1-915371-60-7(Online)
Editor：Omer Burak Istanbullu
Conference website: http://www.confspml.org
Conference date: 25 February 2023
Series: Applied and Computational Engineering
Volume number: Vol.6
ISSN：2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

References

[1]. Guo, Y., Mustafaoglu, Z., and Koundal, D 2022 Journal of Computational and Cognitive Engineering. Wang, F., Ko, R., and Mickens, J 2019 In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19) 615-630.

[2]. Gordon C, and Thomas L 2007 Online Supervised Spam Filter Evaluation. ACM Transactions on Information Systems 25 3 11 (Preprint https://doi.org/10.1145/1247715.1247717)

[3]. John G 2004 How to beat an adaptive spam filter MIT Spam Conference Cambridge

[4]. Cindy C, Annalee N 2011 Noncommercial Email Lists: Collateral Damage in the Fight against Spam Electronic Frontier Foundation: White Paper Electronic Frontier Foundation

[5]. Jacob D, Ming-Wei C, Kenton L, and Kristina T 2019 Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Minneapolis Association for Computational Linguistics 1 4171–4186 (Preprint https://doi.org/10.18653/v1/n19-1423)

[6]. Ashish V, Noam S, Niki P, Jakob U, Llion J, Aidan G, Lukasz K, and Illia P 2017 Attention is All You Need Advances in Neural Information Processing Systems 30 (Preprint https://doi.org/10.48550/arXiv.1706.03762)

[7]. The University of California, Berkeley Data 100 Fall 19, Project 2 2019 Kaggle

[8]. Turc I, Ming-Wei C, Kenton L, and Kristina T 2019 Well-Read Students Learn Better: On the Importance of Pre-Training Compact Models Preprint https://doi.org/10.48550/arXiv.1908.08962

[9]. Alec R, Karthik N, Tim S, and Ilya S 2020 Improving Language Understanding with Unsupervised Learning OpenAI

[10]. Martín Abadi et al. 2015 TensorFlow: Large-scale machine learning on heterogeneous systems Preprint https://doi.org/10.5281/zenodo.4724125

For authors
Guide for authors Policies for authors Aims and scope Subscription Open access policy
For editors
Policies for editors Editorial board
For reviewers
Policies for reviewers Privacy policy

Cookies are used by this site.
Copyright © 2024 EWA Publishing, its licensors, and contributors. All rights reserved, including text and data mining, AI training, and similar technologies.