Machine translation of classical Chinese based on unigram segmentation transformer framework

Research Article
Open access

Machine translation of classical Chinese based on unigram segmentation transformer framework

Zhuonan Ju 1 , Yixuan Xin 2* , Mingda Ye 3
  • 1 Nanjing University of Information Science & Technology    
  • 2 Wuhan University of Technology    
  • 3 Beijing Sport University    
  • *corresponding author xinyx@whut.edu.cn
Published on 7 February 2024 | https://doi.org/10.54254/2755-2721/37/20230465
ACE Vol.37
ISSN (Print): 2755-273X
ISSN (Online): 2755-2721
ISBN (Print): 978-1-83558-299-2
ISBN (Online): 978-1-83558-300-5

Abstract

In the translation work of Chinese ancient books, traditional manual translation is difficult and inefficient. As an important field of natural language processing, machine translation is expected to solve this problem. Due to the rapid development of NLP technology, prior works mainly follow the pipeline of Transformer when dealing with the machine translation task, which can extract the high-quality feature representation with its self-attention mechanism. The great success of Transformer has inspired the direction of our ancient text translation work. In this paper, we screen the Unigram word division by exploring and comparing, and propose a solution for the translation of ancient literary texts. Specifically, we adopt the evaluation of BLEU value and achieve the BLEU values of 43.4 and 40.03 for short and long sentences respectively. When compared with the results of Baidu Translation, our BLEU values increase by 8.12 and 5.18. Additionally, our translation results are more in line with the original text than Baidu Translation, demonstrating the potential and advantage of the model in bridging the ancient and modern Chinese era rift.

Keywords:

Machine Translation, Classical Chinese Translation, Transformer, Unigram, BLEU

Ju,Z.;Xin,Y.;Ye,M. (2024). Machine translation of classical Chinese based on unigram segmentation transformer framework. Applied and Computational Engineering,37,23-30.
Export citation

References

[1]. Li X and Hao X 2021 English Machine Translation Model Based on Artificial Intelligence Journal of Physics: Conference Series 1982

[2]. Zhou L 2016 Machine Translation Based on Translation Rules for Processing Natural Language Proceedings of 2016 6th International Conference on Machinery,Materials,Environment,Biotechnology and Computer(MMEBC 2016) 488-91

[3]. Vogel S, Och F J, Tillmann C, Nießen S, Sawaf H and Ney H 2000 Statistical Methods for Machine Translation Verbmobil: Foundations of Speech-to-Speech Translation 377-93

[4]. Bengio Y, Ducharme R, Vincent P and Janvin C 2003 A Neural Probabilistic Language Model J. Mach. Learn. Res. 3 1137-55

[5]. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser L and Polosukhin I 2017 Attention is All you Need Advances in Neural Information Processing Systems 30

[6]. Liu Z 2022 Ancient-Modern Chinese Machine Translation Models Based On Transformer East China Normal University 11 103

[7]. Zhou C and Liu Z 2022 Ancient Text Machine Translation Method Based on Semantic Information Sharing Transformer Technology Intelligence Engineering 8 114-27

[8]. Chung J, Gulcehre C, Cho K H and Bengio Y 2014 Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling ArXiv https://doi.org/10.48550/arXiv.1412.3555

[9]. Huang A, Subramanian S, Sum J, Almubarak K and Biderman S 2022 The Annotated Transformer http://nlp.seas.harvard.edu/annotated-transformer/

[10]. Zhou D, He W and Gang C 2011 Research on Tibetan Text Classification Based on N-Gram Model 2011 13th IEEE Joint International Computer Science and Information Technology Conference(JICSIT 2011) 02

[11]. Kim N S, Baldwin T and Kan M-Y 2010 Evaluating N-gram Based Evaluation Metrics for Automatic Keyphrase Extraction The 23rd International Conference on Computational Linguistics Proceedings of the Main Conference 1

[12]. Cui D, Liu X, Chen R, Liu X, Li Z and Qi L 2020 Named Entity Recognition in Field of Ancient Chinese Based on Lattice LSTM Computer Science 47 18-22.

[13]. Zeng X 2019 Technology Implementation of Chinese Jieba Segmentation Based on Python [J]. China Computer & Communication 31 38-39+42.

[14]. Papineni K, Roukos S, Ward T and Zhu W J 2002 BLEU: A Method for Automatic Evaluation of Machine Translation Association for Computational Linguistics 311-8

Cite this article

Ju,Z.;Xin,Y.;Ye,M. (2024). Machine translation of classical Chinese based on unigram segmentation transformer framework. Applied and Computational Engineering,37,23-30.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

Disclaimer/Publisher's Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

About volume

Volume title: Proceedings of the 2023 International Conference on Machine Learning and Automation

ISBN：978-1-83558-299-2(Print) / 978-1-83558-300-5(Online)
Editor：Mustafa İSTANBULLU
Conference website: https://2023.confmla.org/
Conference date: 18 October 2023
Series: Applied and Computational Engineering
Volume number: Vol.37
ISSN：2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

References

[1]. Li X and Hao X 2021 English Machine Translation Model Based on Artificial Intelligence Journal of Physics: Conference Series 1982

[2]. Zhou L 2016 Machine Translation Based on Translation Rules for Processing Natural Language Proceedings of 2016 6th International Conference on Machinery,Materials,Environment,Biotechnology and Computer(MMEBC 2016) 488-91

[3]. Vogel S, Och F J, Tillmann C, Nießen S, Sawaf H and Ney H 2000 Statistical Methods for Machine Translation Verbmobil: Foundations of Speech-to-Speech Translation 377-93

[4]. Bengio Y, Ducharme R, Vincent P and Janvin C 2003 A Neural Probabilistic Language Model J. Mach. Learn. Res. 3 1137-55

[5]. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser L and Polosukhin I 2017 Attention is All you Need Advances in Neural Information Processing Systems 30

[6]. Liu Z 2022 Ancient-Modern Chinese Machine Translation Models Based On Transformer East China Normal University 11 103

[7]. Zhou C and Liu Z 2022 Ancient Text Machine Translation Method Based on Semantic Information Sharing Transformer Technology Intelligence Engineering 8 114-27

[8]. Chung J, Gulcehre C, Cho K H and Bengio Y 2014 Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling ArXiv https://doi.org/10.48550/arXiv.1412.3555

[9]. Huang A, Subramanian S, Sum J, Almubarak K and Biderman S 2022 The Annotated Transformer http://nlp.seas.harvard.edu/annotated-transformer/

[10]. Zhou D, He W and Gang C 2011 Research on Tibetan Text Classification Based on N-Gram Model 2011 13th IEEE Joint International Computer Science and Information Technology Conference(JICSIT 2011) 02

[11]. Kim N S, Baldwin T and Kan M-Y 2010 Evaluating N-gram Based Evaluation Metrics for Automatic Keyphrase Extraction The 23rd International Conference on Computational Linguistics Proceedings of the Main Conference 1

[12]. Cui D, Liu X, Chen R, Liu X, Li Z and Qi L 2020 Named Entity Recognition in Field of Ancient Chinese Based on Lattice LSTM Computer Science 47 18-22.

[13]. Zeng X 2019 Technology Implementation of Chinese Jieba Segmentation Based on Python [J]. China Computer & Communication 31 38-39+42.

[14]. Papineni K, Roukos S, Ward T and Zhu W J 2002 BLEU: A Method for Automatic Evaluation of Machine Translation Association for Computational Linguistics 311-8

For authors
Guide for authors Policies for authors Aims and scope Subscription Open access policy
For editors
Policies for editors Editorial board
For reviewers
Policies for reviewers Privacy policy

Cookies are used by this site.
Copyright © 2024 EWA Publishing, its licensors, and contributors. All rights reserved, including text and data mining, AI training, and similar technologies.