Neural Machine Translation of Electrical Engineering with Fusion of Memory Information
Source abstract
Abstract
This paper proposes a new neural machine translation model of electrical engineering that combines a transformer with gated recurrent unit (GRU) networks. By fusing global information and memory information, the model effectively improves the performance of low-resource neural machine translation. Unlike traditional transformers, our proposed model includes two different encoders: one is the global information encoder, which focuses on contextual information, and the other is the memory encoder, which is responsible for capturing recurrent memory information. The model with these two types of attention can encode both global and memory information and learn richer semantic knowledge. Because transformers require global attention calculation for each word position, the time and space complexity are both squared with the length of the source language sequence. When the length of the source language sequence becomes too long, the performance of the transformer will sharply decline. Therefore, we propose a memory information encoder based on the GRU to improve this drawback. The model proposed in this paper has a maximum improvement of 2.04 BLEU points over the baseline model in the field of electrical engineering with low resources.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques · Topic Modeling · Text Readability and Simplification
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Foreign Languages, Henan University of Science and Technology, Luoyang 471023, China
Henan Province New Energy Vehicle Power Electronics and Power Transmission Engineering Research Center, Luoyang 471023, China
School of Information Engineering, Henan University of Science and Technology, Luoyang 471023, China
Funding
J.Z.
References (27)
Show source-provided references
1. The neural machine translation models for the low-resource Kazakh–English language pair10.7717/peerj-cs.1224 ↗
2. Maučec, M.S., and Donaj, G. (2019). Machine translation and the evaluation of its quality. Recent Trends Comput. Intell., 143.
3. Kalchbrenner, N., and Blunsom, P. (2013). Recurrent convolutional neural networks for discourse compositionality. arXiv.10.3115/v1/P14-1062 ↗
4. Bahdanau, D., Cho, K., and Bengio, Y. (2015, January 7–9). Neural machine translation by jointly learning to align and translate. Proceedings of the ICLR, San Diego, CA, USA.
5. Gehring, J., Auli, M., Grangier, D., Yarats, D., and Dauphin, Y.N. (2017, January 6–11). Convolutional sequence to sequence learning. Proceedings of the ICML, Sydney, Australia.
6. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. arXiv.
7. Araabi, A., and Monz, C. (2020). Optimizing transformer for low-resource neural machine translation. arXiv.10.18653/v1/2020.coling-main.304 ↗
8. Tonja, A.L., Kolesnikova, O., Gelbukh, A., and Sidorov, G. (2023). Low-Resource Neural Machine Translation Improvement Using Source-Side Monolingual Data. Appl. Sci., 13.10.3390/app13021201 ↗
9. Mahsuli, M.M., Khadivi, S., and Homayounpour, M.M. (2023). LenM: Improving Low-Resource Neural Machine Translation Using Target Length Modeling. Neural. Proc. Lett., 1–32.10.1007/s11063-023-11208-1 ↗
10. A Data Augmentation Method for English-Vietnamese Neural Machine Translation10.1109/ACCESS.2023.3252898 ↗
11. English–Assamese neural machine translation using prior alignment and pre-trained language model10.1016/j.csl.2023.101524 ↗
12. Park, Y.H., Choi, Y.S., Yun, S., Kim, S.H., and Lee, K.J. (2022). Robust Data Augmentation for Neural Machine Translation through EVALNET. Mathematics, 11.10.3390/math11010123 ↗
13. Dhar, P., Bisazza, A., and van Noord, G. (2022, January 13). Evaluating Pre-training Objectives for Low-Resource Translation into Morphologically Rich Languages. Proceedings of the Thirteenth Language Resources and Evaluation Conference, Marseille, France.
14. Enhancing low-resource neural machine translation with syntax-graph guided self-attention10.1016/j.knosys.2022.108615 ↗
15. Improving neural machine translation with POS-tag features for low-resource language pairs10.1016/j.heliyon.2022.e10375 ↗
16. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv.
17. Gulcehre, C., Firat, O., Xu, K., Cho, K., Barrault, L., Lin, H.C., and Bengio, Y. (2015). On using monolingual corpora in neural machine translation. arXiv.
18. Wang, Y., Xia, Y., Tian, F., Gao, F., Qin, T., Zhai, C.X., and Liu, T.Y. (2019). Neural machine translation with soft prototype. Adv. Neural Informat. Process. Syst., 32.
19. Cao, Q., and Xiong, D. (November, January 31). Encoding gated translation memory into neural machine translation. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1340 ↗
20. Bimal, K. (2001). Modern Power Electronics and AC Drives, Prentice-Hall.
21. Wang, C., Zhao, J., Yu, Q., and Cheng, H. (2005). Modern Power Electronics and AC Drive, Machinery Industry Press.
22. Wang, Q., and Glover, J.D. (2009). Power System Analysis and Design (Adapted in English), Machinery Industry Press.
23. Wang, Q., Huang, W., Yan, Y., and Ma, Y. (2015). Power System Analysis and Design (Chinese Edition), Machinery Industry Press.
24. Klein, G., Kim, Y., Deng, Y., Senellart, J., and Rush, A.M. (2017). Opennmt: Open-Source Toolkit for Neural Machine Translation. arXiv.10.18653/v1/P17-4012 ↗
25. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002, January 6–12). Bleu: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics (2002), Philadelphia, PA, USA.10.3115/1073083.1073135 ↗
26. Chen, H., Chen, Y., and Zhang, J. (2023). Neural Machine Translation of Electrical Engineering Based on Vector Fusion. Appl. Sci., 13.10.3390/app13042325 ↗
27. Neural Machine Translation by Fusing Key Information of Text