DEVELOPMENT TENDENCIES OF GENERATION MODELS OF THE UKRAINIAN LANGUAGE
DOI:
https://doi.org/10.33042/2522-1809-2024-3-184-10-15Keywords:
neural network, Ukrainian language, machine learning, speech synthesis, text-to-speechAbstract
The article explores the development of language generation technologies, from machine learning models to fluid neural networks for text generation. English is one of the most widespread languages in the world: it is the official language of more than 60 countries. The events of recent years have led to the development of the popularity of the Ukrainian language not only in the country but also abroad. The article analyses scientific sources on this topic, the results of which formed a base for creating a genealogical tree of the development of language synthesis technology. The study pays particular attention to the problem of automating the generation of texts in the Ukrainian language, which includes the absence of strict grammatical constructions of sentences, numerous rules of word formation, presentation of words, placement of accents, and exceptions to almost all rules. The article presents several different transformative architectures that one can use to generate the Ukrainian language. We considered the problems of training such networks and ways to solve them for machine learning models and neural networks. As a result, we determined the stages of development of a neural network for the generation of texts in the Ukrainian language. Even though neural networks are a powerful tool of artificial intelligence, they have certain limitations that the new architecture – liquid neural networks (LNN) – is devoid of. The article also discusses their advantages and disadvantages. The LNN architecture differs from traditional neural networks in its ability to efficiently process continuous or time series data. If new data is available, LNNs can change the number of neurons and connections in a layer. Due to these two main features of LNN, the network expresses the system’s state over time as opposed to the traditional neural network (NN) approach, so it does not require large amounts of observed training data to obtain accurate results. In conclusion, we should note that, despite their shortcomings, LNN networks are more dynamic, adaptive, efficient, and reliable than traditional neural networks and have many potential uses.
References
DOU Моделі text-to-speech для синтезу української мови: як працюють, хто їх створює та де знайти [Electronic resource] – Regime of access: https://dou.ua/forums/topic/40194/, free (date of the application: 10.02.2024).
D. Suendermann, H. H¨oge, and A. Black, “Challenges in speech synthesis” Speech Technology, Boston: Springer – pp. 19–32, – 2010. DOI: 10.1007/978-0-387-73819-2_2
A. Iida, and N. Campbell, “Speech database design for a concatenative text-to-speech synthesis system for individuals with communication disorders,” International Journal of Speech Technology – vol. 6, – no. 4, – pp. 379–392 – 2023; DOI:10.1023/A:1025761017833.
A. Van Den Oord et al. “WaveNet: A generative model for raw audio,” SSW – 2016 – vol. 125 DOI:10.48550/arXiv.1609.03499
State Of The Art of Speech Synthesis at the End of May 2021 [Electronic resource] – Regime of access https://towardsdatascience.com/state-of-the-art-of-speech-synthesis-at-the-end-of-may-2021-6ace4fd512f2, free (date of the application: 13.02.2024)
M.M. Shatkovskyi Modeli kompiuternoho podannia ukrainskoho movlennia dlia problemy konkatenatyvnoho sehmentyvnoho syntezu Inzheneriia prohramnoho zabezpechennia № 3 (7) – 2011. C. 58-67.
ResearchGate H.i Murfi, T. Gowandi, G. Ardaneswari BERT-Based Combination of Convolutional and Recurrent Neural Network for Indonesian Sentiment Analysis 2022, DOI:10.48550/arXiv.2211.05273.
XLNet Embeddings (Large Cased) [Electronic resource] – Regime of access https://sparknlp.org/2021/07/07/xlnet_large_cased_en.html, free (date of the application: 15.02.2024)
T5-large [Electronic resource] – Regime of access https://huggingface.co/google-t5/t5-large, free (date of the application: 17.02.2024)
P. Banerjee, T. Gokhale, C. Baral Self-Supervised Test-Time Learning for Reading Comprehension – 2021, DOI:10.18653/v1/2021.naacl-main.95
Hakan Tekgul, ML Solutions Engineer Published March 16, 2023 Monitoring Text-Based Generative AI Models Using Metrics Like Bleu Score [Electronic resource] – Regime of access https://arize.com/blog-course/generative-ai-metrics-bleu-score/, free (date of the application: 18.02.2024)
O. Okozelsk Reservoir Computing for .NET (RCNet) [Electronic resource] – Regime of access: https://github.com/okozelsk/NET?tab=readme-ov-file#reservoir-computing-for-net-rcnet, free (date of the appli-cation: 18.02.2024).
R. Hasani, M.s Lechner, A. Amini, D. Rus, R. Grosu Liquid Time-constant Networks – 2020 – arXiv:2006.04439v4
Downloads
Published
How to Cite
Issue
Section
License
The authors who publish in this collection agree with the following terms:
• The authors reserve the right to authorship of their work and give the magazine the right to first publish this work under the terms of license CC BY-NC-ND 4.0 (with the Designation of Authorship - Non-Commercial - Without Derivatives 4.0 International), which allows others to freely distribute the published work with a mandatory reference to the authors of the original work and the first publication of the work in this magazine.
• Authors have the right to make independent extra-exclusive work agreements in the form in which they were published by this magazine (for example, posting work in an electronic repository of an institution or publishing as part of a monograph), provided that the link to the first publication of the work in this journal is maintained. .
• Journal policy allows and encourages the publication of manuscripts on the Internet (for example, in institutions' repositories or on personal websites), both before the publication of this manuscript and during its editorial work, as it contributes to the emergence of productive scientific discussion and positively affects the efficiency and dynamics of the citation of the published work (see The Effect of Open Access).
