MINIMISATION OF NETWORK TRAFFIC IN THE RAFT-LIKE CONSENSUS ALGORITHM
DOI:
https://doi.org/10.33042/2522-1809-2024-4-185-2-6Keywords:
distributed databases, RAFT, network traffic, cardinalities, Big Data, IoTAbstract
In distributed databases, network traffic is a critical factor that affects system performance and efficiency. The article develops a new method for minimising network traffic in the RAFT-like Consensus Algorithm. The result of using this method is a reduction in network traffic and query execution time in a distributed database. The authors demonstrate its practical application with the example of an online student gradebook system developed with Laravel using a MySQL relational database.
The developed network traffic optimisation method relies on the preliminary exchange of key vectors and cardinalities between nodes. Such an approach reduces the amount of data transferred by avoiding duplication and transmitting only the necessary data. Applying this method increases system efficiency and lowers network load, which is particularly important for distributed databases with high traffic volumes.
The data materialisation process after query execution allows for storing query results on the nodes that initiate these queries. It ensures quick access to already obtained data when performing similar queries in the future, reducing their execution time and improving system performance. Materialisation also helps to reduce the number of repeated data processing, decreasing the system load and enhancing the overall efficiency of the distributed database.
One of the main advantages of this method is its simplicity of implementation and ability to significantly reduce network traffic, particularly in systems containing a small number of infrequent changes. Compared to existing methods, such as the Semi-Join Query Optimisation method, this method shows advantages in systems with small and infrequent changes.
A significant feature of the new method is its ability to provide high data consistency in a distributed system. The use of key vector exchange allows for more efficient data synchronisation between nodes, lowering the likelihood of conflicts and ensuring the relevance of data across the entire system. It is essential for systems requiring high reliability and data accuracy.
Due to its simplicity of implementation and high efficiency, this method is a promising solution for improving the performance of distributed systems in various fields.
References
Ongaro D., Ousterhout J. In Search of an Understandable Consensus Algorithm (Extended Version). Raft Consensus Algorithm. 2014. Available at: https://raft.github.io/raft.pdf
C. Mohan, D. Haderle, B. Lindsay, H. Pirahesh, P. Schwarz, “ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging,” ACM Transactions on Database Systems (TODS), vol. 17, no. 1, pp. 94-162, 1992. Available at+: https://dl.acm.org/doi/10.1145/1365815.1365816
F. Li, V. Venkataraman, S. Rajamanickam, and J. Zhang, “Spanner: Google’s globally distributed database,” ACM Transactions on Computer Systems (TOCS), vol. 31, no. 2, pp. 8-15, 2013.
D. B. Lomet, K. Tzoumas, C. Weng, P. Larson, S. Blanas, C. Curino, and A. J. Elmore, “Cosmos DB: A System Over-view,” IEEE Data Engineering Bulletin, vol. 42, no. 1, pp. 36-45, 2019.
B. Warnke, S. Fischer, S. Groppe, “Using Machine Learning and Routing Protocols for Optimizing Distributed SPARQL Queries in Collaboration,” Computers, vol. 12, no. 10, 2023. Available at: https://www.mdpi.com/2073-431X/12/10/210https://www.mdpi.com/2076-3417/14/2/846
IDC. Global Data Volume Trends [EB/OL]. 2018. Available online: https://www.seagate.com/files/www-content/our-StoryAmazon/trends/files/idc-seagate-dataage-chine-whitepaper.pdf (accessed on 16 October 2018).
Zeng, K.; Yang, J.; Wang, H.; Shao, B.; Wang, Z. A distributed graph engine for web scale RDF data. Proc. VLDB Endow. 2013, 6, 265–276.
Rohloff, K.; Schantz, R.E. High-performance, massively scalable distributed systems using the MapReduce software framework: The SHARD triple-store. In Programming Support Innovations for Emerging Distributed Applications; As-sociation for Computing Machinery: New York, NY, USA, 2010; pp. 1–5.
Zeng, K.; Yang, J.; Wang, H.; Shao, B.; Wang, Z. A distributed graph engine for web scale RDF data. Proc. VLDB Endow. 2013, 6, 265–276.
Rohloff, K.; Schantz, R.E. High-performance, massively scalable distributed systems using the MapReduce software framework: The SHARD triple-store. In Programming Support Innovations for Emerging Distributed Applications; Association for Computing Machinery: New York, NY, USA, 2010; pp. 1–5.
Downloads
Published
How to Cite
Issue
Section
License
The authors who publish in this collection agree with the following terms:
• The authors reserve the right to authorship of their work and give the magazine the right to first publish this work under the terms of license CC BY-NC-ND 4.0 (with the Designation of Authorship - Non-Commercial - Without Derivatives 4.0 International), which allows others to freely distribute the published work with a mandatory reference to the authors of the original work and the first publication of the work in this magazine.
• Authors have the right to make independent extra-exclusive work agreements in the form in which they were published by this magazine (for example, posting work in an electronic repository of an institution or publishing as part of a monograph), provided that the link to the first publication of the work in this journal is maintained. .
• Journal policy allows and encourages the publication of manuscripts on the Internet (for example, in institutions' repositories or on personal websites), both before the publication of this manuscript and during its editorial work, as it contributes to the emergence of productive scientific discussion and positively affects the efficiency and dynamics of the citation of the published work (see The Effect of Open Access).
