Optimizing Message Transfers in Distributed Messaging Systems through Topic and Partition Management 


Vol. 49,  No. 1, pp. 79-87, Jan.  2024
10.7840/kics.2024.49.1.79


PDF
  Abstract

As stream data processing technology becomes more important, messaging systems such as Apache Kafka, RabbitMQ, and ActiveMQ are being used to transfer large amounts of data fast and without loss. Apache Kafka is a representative distributed messaging system, which can deliver data generated in real time. In Apache Kafka, a broker is composed of multiple topics with different numbers of partitions. As the number of partitions increases, its processing speed also increases, but problems with CPU and memory usages also occur. In this article, we show why the number of partitions should be configured to reduce resource usages without impact on target performance. Based on our extensive experimental results, we propose a mechanism that can change the number of partitions according to the amount of transferred message under different execution environments.

  Statistics
Cumulative Counts from November, 2022
Multiple requests among the same browser session are counted as one view. If you mouse over a chart, the values of data points will be shown.


  Related Articles
  Cite this article

[IEEE Style]

B. Nam and Y. Kwon, "Optimizing Message Transfers in Distributed Messaging Systems through Topic and Partition Management," The Journal of Korean Institute of Communications and Information Sciences, vol. 49, no. 1, pp. 79-87, 2024. DOI: 10.7840/kics.2024.49.1.79.

[ACM Style]

Beomjun Nam and Young-Woo Kwon. 2024. Optimizing Message Transfers in Distributed Messaging Systems through Topic and Partition Management. The Journal of Korean Institute of Communications and Information Sciences, 49, 1, (2024), 79-87. DOI: 10.7840/kics.2024.49.1.79.

[KICS Style]

Beomjun Nam and Young-Woo Kwon, "Optimizing Message Transfers in Distributed Messaging Systems through Topic and Partition Management," The Journal of Korean Institute of Communications and Information Sciences, vol. 49, no. 1, pp. 79-87, 1. 2024. (https://doi.org/10.7840/kics.2024.49.1.79)
Vol. 49, No. 1 Index

Search
(IN TITLE, AUTHOR, ABSTRACT,KEYWORDS)

Advanced Search ▼

POPULAR KEYWORDS
(TOP 10 KEYWORDS)

Recent Publications
(LAST 3 YEARS)






pISSN : 1226 - 4717
eISSN : 2287 - 3880