Software for collecting and analyzing metrics in highly loaded applications based on the Prometheus monitoring system
Вантажиться...
Файли
Дата
2024
Автори
Науковий керівник
Назва журналу
Номер ISSN
Назва тому
Видавець
National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"
Анотація
This paper emphasizes the importance of collecting metrics during application operation for early detection of potential problems. The undisputed leader in this area is the Prometheus monitoring system, which, combined with Grafana – a platform for visualizing collected data in numerous graphs – becomes an indispensable tool for programmers and site reliability engineers. However, the average value of a certain metric is often unrepresentative, because it does not reflect a comprehensive picture. Instead, collecting metrics in terms of various quantiles over a long period is useful to identify even single instabilities. Still, the use of standard tools in the Python ecosystem may require a lot of server resources and long preliminary analysis, which can be quite costly for businesses from a financial point of view. That is why the development of a new approach for collecting and analyzing metrics in highly loaded applications based on the Prometheus monitoring system is relevant. The research aims to improve the efficiency of storing metrics across different quantiles, which will create additional opportunities for further analysis. A review of existing approaches for calculating quantile values on large data sets was conducted. Their comparative characteristics in terms of speed and memory usage were also presented. The chosen method was adapted for use with the real-time data stream and implemented as a Python extension for the official Prometheus library. It opens up opportunities for comprehensive monitoring of highly loaded systems in terms of both server resource usage and the quantity and quality of collected useful data. This solution can be easily implemented on large projects requiring continuous tracking of various metrics to ensure stable and uninterrupted service operation.
Опис
Ключові слова
metrics, quantile, highly loaded applications, Prometheus monitoring system, Python, метрики, квантиль, високонавантажені застосунки, моніторингова система Prometheus
Бібліографічний опис
Stetsenko, I. Software for collecting and analyzing metrics in highly loaded applications based on the Prometheus monitoring system / Inna Stetsenko, Anton Myroniuk // Information, Computing and Intelligent systems. – 2024. – No. 5. – Pp. 17-28. – Bibliogr.: 15 ref.