When to use fluentd vs prometheus vs elasticsearch? There seems to be overlapping between these tools interms of the feature set they provide but I have seen projects using all these 3 tools in unison. Quite confusion why these 3 tools has to be in the same solution. Does these 3 tools need to be used together? as it may requires different management team and expertise.
fluentd - for application log management not metrics - collects log, do transformation and ingest to various destinations
prometheus -for scraping metrics especially on container environment. Does this do application log management as well? not sure
elastricsearch - scales well for log storage & do efficient search queries.
Does I missed anything form above understanding? need your expert opinions.
Fluentd: Log collection and forwarding.
Prometheus: Metrics collection and alerting.
Elasticsearch: Log storage, search, and visualization.
While there's an overlap, Fluentd, Prometheus, and Elasticsearch each serve different primary purposes. However, in a complex environment, especially in the context of microservices or container orchestration systems like Kubernetes, having a comprehensive view through metrics (Prometheus) and logs (Fluentd + Elasticsearch) is valuable. That said, teams should evaluate their actual needs and the associated overhead before adopting all three simultaneously.