Spark Streaming - Monitoring and Metrics
Spark Streaming exposes metrics through multiple layers: the Spark UI, REST API, and programmatic listeners. The streaming tab in Spark UI displays real-time statistics, but production systems…
Read more →Metrics
Prometheus: Metrics Collection and Alerting
Prometheus is an open-source monitoring system built specifically for dynamic cloud environments. Unlike traditional monitoring tools that rely on agents pushing metrics to a central server,…
Read more →Design a Metrics and Monitoring System
Observability rests on three pillars: metrics, logs, and traces. While logs tell you what happened and traces show you the path through your system, metrics answer the fundamental question: ‘Is my…
Read more →