Metrics visualisation and collection with Graphite, Grafana and python

In this post I'll demonstrate how to collect various metrics using a simple python daemon, send them to Graphite and create dashboards with Grafana that are stored in Elasticsearch.

For more detailed explanation of what Graphite is and it's Carbon and Whisper components please read my other blog post here.

First lets install and configure Graphite:


Next lets install Grafana:

Lastly lets install Elasticsearch to store the dashboards created with Grafana:

There are many fine tools to collect various metrics from logs and snmp, like collectd, Diamond, Ganglia, Graphene and statsd, just to name a few. In most cases however I always needed to graph data that is stored in MySQL, or in SNMP, so I wrote this small python daemon that is highly configurable and perfect for fetching numbers:

https://github.com/kaivanov/Scripts/tree/master/DBaaS/mcollector