How I Built a Smart Home Monitoring System with MQTT, Go, Prometheus, and Grafana | by Łukasz Kobylski | Oct, 2024

The goal was simple: to be able to monitor and visualize the state of my smart home using a centralized system. This entire setup is running on a Raspberry Pi.

I use many sensors in my smart home setup (using Home Assistant and Node-RED). These sensors communicate via MQTT (using Zigbee2mqtt) as well as Bluetooth thanks to integration with Home Assistant.

I started wondering about the differences in the operation of temperature sensors, as they don’t always report the same temperature and do so with varying frequency. I wanted a unified dashboard for all types of sensors to have an overall view of their operation and to simplify analysis and comparison. Therefore, I decided to build a system that listens to the topics I’m interested in, records the data, and forwards it to Prometheus for visualization in Grafana. I was already using MQTT to communicate with various IoT devices, and I wanted to leverage this to create a monitoring solution that could track metrics like temperature, humidity, and energy consumption over time.

I chose Go as the programming language because it’s fast and well-suited for concurrent processing, making it ideal for an MQTT client.

Prometheus and Grafana were perfect for data collection and visualization, respectively. Docker and Docker Compose helped me easily run all of these services without worrying about setup complexity.

The system’s architecture consists of several components:

  • MQTT Broker (optional) — A central server for sending and receiving messages from smart devices.
  • Prometheus — A time-series database for gathering metrics.
  • Grafana — A visualization tool to create insightful dashboards.
  • Go Application — My MQTT client, responsible for subscribing to MQTT topics, extracting relevant data, and exposing metrics to Prometheus.

Here’s a look at the docker-compose.yaml file for managing all these services.

Step 1: Setting Up MQTT in Go

First, we need to set up an MQTT client to subscribe to topics from our smart home sensors. In this step, I would provide detailed guidance on configuring the paho.mqtt.golang library.

Install the required MQTT library:
In your Go project, you can add the required dependency with:

go get github.com/eclipse/paho.mqtt.golang

Configure MQTT options:
You need to configure the broker address and provide a unique client ID:

opts := mqtt.NewClientOptions().AddBroker("tcp://mqtt-broker:1883")
opts.SetClientID("smart-home-monitor")

Connect to the MQTT broker:

client := mqtt.NewClient(opts)
if token := client.Connect(); token.Wait() && token.Error() != nil {
log.Fatalf("Could not connect to MQTT broker: %v", token.Error())
}

Step 2: Subscribing to Topics

We subscribe to topics related to various sensor data. Here’s how to do it:

  • Define the topics you want to subscribe to, for example, temperature, humidity, etc.
topic := "smarthome/sensor/temperature"
client.Subscribe(topic, 0, func(client mqtt.Client, msg mqtt.Message) {
log.Printf("Received message: %s from topic: %s", msg.Payload(), msg.Topic())
})

You can add more subscriptions for different sensors as needed, and this can all be done inside a function to handle subscriptions systematically.

Step 3: Handling Incoming Data

The incoming data needs to be processed and stored. Let’s dive into exporting it to Prometheus:

You will need to convert the MQTT data into Prometheus metrics. Use the prometheus/client_golang library for this:

  • Install the Prometheus client library:
go get github.com/prometheus/client_golang/prometheus
go get github.com/prometheus/client_golang/prometheus/promhttp
  • Create Prometheus metrics:
var temperatureGauge = prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "temperature",
Help: "Temperature gauge from sensors",
},
[]string{"sensor"},
)

prometheus.MustRegister(temperatureGauge)

  • Update metrics from incoming MQTT messages:
client.Subscribe("smarthome/sensor/temperature", 0, func(client mqtt.Client, msg mqtt.Message) {
temperature, err := strconv.ParseFloat(string(msg.Payload()), 64)
if err == nil {
temperatureGauge.With(prometheus.Labels{"sensor": "temperature"}).Set(temperature)
} else {
log.Printf("Error converting temperature: %v", err)
}
})

Step 4: Exposing Metrics to Prometheus

  • Run an HTTP server to expose the metrics:
    Prometheus scrapes data via HTTP endpoints, so you need to start a web server to serve these metrics:
go func(){
http.Handle("/metrics", promhttp.Handler())
log.Fatal(http.ListenAndServe(":8080", nil))
}()

This will allow Prometheus to collect the metrics at the /metrics endpoint.

To keep everything easy to manage, I put all components into Docker containers.

Here’s an Dockerfile that builds the Go application.

This is my docker-compose to join all services togeather.

With this setup, starting all services becomes as simple as running:

docker-compose up -d

Access Prometheus and Grafana:

Use Grafana to create insightful dashboards to monitor your smart home metrics.

With Prometheus collecting the data and Grafana providing dashboards, I now have a powerful tool for monitoring the state of my home. It gives me the ability to track changes over time, visualize patterns, and detect any issues instantly.

If you want to explore the full code, check out my GitHub repository.

This project allowed me to bridge my interest in IoT and my love for Go. It also provided me with an excellent tool for keeping tabs on my smart home. I hope this guide has given you a quick start for building your own monitoring solution.

If you have questions or need help setting up your environment, feel free to reach out or refer to the GitHub repository for more details.