Visualization - revision history

Dank: /* Data sources */

2023-06-22T04:28:42Z

Data sources

← Older revision		Revision as of 04:28, 22 June 2023
Line 4:		Line 4:

	==Data sources==		==Data sources==
	I've got an [[ESP8266]]-powered [https://sonoff.tech/product/smart-plugs/s31-s31lite/ Sonoff S31] smart plug on pretty much every outlet plate. They can be found for less than $20 on sale and consume negligible parasitic power. The vendor firmware is cursed, and ~~they~~ ought immediately ~~be flashed~~ with open source [https://tasmota.github.io/docs/ Tasmota] (this takes about ten minutes per device). Once this is done, enable MQTT. From these, I sample wattage, AC voltage, and wireless quality.		I've got an [[ESP8266]]-powered [https://sonoff.tech/product/smart-plugs/s31-s31lite/ Sonoff S31] smart plug on pretty much every outlet plate. They can be found for less than $20 on sale and consume negligible parasitic power. The vendor firmware is cursed, and ought be immediately replaced with open source [https://tasmota.github.io/docs/ Tasmota] (this takes about ten minutes per device). Once this is done, enable MQTT. From these, I sample wattage, AC voltage, and wireless quality.

	[[File:Voltage-grafana.png\|24 hour history of AC voltages in my condo]]		[[File:Voltage-grafana.png\|24 hour history of AC voltages in my condo]]

Dank at 04:25, 22 June 2023

2023-06-22T04:25:15Z

← Older revision		Revision as of 04:25, 22 June 2023
Line 1:		Line 1:
	My current home metrics + visualization stack is self-hosted [https://grafana.com/ Grafana OSS] atop a [https://github.com/prometheus Prometheus] time series database. Prometheus is fed by [https://github.com/hikhvar/mqtt2prometheus mqtt2prometheus], which reads [https://mosquitto.org/ Mosquitto]-brokered metrics from my [https://github.com/dankamongmen/counterforce bespoke scripts] and various IoT devices. It also ingests [[SNMP]] via its own snmp_exporter. I was originally using Prometheus's node_exporter, but thought it horribly heavyweight.		My current home metrics + visualization stack is self-hosted [https://grafana.com/ Grafana OSS] atop a [https://github.com/prometheus Prometheus] time series database. Prometheus is fed by [https://github.com/hikhvar/mqtt2prometheus mqtt2prometheus], which reads [https://mosquitto.org/ Mosquitto]-brokered metrics from my [https://github.com/dankamongmen/counterforce bespoke scripts] and various IoT devices. Grafana also ingests [[SNMP]] from my switches via its own snmp_exporter. I was originally using Prometheus's node_exporter, but thought it horribly heavyweight and kicked that bitch to the curb.

	At Microsoft, I use Grafana atop an otherwise custom stack. At Google, I used an entirely custom stack because hey, SWEs gotta get promoted, and the authors of Monarch weren't gonna demonstrate complexity by adapting standard open source tooling.		At Microsoft, I use Grafana atop an otherwise custom stack. At Google, I used an entirely custom stack because hey, SWEs gotta get promoted, and the authors of Monarch weren't gonna demonstrate complexity by adapting standard open source tooling.

Dank: /* Data sources */

2023-06-11T10:59:57Z

Data sources

← Older revision		Revision as of 10:59, 11 June 2023
Line 7:		Line 7:

	[[File:Voltage-grafana.png\|24 hour history of AC voltages in my condo]]		[[File:Voltage-grafana.png\|24 hour history of AC voltages in my condo]]

			On each machine, I run a script (from [[systemd]], of course) that periodically samples a bunch of crap, marshals it into JSON with <tt>jq</tt>, and then sends it off as a unit with <tt>mosquitto_pub</tt>. Try to get everything for a single machine into a single block so as not to make more than one TLS connection per sampling period (note that this means all metrics will arrive under one topic, which might not jive with your schema). I run with a 15s quantua across all my devices. I recommend being liberal but loud in the face of sampling errors; ideally, errors in sampling will themselves be graphed, and brought to your immediate attention without breaking the device's reporting entirely.

			For different machines, supply a different <tt>--id</tt> to <tt>mosquitto_pub</tt>. This will allow you to differentiate them via the <tt>sensor</tt> label in Grafana.

	==Prometheus==		==Prometheus==

Dank: /* Prometheus */

2023-06-11T10:45:37Z

Prometheus

← Older revision		Revision as of 10:45, 11 June 2023
Line 12:		Line 12:

	By default, Prometheus provides a data explorer UI and API on port 9090. Exporters (e.g. snmp_exporter) usually run their own UIs/APIs on their own ports (e.g. 9116 for snmp_exporter).		By default, Prometheus provides a data explorer UI and API on port 9090. Exporters (e.g. snmp_exporter) usually run their own UIs/APIs on their own ports (e.g. 9116 for snmp_exporter).

			===mqtt2prometheus===

	===[[SNMP]]===		===[[SNMP]]===

Dank: /* Data sources */

2023-06-11T10:45:07Z

Data sources

← Older revision		Revision as of 10:45, 11 June 2023
Line 4:		Line 4:

	==Data sources==		==Data sources==
	I've got an [[ESP8266]]-powered [https://sonoff.tech/product/smart-plugs/s31-s31lite/ Sonoff S31] smart plug on pretty much every outlet plate. They can be found for less than $20 on sale and consume negligible parasitic power. The vendor firmware is cursed, and they ought immediately be flashed with [https://tasmota.github.io/docs/ Tasmota]. Once this is done, enable MQTT. From these, I sample wattage, AC voltage, and wireless quality.		I've got an [[ESP8266]]-powered [https://sonoff.tech/product/smart-plugs/s31-s31lite/ Sonoff S31] smart plug on pretty much every outlet plate. They can be found for less than $20 on sale and consume negligible parasitic power. The vendor firmware is cursed, and they ought immediately be flashed with open source [https://tasmota.github.io/docs/ Tasmota] (this takes about ten minutes per device). Once this is done, enable MQTT. From these, I sample wattage, AC voltage, and wireless quality.

	[[File:Voltage-grafana.png\|24 hour history of AC voltages in my condo]]		[[File:Voltage-grafana.png\|24 hour history of AC voltages in my condo]]

Dank at 10:44, 11 June 2023

2023-06-11T10:44:24Z

← Older revision		Revision as of 10:44, 11 June 2023
Line 2:		Line 2:

	At Microsoft, I use Grafana atop an otherwise custom stack. At Google, I used an entirely custom stack because hey, SWEs gotta get promoted, and the authors of Monarch weren't gonna demonstrate complexity by adapting standard open source tooling.		At Microsoft, I use Grafana atop an otherwise custom stack. At Google, I used an entirely custom stack because hey, SWEs gotta get promoted, and the authors of Monarch weren't gonna demonstrate complexity by adapting standard open source tooling.

			==Data sources==
			I've got an [[ESP8266]]-powered [https://sonoff.tech/product/smart-plugs/s31-s31lite/ Sonoff S31] smart plug on pretty much every outlet plate. They can be found for less than $20 on sale and consume negligible parasitic power. The vendor firmware is cursed, and they ought immediately be flashed with [https://tasmota.github.io/docs/ Tasmota]. Once this is done, enable MQTT. From these, I sample wattage, AC voltage, and wireless quality.

			[[File:Voltage-grafana.png\|24 hour history of AC voltages in my condo]]

	==Prometheus==		==Prometheus==

Dank: /* SNMP */

2023-06-08T21:39:37Z

SNMP

← Older revision		Revision as of 21:39, 8 June 2023
Line 9:		Line 9:

	===[[SNMP]]===		===[[SNMP]]===
	Debian's [https://packages.debian.org/source/bullseye/prometheus-snmp-exporter prometheus-snmp-exporter] package provides <tt>prometheus-snmp-generator</tt> and the <tt>prometheus-snmp-exporter</tt> server. The former is used to build <tt>/etc/prometheus/snmp.yml</tt>. The package cannot Depend on snmp-mibs-downloader due to this package being in non-free, but you'll want it for the <tt>download-mibs</tt> script.		Debian's [https://packages.debian.org/source/bullseye/prometheus-snmp-exporter prometheus-snmp-exporter] package provides <tt>prometheus-snmp-generator</tt> and the <tt>prometheus-snmp-exporter</tt> server. The former is used to build <tt>/etc/prometheus/snmp.yml</tt>. The package cannot Depend on [https://packages.debian.org/source/buster/snmp-mibs-downloader snmp-mibs-downloader] due to this package being in non-free, but you'll want it for the <tt>download-mibs</tt> script.

	==Grafana==		==Grafana==

Dank: /* SNMP */

2023-06-08T21:39:07Z

SNMP

← Older revision		Revision as of 21:39, 8 June 2023
Line 9:		Line 9:

	===[[SNMP]]===		===[[SNMP]]===
	~~The~~ [~~[Debian]]~~ prometheus-snmp-exporter package provides <tt>prometheus-snmp-generator</tt> and the <tt>prometheus-snmp-exporter</tt> server. The former is used to build <tt>/etc/prometheus/snmp.yml</tt>. The package cannot Depend on snmp-mibs-downloader due to this package being in non-free, but you'll want it for the <tt>download-mibs</tt> script.		Debian's [https://packages.debian.org/source/bullseye/prometheus-snmp-exporter prometheus-snmp-exporter] package provides <tt>prometheus-snmp-generator</tt> and the <tt>prometheus-snmp-exporter</tt> server. The former is used to build <tt>/etc/prometheus/snmp.yml</tt>. The package cannot Depend on snmp-mibs-downloader due to this package being in non-free, but you'll want it for the <tt>download-mibs</tt> script.

	==Grafana==		==Grafana==

Dank at 21:38, 8 June 2023

2023-06-08T21:38:31Z

← Older revision		Revision as of 21:38, 8 June 2023
Line 1:		Line 1:
	My current home metrics + visualization stack is self-hosted [https://grafana.com/ Grafana OSS] atop a [https://github.com/prometheus Prometheus] time series database. Prometheus is fed by [https://github.com/hikhvar/mqtt2prometheus mqtt2prometheus], which reads [https://mosquitto.org/ Mosquitto]-brokered metrics from my [https://github.com/dankamongmen/counterforce bespoke scripts] and various IoT devices. It also ingests SNMP via its own snmp_exporter. I was originally using Prometheus's node_exporter, but thought it horribly heavyweight.		My current home metrics + visualization stack is self-hosted [https://grafana.com/ Grafana OSS] atop a [https://github.com/prometheus Prometheus] time series database. Prometheus is fed by [https://github.com/hikhvar/mqtt2prometheus mqtt2prometheus], which reads [https://mosquitto.org/ Mosquitto]-brokered metrics from my [https://github.com/dankamongmen/counterforce bespoke scripts] and various IoT devices. It also ingests [[SNMP]] via its own snmp_exporter. I was originally using Prometheus's node_exporter, but thought it horribly heavyweight.

	At Microsoft, I use Grafana atop an otherwise custom stack. At Google, I used an entirely custom stack because hey, SWEs gotta get promoted, and the authors of Monarch weren't gonna demonstrate complexity by adapting standard open source tooling.		At Microsoft, I use Grafana atop an otherwise custom stack. At Google, I used an entirely custom stack because hey, SWEs gotta get promoted, and the authors of Monarch weren't gonna demonstrate complexity by adapting standard open source tooling.

Dank: /* Prometheus */

2023-06-08T21:38:21Z

Prometheus

← Older revision		Revision as of 21:38, 8 June 2023
Line 7:		Line 7:

	By default, Prometheus provides a data explorer UI and API on port 9090. Exporters (e.g. snmp_exporter) usually run their own UIs/APIs on their own ports (e.g. 9116 for snmp_exporter).		By default, Prometheus provides a data explorer UI and API on port 9090. Exporters (e.g. snmp_exporter) usually run their own UIs/APIs on their own ports (e.g. 9116 for snmp_exporter).

			===[[SNMP]]===
			The [[Debian]] prometheus-snmp-exporter package provides <tt>prometheus-snmp-generator</tt> and the <tt>prometheus-snmp-exporter</tt> server. The former is used to build <tt>/etc/prometheus/snmp.yml</tt>. The package cannot Depend on snmp-mibs-downloader due to this package being in non-free, but you'll want it for the <tt>download-mibs</tt> script.

	==Grafana==		==Grafana==