05 Dec 2018
In this post I’ll set up the software needed to store data from openHAB
and display it on a dashboard. I’ll use a z-wave thermostat I have already set
up on my network, and I’ll add Grafana and InfluxDB.

Set up the containers
Lately I’ve been trying to compartmentalize things, so I want to run Grafana and
InfluxDB in Docker. There are options that run them all in the same container,
but I liked nicolargo/docker-influxdb-grafana which is essentially just a
simple docker-compose file that uses base (read: maintained) images.
$ git checkout https://github.com/nicolargo/docker-influxdb-grafana.git
In docker-compose.yml
there are two volumes
keys that link out to the
disk. I modified those paths, but otherwise didn’t make any changes. By
setting up the volumes here, I’ll have persistent storage beyond restarts.
When I first tried to run the containers, the Grafana container failed with
permission problems. Grafana mentions some permissions changes in a different context,
but following these steps fixed my issues also:
$ docker run -ti --user root --volume "/data/docker/grafana/data:/var/lib/grafana" --entrypoint bash grafana/grafana:latest
# then within the container, run:
$ chown -R root:root /etc/grafana
$ chmod -R a+r /etc/grafana
$ chown -R grafana:grafana /var/lib/grafana
$ chown -R grafana:grafana /usr/share/grafana
After that the containers fired right up:
Set up InfluxDB
InfluxDB is running now, but I need to add the new user and database. First I
opened a shell with docker exec -it influxdb influx
and then made the
changes:
> create database openhab
> create user openhab with password "openhab"
> grant all on openhab to openhab
Getting data into InfluxDB
As with the rest of this walkthrough, someone already did the hard parts, so I
really just need to install a plugin for openHAB.
- Load the PaperUI in openHAB
- Click Add-ons in the left menu
- Click Persistence in the top menu
- Click Install next to InfluxDB Persistence
Next I’ll SSH into the openHAB box and edit or create two configuration files.
Copies of mine are below.
# /etc/openhab2/services/influxdb.cfg
# This is the URL for influxdb. Don't forget the port.
url=http://192.168.1.2:8086
user=openhab
password=openhab
db=openhab
# /etc/openhab2/persistence/influxdb.persist
Strategies {
everyMinute : "0 * * * * ?"
everyHour : "0 0 * * * ?"
everyDay : "0 0 0 * * ?"
default = everyChange
}
Items {
* : strategy = everyChange, everyHour
}
I have an *
in my Items declaration because I want to send everything to
InfluxDB. That can be changed to specific Items or Groups to narrow down what
is sent if desired.
With those files in place, openHAB will send data to InfluxDB every time
something changes and every hour. I had to restart openHAB for the changes to
take effect:
$ sudo systemctl restart openhab2
Make sure data is flowing
Look at the logs for InfluxDB by running docker-compose logs influxdb
.
openHAB should have sent a hello:
influxdb |
[httpd] 192.168.1.1 - - [02/Dec/2018:02:21:49 +0000] “GET /ping HTTP/1.1” 204 0 “-“ “okhttp/2.4.0” |
Depending on how much activity there is in openHAB there may be other entries in
the logs also. To double check inside InfluxDB itself try this:
$ docker exec -it influxdb influx
# And then inside the influxdb client
> use openhab
> show measurements
A list of all the measurements sent so far will display. select * from
somekey
will dig into the data itself.
Set up Grafana
The Grafana interface is on port 3000 by default and the credentials are
admin/admin. On first login Grafana shows a wizard interface, so this moves
along pretty quickly. The new Data Source is the InfluxDB type, probably with
a URL of http://influxdb:8086
. Docker-compose does the DNS for free here.
I’ll put a Graph panel on a new dashboard and click the title of it to edit.
The metrics tab has the queries and since the data is already flowing in, the
correctly named Keys appear in a dropdown when I’m editing them. The query can
stay mostly as defaults, here is what mine looks like:

My Key names (eg. HVAC_Temperature
) come from how I set the device up in openHAB.
The only other thing to change here is in the Display tab: I set the Null Value
to connected
so I’ll have a nicely drawn line chart even though the
Temperature only changes periodically.
One more bit of fanciness
I have a wood stove that I use in winter, but if it gets below a certain
temperature (depending on the time of day) I have my furnace kick on. I wanted
to be able to see when my furnace was coming on, so I decided to make use of
Grafana’s Annotations capability (the red vertical lines in the top image).
From within the dashboard I clicked the Settings Gear in the header (not the
one on the left) and chose Annotations. From there I made a new Annotation
with a custom query:
SELECT value from HVAC_Operating_State WHERE value=1 AND $timeFilter ORDER BY time ASC LIMIT 10
The AND $timeFilter
is important. As before, mine is named
HVAC_Operating_State
because that’s what I named the Item in openHAB when I
set it up.
And that’s it. Returning to the dashboard after 24 hours should show something
like the graph at the top of this post.
21 Feb 2018
Spoiler: It doesn’t yet support my hardware and I didn’t get it working, but
I’m posting anyway because it’s ok to fail. :)
Last weekend I was inspired by Lars’s posts on IoT to try Mozilla’s Things
Gateway.
I have an old Raspberry Pi 1 Model B with a RaZberry Z-Wave Daughterboard
which I had soldered a larger external antenna on to last year. I used to run
OpenHAB on it to control some z-wave devices before I moved last year and since
then it’s just been in a box. Let’s fire it up!
This original Raspberry Pi is a single core 700mhz CPU, so I’m planning on
running it headless and doing everything remotely over SSH to save on GUI
resources.

Firstly, since I hadn’t plugged this in forever, I booted it up, uninstalled
OpenHAB and the RaZberry software and upgraded Raspbian to the latest
version. It’s still as
easy as it should be, just edit the sources.list and:
# apt-get update
# apt-get dist-upgrade
I left the upgrade running over night because it was taking so long – a
foreshadowing of things to come! 😩
On the suggestion of the page I linked to above, I removed pulseaudio
since I
don’t use it.
# sudo apt-get purge "pulseaudio*"
The instructions for installing the Gateway are great and easy to follow.
I installed nvm
as recommended, although so far I’m regretting it. It takes
12 seconds to start a new login shell now while nvm executes the stuff it put
in .bashrc
. If I end up logging in very often I’ll get rid of it.
On the other hand, the Node world does do permissions well. Most tools can
install in a users home directory without any root permissions, which I was
happy to do:
$ curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.8/install.sh | bash
$ nvm install --lts
$ nvm use --lts
$ npm --version
5.6.0
$ node --version
v8.9.4
$ npm install --global yarn
Next up was OpenZWave. Their documentation mentions supporting the Razberry
card, so I felt good about things working. When you get to the make
command,
go grab some lunch – this also took plenty of time.
$ git clone https://github.com/OpenZWave/open-zwave.git
$ cd open-zwave
$ make && sudo make install
$ sudo ldconfig
Finally, I’m ready to install the Things Gateway. For a project that is still
new and moving quickly I like to install directly out of Git which makes
testing patches and upgrading to new versions easy (I hope!).
$ git clone https://github.com/mozilla-iot/gateway.git
$ cd gateway
$ yarn # Yarn actually prints how long it takes to execute. This took 6070.32 seconds! 💤
I don’t want to use Mozilla’s tunneling service, so I faked some SSL:
$ mkdir -p ssl
$ openssl genrsa -out ssl/privatekey.pem 2048
$ openssl req -new -sha256 -key ssl/privatekey.pem -out ssl/csr.pem
$ openssl x509 -req -in ssl/csr.pem -signkey ssl/privatekey.pem -out ssl/certificate.pem
And then start the web server.
$ npm start # This takes around 4 minutes to finish starting :(
Loading the site in my browser works! I click settings
-> add-ons
and
install z-wave device support
. However, watching the console I can see it
fail to find the z-wave interface.
I dug around in the code and found the findZWavePort() code that does the
detection, and confrimed that it’s looking at USB ports which won’t work for
the Razberry which comes in on /dev/ttyAMA0
.
I'm just going to stick in a little aside here...
I modified the zwave-adapter.js
file to add some logging and
make sure that was as far as it was getting and restarted the server (which,
took 4 minutes...) and then saw the module failed to load because the checksum
didn't match. I deleted the checksum file (4 minutes...), and it failed again,
so I finally just removed the one line from the checksum file.
The checksum is a great idea and I assume there is some kind of flag I can pass
to the server to ignore it, but I haven't found it yet.
One last thing I wanted to verify before I dug too far into the Gateway was to
make sure OpenZWave was working. They include a program with their code called
MinOZW
. I didn’t see documentation for it, but it takes the port to monitor
as it’s first parameter:
That command gave me a flood of Z-wave data, so I knew it was seeing the
controller properly.
Searching the open Mozilla IOT issues revealed that the request to support the Razberry
board was filed a couple weeks ago. So, I CC’d myself to keep an eye on
it, and that’s as far as I got. So far:
- I have really high hopes for the Mozilla IoT Gateway
- A first generation Raspberry Pi might be too slow for these shenanigans 😞
07 Jul 2017
I use some GMail filters to help sort out the flood of emails I get from Mozilla
projects. They help me identify bugs and issues I’ve been asked for
information on versus the mail I get from, for example, being an owner for our
GitHub organization.
I thought I blogged about this before but I couldn’t find the post, and maybe
Google’s new Inbox will make this irrelevant, but in the mean time:
I have some labels set up in Gmail:

And some filters:
Matches: from:(bugzilla-daemon@mozilla.org) to:(wclouser@mozilla.com) ("X-Bugzilla-Reason: Reporter" OR "X-Bugzilla-Reason: CC")
Do this: Apply label "bug/cc"
Matches: from:(bugzilla-daemon@mozilla.org) to:(wclouser@mozilla.com) "X-Bugzilla-Severity: blocker"
Do this: Apply label "bug/blocker"
Matches: from:(bugzilla-daemon@mozilla.org) to:(wclouser@mozilla.com) "X-Bugzilla-Assigned-To: wclouser@mozilla.com"
Do this: Apply label "bug/mine"
Matches: from:(bugzilla-daemon@mozilla.org) to:(wclouser@mozilla.com) "needinfo?wclouser"
Do this: Apply label "bug/mine"
Matches: from:(notifications@github.com)
Do this: Skip Inbox, Apply label "github"
Matches: from:(notifications@github.com) "You are receiving this because you are on a team that was mentioned."
Do this: Skip Inbox, Apply label "github/mine"
Matches: from:(notifications@github.com) to:(wclouser@mozilla.com) "You are receiving this because you were mentioned."
Do this: Apply label "github/mine"
Then I just keep an eye on the labels where things are assigned to me and I can
mostly stay up with requests. This system isn’t perfect – how do you handle
the mail? :)
27 Oct 2016
One of my Q3 goals was to migrate the Legacy Test Pilot users into our new
Test Pilot program (some background on the two programs). The previous
program was similar in that people could give feedback on experiments, but
different enough that we didn’t feel comfortable simply moving the users to the
new program without some kind of notification and opting-in.
We decided the best way to do that was simply push out a new version of the
legacy add-on which opened a new tab to the Test Pilot website and then
uninstalled itself. This lets people interested in testing experiments know
about the new program without being overbearing. Worst case scenario, they
close the tab and have one less add-on loading every time Firefox is started.
In our planning meeting it was suggested that getting three percent of users
from the old program to the new would be a reasonable compromise between
realistic and optimistic. I guffawed, suggested that the audience had already
opted-in once, and put 6% in as our goal and figured it would be way higher.
Spoiler alert: I was wrong.
I’ll spare you the pain of writing the add-on (most of the trouble was that the
legacy add-on was so old you had to restart Firefox to uninstall it which really
broke up the flow). On August 24th, we pushed the update to the old program.

In the daily usage graph, you can see we successfully uninstalled ourselves from
several hundred thousand profiles, but we still have a long tail that doesn’t
seem to be uninstalling. Fortunately, AMO has really great statistics
dashboards (these stats are public, by the way) and we can dig a little
deeper. So, as of today there are around 150k profiles with the old add-on
still installed. About half of those are reporting running the latest version
(the one that uninstalls itself) and about half are disabled by the user. I
suspect those halves overlap and account for 75k of the installed profiles.
The second 75k profiles are on older add-on versions and are not upgrading to a
new version. There could be many reasons when we’re dealing with profiles this
old: they could be broken, they might not have write permissions to their
profile, their network traffic could be being blocked, an internet security
suite could be misbehaving, etc. I don’t think there is much more we can do
for these folks right now, unfortunately.
Let’s talk about the overall goal though - how many people joined the new
program as a result of the new tab?

As of the end of Q3, we had just over 26k conversions making for a 3.6%
conversion rate. Quite close to what was suggested in the original meeting by
the people who do this stuff for a living, and quite short of my brash guess.
Overall we got a 0.6 score on the quarterly OKR.
Since I’m writing this post a few weeks after the end of Q3, I can see that
we’re continuing to get about 80 new users per day from the add-on prompt.
Overall that makes for about 28.5k total conversions as of Oct 27th.
14 Oct 2016
The Test Pilot 2016 Q4 OKRs are published. Primarily we’ll be focused on
continued growth of users (our overall 2016 goal). We deprioritized
localization last quarter and over-rotated on publishing experiments by
launching four when we were only aiming for one. This quarter we’ll turn that
knob back down (we’re aiming for two new experiments) and get localization done.
We also failed to graduate any experiments last quarter – arguably the most
important part of our entire process since it includes drawing conclusions and
publishing our results. This quarter we’ll graduate three experiments from Test
Pilot, publish our findings so we can improve Firefox, and clear out space in
Test Pilot for the next big ideas.