Trying Mozilla's Things Gateway

Spoiler: It doesn't yet support my hardware and I didn't get it working, but I'm posting anyway because it's ok to fail. :)

Last weekend I was inspired by Lars's posts on IoT to try Mozilla's Things Gateway.

I have an old Raspberry Pi 1 Model B with a RaZberry Z-Wave Daughterboard which I had soldered a larger external antenna on to last year. I used to run OpenHAB on it to control some z-wave devices before I moved last year and since then it's just been in a box. Let's fire it up!

This original Raspberry Pi is a single core 700mhz CPU, so I'm planning on running it headless and doing everything remotely over SSH to save on GUI resources.

Firstly, since I hadn't plugged this in forever, I booted it up, uninstalled OpenHAB and the RaZberry software and upgraded Raspbian to the latest version. It's still as easy as it should be, just edit the sources.list and:

# apt-get update
# apt-get dist-upgrade

I left the upgrade running over night because it was taking so long -- a foreshadowing of things to come! 😩

On the suggestion of the page I linked to above, I removed pulseaudio since I don't use it.

# sudo apt-get purge "pulseaudio*"

The instructions for installing the Gateway are great and easy to follow.

I installed nvm as recommended, although so far I'm regretting it. It takes 12 seconds to start a new login shell now while nvm executes the stuff it put in .bashrc. If I end up logging in very often I'll get rid of it.

On the other hand, the Node world does do permissions well. Most tools can install in a users home directory without any root permissions, which I was happy to do:

$ curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.8/install.sh | bash
$ nvm install --lts
$ nvm use --lts
$ npm --version
5.6.0
$ node --version
v8.9.4
$ npm install --global yarn

Next up was OpenZWave. Their documentation mentions supporting the Razberry card, so I felt good about things working. When you get to the make command, go grab some lunch -- this also took plenty of time.

$ git clone https://github.com/OpenZWave/open-zwave.git
$ cd open-zwave
$ make && sudo make install
$ sudo ldconfig

Finally, I'm ready to install the Things Gateway. For a project that is still new and moving quickly I like to install directly out of Git which makes testing patches and upgrading to new versions easy (I hope!).

$ git clone https://github.com/mozilla-iot/gateway.git
$ cd gateway
$ yarn  # Yarn actually prints how long it takes to execute.  This took 6070.32 seconds! 💤

I don't want to use Mozilla's tunneling service, so I faked some SSL:

$ mkdir -p ssl
$ openssl genrsa -out ssl/privatekey.pem 2048
$ openssl req -new -sha256 -key ssl/privatekey.pem -out ssl/csr.pem
$ openssl x509 -req -in ssl/csr.pem -signkey ssl/privatekey.pem -out ssl/certificate.pem

And then start the web server.

$ npm start  # This takes around 4 minutes to finish starting :(

Loading the site in my browser works! I click settings -> add-ons and install z-wave device support. However, watching the console I can see it fail to find the z-wave interface.

I dug around in the code and found the findZWavePort() code that does the detection, and confrimed that it's looking at USB ports which won't work for the Razberry which comes in on /dev/ttyAMA0.

I'm just going to stick in a little aside here...

I modified the zwave-adapter.js file to add some logging and make sure that was as far as it was getting and restarted the server (which, took 4 minutes...) and then saw the module failed to load because the checksum didn't match. I deleted the checksum file (4 minutes...), and it failed again, so I finally just removed the one line from the checksum file.

The checksum is a great idea and I assume there is some kind of flag I can pass to the server to ignore it, but I haven't found it yet.

One last thing I wanted to verify before I dug too far into the Gateway was to make sure OpenZWave was working. They include a program with their code called MinOZW. I didn't see documentation for it, but it takes the port to monitor as it's first parameter:

$ MinOZW /dev/ttyAMA0

That command gave me a flood of Z-wave data, so I knew it was seeing the controller properly.

Searching the open Mozilla IOT issues revealed that the request to support the Razberry board was filed a couple weeks ago. So, I CC'd myself to keep an eye on it, and that's as far as I got. So far:

  • I have really high hopes for the Mozilla IoT Gateway
  • A first generation Raspberry Pi might be too slow for these shenanigans 😞

Some GMail filters to make sorting through Mozilla email easier

I use some GMail filters to help sort out the flood of emails I get from Mozilla projects. They help me identify bugs and issues I've been asked for information on versus the mail I get from, for example, being an owner for our GitHub organization.

I thought I blogged about this before but I couldn't find the post, and maybe Google's new Inbox will make this irrelevant, but in the mean time:

I have some labels set up in Gmail:

And some filters:

Matches: from:(bugzilla-daemon@mozilla.org) to:(wclouser@mozilla.com) ("X-Bugzilla-Reason: Reporter" OR "X-Bugzilla-Reason: CC")
Do this: Apply label "bug/cc"
Matches: from:(bugzilla-daemon@mozilla.org) to:(wclouser@mozilla.com) "X-Bugzilla-Severity: blocker"
Do this: Apply label "bug/blocker"
Matches: from:(bugzilla-daemon@mozilla.org) to:(wclouser@mozilla.com) "X-Bugzilla-Assigned-To: wclouser@mozilla.com"
Do this: Apply label "bug/mine"
Matches: from:(bugzilla-daemon@mozilla.org) to:(wclouser@mozilla.com) "needinfo?wclouser"
Do this: Apply label "bug/mine"
Matches: from:(notifications@github.com)
Do this: Skip Inbox, Apply label "github"
Matches: from:(notifications@github.com) "You are receiving this because you are on a team that was mentioned."
Do this: Skip Inbox, Apply label "github/mine"
Matches: from:(notifications@github.com) to:(wclouser@mozilla.com) "You are receiving this because you were mentioned."
Do this: Apply label "github/mine"

Then I just keep an eye on the labels where things are assigned to me and I can mostly stay up with requests. This system isn't perfect -- how do you handle the mail? :)

Test Pilot Legacy Program Final Review

One of my Q3 goals was to migrate the Legacy Test Pilot users into our new Test Pilot program (some background on the two programs). The previous program was similar in that people could give feedback on experiments, but different enough that we didn't feel comfortable simply moving the users to the new program without some kind of notification and opting-in.

We decided the best way to do that was simply push out a new version of the legacy add-on which opened a new tab to the Test Pilot website and then uninstalled itself. This lets people interested in testing experiments know about the new program without being overbearing. Worst case scenario, they close the tab and have one less add-on loading every time Firefox is started.

In our planning meeting it was suggested that getting three percent of users from the old program to the new would be a reasonable compromise between realistic and optimistic. I guffawed, suggested that the audience had already opted-in once, and put 6% in as our goal and figured it would be way higher. Spoiler alert: I was wrong.

I'll spare you the pain of writing the add-on (most of the trouble was that the legacy add-on was so old you had to restart Firefox to uninstall it which really broke up the flow). On August 24th, we pushed the update to the old program.

In the daily usage graph, you can see we successfully uninstalled ourselves from several hundred thousand profiles, but we still have a long tail that doesn't seem to be uninstalling. Fortunately, AMO has really great statistics dashboards (these stats are public, by the way) and we can dig a little deeper. So, as of today there are around 150k profiles with the old add-on still installed. About half of those are reporting running the latest version (the one that uninstalls itself) and about half are disabled by the user. I suspect those halves overlap and account for 75k of the installed profiles.

The second 75k profiles are on older add-on versions and are not upgrading to a new version. There could be many reasons when we're dealing with profiles this old: they could be broken, they might not have write permissions to their profile, their network traffic could be being blocked, an internet security suite could be misbehaving, etc. I don't think there is much more we can do for these folks right now, unfortunately.

Let's talk about the overall goal though - how many people joined the new program as a result of the new tab?

As of the end of Q3, we had just over 26k conversions making for a 3.6% conversion rate. Quite close to what was suggested in the original meeting by the people who do this stuff for a living, and quite short of my brash guess.

Overall we got a 0.6 score on the quarterly OKR.


Since I'm writing this post a few weeks after the end of Q3, I can see that we're continuing to get about 80 new users per day from the add-on prompt. Overall that makes for about 28.5k total conversions as of Oct 27th.

Test Pilot 2016 Q4 OKRs

The Test Pilot 2016 Q4 OKRs are published. Primarily we'll be focused on continued growth of users (our overall 2016 goal). We deprioritized localization last quarter and over-rotated on publishing experiments by launching four when we were only aiming for one. This quarter we'll turn that knob back down (we're aiming for two new experiments) and get localization done.

We also failed to graduate any experiments last quarter -- arguably the most important part of our entire process since it includes drawing conclusions and publishing our results. This quarter we'll graduate three experiments from Test Pilot, publish our findings so we can improve Firefox, and clear out space in Test Pilot for the next big ideas.

testpilot.firefox.com just got a lot easier to work on

We originally built Test Pilot on top of Django and some JS libraries to fulfill our product requirements as well as keep us flexible enough to evolve quickly since we were a brand new site.

As the site has grown, we've dropped a few requirements, and realized that we were using APIs from our engagement team to collect newsletter sign ups, APIs from our measurement team for our metrics, and everything else on the site was essentially HTML and JS. We used the Django scaffolding for updating the experiments, but there was no reason we needed to.

I'm happy to highlight that as of today testpilot.firefox.com is served 100% statically. Moving to flat files means:

  • Easier to deploy. All we do is copy files to an S3 bucket. No more SQL migrations or strange half-pushed states.

  • More secure. With just flat files we have way less surface area to attack.

  • Easier to participate in. You'll no longer need to set up Docker or a database. Just check out the files, run npm install and you're done. (disclaimer: we just pushed this today, so we actually still need to update the documentation)

  • Excellent change control. Instead of using an admin panel on the site, we now use GitHub to manage our static content. This means all changes are tracked for free, we already have a process in place for reviewing pull requests, and it's easy to roll back or manipulate the data because it's all in the repository already.

If you want to get involved with Test Pilot, come join us in #testpilot (or webchat)!