14 Oct 2016
The Test Pilot 2016 Q4 OKRs are published. Primarily we'll be focused on
continued growth of users (our overall 2016 goal). We deprioritized
localization last quarter and over-rotated on publishing experiments by
launching four when we were only aiming for one. This quarter we'll turn that
knob back down (we're aiming for two new experiments) and get localization done.
We also failed to graduate any experiments last quarter -- arguably the most
important part of our entire process since it includes drawing conclusions and
publishing our results. This quarter we'll graduate three experiments from Test
Pilot, publish our findings so we can improve Firefox, and clear out space in
Test Pilot for the next big ideas.
11 Oct 2016
We originally built Test Pilot on top of Django and some JS libraries to fulfill
our product requirements as well as keep us flexible enough to evolve quickly
since we were a brand new site.
As the site has grown, we've dropped a few requirements, and realized that we
were using APIs from our engagement team to collect newsletter sign ups,
APIs from our measurement team for our metrics, and everything else on the
site was essentially HTML and JS. We used the Django scaffolding for updating
the experiments, but there was no reason we needed to.
I'm happy to highlight that as of today testpilot.firefox.com is served
100% statically. Moving to flat files means:
Easier to deploy. All we do is copy files to an S3 bucket. No more SQL
migrations or strange half-pushed states.
More secure. With just flat files we have way less surface area to attack.
Easier to participate in. You'll no longer need to set up Docker or a
database. Just check out the files, run
npm install and you're done.
(disclaimer: we just pushed this today, so we actually still need to update
Excellent change control. Instead of using an admin panel on the site, we now
use GitHub to manage our static content. This means all changes are
tracked for free, we already have a process in place for reviewing pull
requests, and it's easy to roll back or manipulate the data because it's all
in the repository already.
If you want to get involved with Test Pilot, come join us in #testpilot (or
10 Oct 2016
For the third quarter of 2016 the Test Pilot team decided to try using the OKR method (an OKR overview) for
our goal setting.
We all sat down in London and hashed out what direction we wanted to move in for
Q3, what we thought we could do in that timeframe, prioritized the results
and then I published the results on the wiki. If you're interested in what
Test Pilot did in Q3 you should read that link because it has a bunch of
comments in it.
I knew we deprioritized some of our goals mid-quarter, but I was surprised to
see us come up with a pretty modest .61. My takeaways from my first time
using the OKR method is:
Wording is really important. Even if you all agree on some words while
sitting around a table, look them over again the next day because they might
not make as much sense as you think.
Getting the goals for your quarter planned before the quarter starts is tops.
Having a public list of goals you can point people to is great for your team,
other teams you work with, and anyone in the community interested in your
Estimates for how long things will take you is still a Really Hard Problem.
The feedback I've received about the OKR process we followed has been
really positive and I expect to continue it in the future.
28 Sep 2016
The Test Pilot team has been heads-down for months working on three new
experiments for Firefox and you can get them all today!
Min Vid is an add-on that allows you to shrink a video into a small
always-on-top frame in the corner of your browser. This lets you watch and
interact with a video while browsing the web in other tabs. Opera and Safari
are implementing similar features so this one might have some sticking power.
Thanks to Dave, Jen, and Jared for taking this from some prototype code to in
front of Firefox users in six months.
Luke has been working hard on Tracking Protection - an experiment
focused on collecting feedback from users about which sites break when Firefox
blocks the trackers from loading. As we collect data from everyday users we can
make decisions about how best to block what people don't want and still show
them what they do. Eventually this could lead to us protecting all Firefox
users with Tracking Protection by default.
Page Shot is a snappy experiment that enables users to quickly take
screenshots in their browser and share them on the internet. There are a few
companies in this space already, but their products always felt too heavy to me,
or they ignored privacy, or some simply didn't even work (this was on Linux).
Page Shot is light and quick and works great everywhere.
As a bonus, a feature I haven't seen anywhere else, Page Shot also offers
searching the text within the images themselves. So if you take a screenshot of
a pizza recipe and later search for "mozzarella" it will find the recipe.
I was late to the Page Shot party and my involvement is just standing on the
shoulders of giants at this point: by the time I was involved the final touches
were already being put on. A big thanks to Ian and Donovan for bringing
this project to life.
I called out the engineers who have been working to bring their creations to
life, but of course there are so many teams who were critical to today's
launches. A big thank you to the people who have been working tirelessly and
congratulations on launching your products! :)
16 Sep 2016
I installed Ubuntu 16.04.1 this week and decided to try out Unity, the default
window manager. After I installed Nightly I assumed it would be simple to get
the icon to stay in the dock, but Unity seemed confused about Nightly vs the
built-in Firefox (I assume because the executables have the same name).
It took some doing to get Nightly to stick to the Dock with its own icon. I
retraced my steps and wrote them down below.
My goal was to be able to run a couple versions of Firefox with several
profiles. I thought the easiest way to accomplish that would be to add a new
icon for each version+profile combination and a single left click on the
icon would run the profile I want.
After some research, I think the Unity way is to have a single icon for each
version of Firefox, and then add
Actions to it so you can right click on the
icon and launch a specific profile from there.
If you don't have Nighly yet, download Nightly (these steps should work fine with Aurora or Beta
also). Open a terminal:
$ mkdir /opt/firefox
$ tar -xvjf ~/Downloads/firefox-51.0a1.en-US.linux-x86_64.tar.bz2 /opt
You may need to
chown some directories to get that in
/opt which is fine.
At the end of the day, make sure your regular user can write to the directory or
else you won't be able to install Nightly's updates.
Adding the icon to the dock
Then create a file in your home directory named
nightly.desktop and paste this
Comment=Browse the World Wide Web
[Desktop Action Default]
Exec=/opt/firefox/firefox --no-remote -P minefield-default
[Desktop Action Mozilla]
Exec=/opt/firefox/firefox --no-remote -P minefield-mozilla
[Desktop Action ProfileManager]
Exec=/opt/firefox/firefox --no-remote --profile-manager
Adjust anything that looks like it should change, the main callout being the
Exec line should have the names of the profiles you want to use (in the above
file mine are called
minefield-mozilla). If you have
more profiles just make more copies of that section and name them appropriately.
If you think you've got it, run this command:
$ desktop-file-validate nightly.desktop
No output? Great -- it passed the validator. Now install it:
$ desktop-file-install --dir=.local/share/applications nightly.desktop
Two notes on this command:
- If you leave off --dir it will write to
affect all users of the computer. You'll probably need to
command if you want that.
- Something is weird with the parsing. Originally I passed in
--dir=~/.local/... and it literally made a directory named
~ in my home
directory, so, if the menu isn't updating, double check the file is getting
copied to the right spot.
Some people report having to run
unity again to get the change to appear, but
it showed up for me. Now left-clicking runs Nightly and right-clicking opens a
menu asking me which profile I want to use.
Modifying the Firefox Launcher
I also wanted to launch profiles off the regular Firefox icon in the same way.
The easiest way to do that is to copy the built-in one from
/usr/share/applications/firefox.desktop and modify it to suit you.
Conveniently, Unity will override a system-wide .desktop file if you have one
with the same name in your local directory so installing it with the same
commands as you did for Nightly will work fine.
I should probably add a disclaimer that I've used Unity for all of two days and there
may be a smoother way to do this. I saw a couple of 3rd-party programs that
will generate .desktop files but I didn't want to install more things I'd rarely
use. Please leave a comment if I'm way off on these instructions! :)