Skip to content

The sunset of getpersonas.com

Over four years ago getpersonas.com was built as a gallery to hold Personas – easy to make and use themes for Firefox. A lot of programs were supporting skins at the time, but I don’t know of any which were so simple (literally, move your mouse over an image on a webpage and it’s applied). A community of artists formed and many designs emerged – some amazing and popular, some focused on specific things (hat tip to CollieSmile), many for sports, or brands, or just people who wanted pictures of their family in a spot they noticed all day long.

Honestly, I thought they seemed gimmicky at first, but I started using them casually and they can be pretty fun. Every once in a while I’ll look up at my search box and see this guy and crack a smile and that’s enough reason to keep using them for me.

Now I’m treading down the dangerous path of naming names – a road full of mine shafts and sharp turns – and if I make a mistake, my apologies, please let me know or leave a comment. I wasn’t involved at the start of the Personas project but I think credit for the idea goes to Chris Beard, original add-on development to Myk Melez, original Firefox development work to the Mozilla Labs team, Toby Elliott and Ryan Doherty as the primary web developers, Stephen Donner and Krupa Raj for QA, and Jeremy Orem for IT Support. I know other teams worked on code here and there, but I think those were the primary contributors. I was focused on the engineering aspects so I’m not sure who to credit for the operational side of things – I know someone was managing day to day life at the time (it was Deb Richardson at some point, and today it’s Amy Tsay) and there were numerous volunteers reviewing submissions as they came in. It’s probably worth noting that in all the queues of all the sites at Mozilla the getpersonas.com queue was the only one I regularly saw completely finished and empty – kudos to those reviewers.

After seeing the rapid success of Personas we began talking about what we should do with them. The site didn’t have anyone adding features or doing maintenance on it – Ryan was the closest person to it and officially he was working on AMO. So, after some discussion we decided we would migrate all the data in getpersonas.com to AMO which would keep all of our themes and add-ons in one place. We finished the first steps (relatively) quickly – we wrote a questionable script which ran periodically and synced Personas listings from getpersonas.com to AMO. The files themselves and all the users remained on getpersonas.com. Looking purely at the number of times the script finished vs failed, it did pretty well, but there were times when it would fail constantly until someone could find time to fix it, much to the concern of the artists who didn’t see new data showing up on AMO.

Meanwhile, our security team decided to add getpersonas.com to the bug bounty program which surprised everyone and which we’ve playfully chided them about ever since. Some of our oldest and hacked together code (remember, this was originally an experiment) was now coming under scrutiny from security professionals and bored/gifted students from around the world. Special thanks to Matthew Noorenberghe and Brandon Savage for writing patches to fight back the flood of incoming security bugs.

A final spur to getting the migration finished came in the form of Mozilla Persona – an aptly named identity system, also from our company. After some spirited debate it was decided the identity project would use the Persona name and the personas on getpersonas.com would be referred to as Themes (and the existing themes we had as Complete Themes). With all the logistics out of the way we rapidly adopted the new names everywhere save getpersonas.com which continued to cling to life.

Late last year Chris Van Wiemeersch spearheaded a clandestine operation to finally finish closing down getpersonas.com (with special mention to Kevin Ngo for contributing piles of code all the while finishing his final exams, and Kris Maglione for updating the Personas Plus add-on).

After another quarter of late nights balancing the migration with developing the Firefox Marketplace it’s finally happened: I’m happy to announce that getpersonas.com has now officially retired and all the content and artist accounts have been migrated to AMO.

It’s exciting from a Mozilla point of view because the confusion of projects can finally end and we can stop pretending to maintain a site that we haven’t actually done anything with for years. The artists and users have reason to be happy also, though – moving to AMO means it’s on an actively maintained site. Improvements can happen, features can be added, and new ideas can be explored. Finishing this long awaited migration will breathe new life into the themes community and I’m looking forward to all the fun new ways we’ll see Perso-uhh – I mean – themes, being used.

A screenshot of new users on AMO on the night we imported the accounts. Our usual numbers of 1500/day are eclipsed by 1.5 million that night. Also a testament to the site’s scalability that it didn’t miss a beat.

Tagged , , ,

Inception: projects within projects

When we converted addons.mozilla.org from PHP to Python I mentioned how deceptively large (lines of code) the site had grown with so many views and features. We’ve since built marketplace.firefox.com on literally the same code base (a decision I hoped to blog about some day, alas) and the line count continues upwards. The MVC paradigm is one of the most common out there, easy to understand, and built in to our frameworks so it was naturally a good choice. However, as the site continues to expand we’ve had to review what the best architecture for the code would be and the main point that stands out is that our entire site and its services are a monolithic blob of code. Both sites, all of our APIs, all of the PayPal code, the administrative tools, the developer control panel, the reviewer queues, the list goes on – it’s all in one repository and changing one means potentially affecting all of them.

With big plans for 2013 and no signs of slowing our code expansion we’ve hit the point where it makes more sense to break the site up into smaller projects. We’ve already started with our payment processor and metrics processor and more are soon to follow. These projects will be tied together around a central API on the back end. We’ve had partial front end APIs for years (like search) which we will expand to make the entire site accessible via API. At that point our front end code will be completely separated and can be optimized to speak directly to JSON APIs instead of pulling HTML with every request (something which the developers have been demoing the past couple weeks at our show and tells). A rough diagram is below, subject to change, of course.

Marketplace Architecture Diagram

Some benefits of this approach include:

  • Physical separation of code which will easily allow us separate SLAs or even separate conformance agreements, like PCI compliance
  • Speaking to APIs means we’ll have specific failure cases we can write code for so when a piece of the site dies the entire site doesn’t crash
  • Each project can be written to scale horizontally without affecting the other projects
  • Developers and contributors can focus on a single project without having to check out the entire site (and deal with substantial configuration maintenance)
  • Localizations will be broken up into separate .po files, a long requested feature from our localizing friends
  • Deployments can happen for specific projects without updating all of them. These will be faster, easier to scope, and easier to roll back if necessary
  • 3rd parties can build full featured sites off our APIs

There are a couple of downsides: maintenance/configuration of each project and integration testing. The former is diminished a bit because not every developer works on every project so with some prudent mocking and a few virtual machines there should be an easy flow there. The second is something we currently neglect but will become critical with these changes so expect some solid developments there. With all the other work on our schedule I think we’ll be touching parts of this project in every quarter this year. If you hear the codename INCEPTION it’s referring to this project.

Tagged , , , , ,

RSVP

A man boarding a train

The situation had soured. Tony Six and his goons had known I was coming for four hours which means anyone that owed him a favor or had a score to settle with me was already on their way and settling in to wait. It wasn’t the kind of party I liked to crash but you don’t always get to choose the venue. I had one ace they wouldn’t be expecting. They didn’t know I’d have a train.

Tagged ,

Mozilla t-shirt time capsule

I moved around a lot over the past year and ended up putting a bunch of old t-shirts into a trunk and forgetting about them. My one-o-clock-in-the-morning brain decided now was a good time to go through the trunk and get rid of all those old shirts I never wear. It turns out we Mozillians used to really like making t-shirts. Below is a pile of shirts I found from around 2005-2011 or so. How many can you identify?

Mozilla Shirts

Tagged

A successful first FLOSSHack

A few months ago Tim Morgan emailed the Portland OWASP chapter and suggested that we organize a meeting where everyone could get together and audit some existing software. When vulnerabilities were found we would follow the responsible disclosure life cycle and notify the maintainers before publicly disclosing. It would be a fun way to spend an afternoon, people would get experience with responsible hacking, and the software maintainers would have the opportunity improve their software. The idea gained traction and FLOSSHack One was born.

I coordinated with Ushahidi, an open source mapping application, to be the subject of the audit. They were very helpful, even participating during our meeting and answering questions about how their application was built. We had about ten people show up on a Sunday afternoon and everyone got to work in a supportive environment for penetration testing. By working with the developers we were able to cover a lot of the code and came up with several vulnerabilities to report which will be announced on security.ushahidi.com as appropriate.

Thanks to Tim for organizing the FLOSSHack, Ushahidi for supporting us on their end, and thanks to everyone who participated. If you’re interested in doing your own, Tim wrote this generic wiki page which should help.

Tagged , , ,

Adding a debug language to ȧḓḓ-ǿƞş.ḿǿzīŀŀȧ.ǿřɠ

Last week Greg Koberger finally got me to cross “add a test locale to AMO” off my list – and it turns out it only took a few minutes of actual coding. It sounds like others have had some troubles so I wanted to run through what I did.

Firstly, you can see what I’m talking about at the AMO dev site. I’m using a script in the Translate Toolkit called podebug to generate a .po file filled with unicode characters. You can also generate one which surrounds the strings with “xxx” or blanks them out, etc. The goal is to easily see what strings on your website can’t be localized (this means they aren’t in the .po files for localizers).

If you’re using product-details to get locale names make sure you’re using the latest revision – I added dbg to the array so it will show up in language dropdowns like all the other languages. After that I added dbg to Django’s LANGUAGES define[1]. At that point the language showed up, but we’d get tracebacks if we tried to format money in a string since ‘dbg’ was a fake language. AMO uses Babel for details about each locale so I just made a wrapper for the Babel Locale() function and told it to use ‘en’ if it saw ‘dbg’. This way ‘dbg’ is used for all the strings on the site but for any complex formatting we fall back to en. This works since dbg is based off en anyway, as far as pluralizations are concerned. You can see my commit which added that function and made sure we started using it here.

This also works on the marketplace.

[1] I actually added it to AMO_LANGUAGES which gets to LANGUAGES in a slightly round-about way. See our settings.py file for how that flow works.

Tagged , , , , ,

How to get a development instance of AMO set up in about 10 minutes

Last year we set up landfill.amo to give contributors an easy base to set up the addons.mozilla.org site. Easy is relative here, of course, but it was a big leap over what we had at the time.

Kumar leapfrogged that milestone by adding Vagrant configuration scripts to our repository. Now you can have a running version of AMO on your local system in about 3-5 commands[1].

Check out the steps to install AMO with Vagrant to see how. I set it up on OS X last week and aside from waiting for the download it only took a few minutes. Drop by #amo on IRC if you run into any troubles.

[1] Setting this up on Windows is apparently more difficult although one contributor did find success after fighting with it for some time.

Edit: Updated the installation instructions URL

Tagged , , , ,

10 years of Irssi use and I switched to WeeChat last weekend

I started using Irssi almost 10 years ago when I first started trolling wandering around the world of IRC. My main use is to run it in a screen and stay connected all the time. To chat I’ll just ssh into the server and reconnect to the screen. Generally I leave my terminal open, stretched across the top of my monitor so it’s really wide but only about 8 or 10 lines high. This way I can keep IRC visible all day long and respond quickly to questions.

I’ve customized irssi with custom highlights and commands and all has been well, save one thing: bug 310 – vertical splits. Irssi can’t do vertical window splits and with the trends giving us widescreen monitors, horizontal splits aren’t really useful to me. The bug has been open since 2005 and I’ve all but given up on it.

Then the other day Chris McDonald claimed the unfortunately named WeeChat was superior to Irssi. I was all ready to defend Irssi’s honor but I read through the documentation and WeeChat was pretty compelling – most notably its support for vertical splits!

So, long story short, I switched last week and it’s awesome. I don’t miss Irssi at all and in fact WeeChat offers me things I never even knew I’d want (like per-buffer history when hitting the up-arrow). I’ve customized it so it essentially looks like Irssi (no nickname list, etc.) and I’ve added a lot of aliases to make it easier for me to use (mostly vim keys). If you’re an Irssi user you owe it to yourself to at least read their docs and see what you think.

Tagged

PHP is dead! (on addons.mozilla.org)

This is just a short note to recognize the long coming milestone of PHP being effectively off[1] on addons.mozilla.org. We started the migration in 2010 and just finished it up a couple weeks ago. After the major pages were completed it was hard to budget time for all the minor details we had implemented since there was so much other important stuff to do (I’m looking at you, marketplace.mozilla.org). Now that the switch is done though we can simplify our setup instructions for AMO, simplify our infrastructure, optimize apache for python only, have full unit test coverage – the list goes on.

A big thanks to all the developers who made the switch possible, and especially the ones at the end who were working on migrating PHP scripts instead of more glorious projects. Thanks to the management and all the other folks affected by the switch for being patient with the scheduling. Thanks to the localizers for dealing with the crazy merged .po files, and, of course, thanks to the users of AMO for reporting bugs when they happened and generally being an all around great community to work with.

[1] Currently all traffic is being rewritten to a WSGI handler for Python. PHP is still on the server but nothing uses it. We’ll be removing it completely in the near future.

Tagged , , , , , , ,

Choosing your own greener grass

A lot of my time is spent trying to arrange projects and schedules so we can get code shipped in a reasonable time. AMO has the blessing/curse of being broad enough that there is work to do in nearly every area on the site. Once the highest priority areas have people working on them there is still plenty of work to go around that isn’t as time critical.

Last March there was some feedback from developers about being tired of the code they were working on and eyeing other technologies but not having the time to work with them. This is a fragile balance which every manager and developer has had to struggle with. Writing new code is always sexier than maintaining old code, but the old code is the bread and butter that keeps you in business. Finding a happy medium is a noble, elusive, and unfortunately, shifting goal.

Enjoying what you work on is crucial to being happy and productive though so I’ll continue to pursue that goal.

One of the changes we experimented with at the time was to let developers choose their own focus for the quarter. We still had high level quarterly goals that needed to be done, but that left plenty of other time throughout the quarter to work on any of those areas which were all equally important but perhaps not equally as interesting. I filled up a whiteboard with ideas one morning (and left space for developers to add more) and we had a meeting later in the day where anyone could discuss the ideas and sign up for what was interesting to them. This was a short term commitment from the developers and they got to work on what they cared about. From a managerial perspective, it increased motivation but didn’t sacrifice accountability. Since I knew the focus of the developers early in the quarter, I could help clear out roadblocks that they’d meet before they even started working on their areas.

The quarter is over this week and overall the idea was a success. All the feedback from developers was positive – if you’re looking for a way to spice up the top-down approach to goal setting this was effective. It also turned the goal setting into more of a discussion rather than an edict that just shows up.

I wrote this almost 9 months ago but apparently never hit publish. So, here it is. We’ve played with it a bit since but haven’t had a large quarterly planning meeting like that because we’re mainly focused on the marketplace. Experimenting with smaller goals is a next step here, and doing the meetings on a weekly or biweekly basis.

Tagged , ,