Welcome to the Landfill

Anyone who has tried to set up AMO knows it's no walk in the park even with the respectable amount of documentation. There are two big stumbling blocks: the database is large and complex, and a portion of the site functionality is still in PHP. Django's syncdb can make a database, but the relationships in the data is what's hard and trying to load fixtures from the test cases is an exercise in frustration since they may or may not all combine into a useful set of data. With the launch of landfill.amo[1] we bypass the entire headache. The site started with a clean database and I uploaded an add-on to show it worked, but otherwise it's empty. It's compact, fast, and simple to use. The beauty of the site for volunteers and casual developers is that the database and the filesystem are available in their entirety to download. This means you can check out the code, fill in the configuration, import the landfill database and have the site 90% running.[2] Perhaps a testament to the obscene number of open bugs for AMO right now, but this also solves a second long standing problem where localizers couldn't see the entire site. On landfill, anyone can be an administrator, an editor, or any other permission level they'd like; and they'll be able to see the entire site. If you've been overwhelmed or frustrated trying to set up AMO in the past, now is a good time to give it another shot. The landfill should just get better with age and use - if a few people register and add some data the available database dumps will get richer. If there is a part of the site that isn't working and you need it to be, let me know. Keep in mind this is only the new Python code, so the few parts that are still on PHP (like the admin panel) won't be available until they are ported. Code is updated near-instantly on commit, localization changes are updated every 5 minutes. [1] Forgive the fake certificate. This is a sandbox for developers, y'all know what you're doing. :) [2] Honestly, 90% is really all you need. We do a lot of stuff for scalability, statistics, etc. and unless you're actually working on that part of the site, you don't need those elements running. Of course, you're more than welcome to turn them on, I'm just trying to make it easy.

High level perspective on the switch from PHP to Python

It may be fatuous to write this post before we’ve actually finished the transition from PHP to Python, but I started writing a different post and this is what came out. Sometimes that happens.

Status Watch: An add-on for noticing HTTP error codes

Often on complex pages with many assets it can be easy to overlook assets which don't load. Usually they are minor JS, CSS, or tracking pixels which aren't noticed until you've spent way too long trying to track down the problem (or a month later you log into your stats dashboard to discover you haven't been collecting stats). With the launch of the new Add-on Builder (still an alpha product, but usable) I decided to make my first Jetpack to fix this. In only about 20 lines of code I was able to look for any 4xx or 5xx errors in the HTTP traffic and show a brief notification to the user about what went wrong and where. The builder was a great development experience (lack of documentation aside) and was a breeze to do something relatively complex. If you've wanted a similar add-on, feel free to use Status Watch. It's a Jetpack so you'll need Firefox 4, but on the bright side, you won't even need to restart the browser. Just click and go. I've had some requests to support a whitelist of sites so you don't get notifications all over the web and I'll add that when I get time. It's surprising how many sites have 403s and 404s though. Update 2011-02-17: Version 2 is on AMO now which supports a whitelist. See the AMO page for details.

A Night in the Emergency Department

Within minutes of my arrival at the Emergency Department a call comes in that an ambulance will arrive shortly transporting a man in cardiac arrest.  Orientation can wait.  Over the next 20 minutes he is given a regiment of drugs.  I follow him to a unit that will try to locate and destroy the clot in his heart.  In the next hour his heart stops four times while technicians put two femoral catheters in his legs and follow a dye through his blood stream.  Eventually they finish what they can do and ship him to the Cardiac Care Unit.  No one knows about permanent damage.  On my way back to the Emergency Department I pass a frantic looking woman with a cell phone.  She’s just spied her teenage daughter running in and cries, “they say it’s his heart and it’s serious.”  I don’t make eye contact.

md5verify: A script to automatically verify file integrity

I have a lot of files on my computer. Email archives, personal documents, stuff for work, photos I've taken...the list goes on - I'm sure most people reading this are in a similar boat. On occasion I've found some files to be missing or corrupt which is disturbing but is probably something to be expected. The bad part is, I keep backups, but I rotate them out when they reach a certain age which means if I don't notice a file is corrupt or missing I'll eventually lose it forever. I stayed up late a few nights ago and wrote a script to raise an alert when something has changed. On its first run the script will recursively walk a directory tree hashing each file and storing the hashes in the directory (in an md5sum compatible formatted file). On subsequent runs it will begin tracking new files automatically but it will also print messages for missing and changed files. By saving the checksums in each directory it becomes portable - you can copy a directory somewhere else and still be able to verify nothing changed (a quick md5sum -c checksums.txt will let you know). By default the script only prints messages when it sees something fishy so it's perfect to drop into cron and it uses exit statuses so it'll work for nagios too. I've been running it for a few months and have found a couple files that have changed - nothing critical yet but it's nice to know it's there.