When I get mail from Bugzilla it’s not obvious which project it’s from. My email client (GMail) only shows the “from name” so all I see for these projects is:

Mozilla: bugzilla-daemon
Pootle: bugzilla-daemon
Miro: bugzilla
kernel.org: bugme-daemon
Apache: bugzilla

Wouldn’t it make sense to differentiate each projects’ emails in the from name? Maybe even by default (something like “%SITE_NAME% Bugzilla”)?

Reed says it’s a personal problem because his mail client shows the full address. Am I the only one?

When I started the planning for branching Pootle I was very focused on scalability (or lack thereof) and most of my initial goals were to improve that including replacing flat files with mysql, creating a cacheable URL structure, etc. In hindsight, I should have realized that this project wasn’t going to be getting nearly the traffic load some of our other sites were getting and my priorities were out of order. What I should have been thinking about was usability and interface improvements. Due to my lack of foresight the project launched with enhancements in both areas but I think the time we spent on scalability was premature and the user interface suffered.

Whether it’s writing more comments than code or making sure meetings have agendas I’m a huge fan of communication. When branching a project, particularly when there are plans to merge the branch back into trunk, communication is vital. I think our meetings are productive but communication on a smaller scale is still a struggle. Both Pootle and Verbatim ended up writing the same code in a few cases which could have easily been avoided. In this particular case the timezones can make it difficult to synchronize but it’s something I’ll work at more in the future.

Something we have done a good job with is making a schedule and updating it with new developments. I really want to expand our effort here though. I think one of the difficulties of someone joining a project like this is direction; what are the goals of the project and how are we getting there? Several of us have talked about it on IRC and we all have a good general idea but for someone that isn’t as involved it’s hard to follow. Once we get over the next big hump (replacing jToolkit) I think this will begin to fall into place with smaller bugs/features revealing themselves and providing a way for volunteers to get footholds on the project as a whole.

Lastly, it might be obvious, but if you’re planning on maintaining the branch or merging back to trunk make sure you get along with the lead developers. I’m fortunate to work with the Pootle developers who clearly care deeply about the project. From talking to them it’s obvious they have the end users’ best interests in mind and are excited that we can all work together to improve the end product. And really, that’s what open source is all about and it’s great to be a part of it.

Why the disconnect? Did we suddenly have a budget for planning a website beyond the back of a napkin? Did graphic designers figure out how to translate their pen and paper skills onto the web? Did the web slowly evolve into something that could provide a canvas for more than just plain text and drawing boxes with ASCII? Well, yeah – all the above. But what’s on my mind right now is the mental attitude – what I see as the mindset of the 90’s.

I suspect the first kind of site is purely legacy and only exists because of habit. It’s maintained out of habit, it has a budget out of habit, and people visit out of habit. If the site launched today with zero users it would probably remain that way until it was retired as a failure. These are not pleasing sites for either the current users or the new visitors. The only things these sites have going for them are momentum and division.

Their momentum is driven by recognition – the sites have been around so long people either know about them or they show up first in search results. This is a valuable position to be in but it’s not permanent and without proper maintenance will change.

The other leg of their shaky foundation, how they retain users, is the division. They separate themselves from the crowd by giving their users just enough reason not to leave. Often that reason is that people have already invested so much time, money, and energy putting information into the site that they don’t want to leave even if another site is substantially better. Not only does this hurt the user, it hurts the web (and thus, all of us). When people continue to use mediocre sites it continues to send the message that it’s OK to not improve and to not add value.

The second style of sites – the breath of fresh air style – is the result of a newer way of thinking. I’m not just talking about colors that don’t hurt your eyes. I’m talking about a fundamental shift of viewpoints. Integrate with other sites? Sure. Provide useful content in forms that other sites can consume? You bet. Publish API’s that let people do things you haven’t thought of? Let users export their data and do what they want with it? Share your content with a permissive license? Hell yeah.

This is what we should be demanding as users of the web. Give us accessible content. Give us thought-out workflow. Give us options. Give us value, and we’ll choose you.

After my positive experience with python I was gearing up for a script to do some repository analysis when I ran across MPY SVN Stats. After a fast download and a one line command I had charts and tables full of info. So, here’s some late night stat work for everyone:

We had a total of 103 people that committed code directly to SVN, 69 of which had 10 or more commits. The top 25 committers by total numbers of commits are:

No    Author                                Commits    Percentage
1  	wclouser#mozilla.com                1558       15.58%
2 	fligtar#gmail.com                   739        7.39%
3 	steven#silverorange.com             707        7.07%
4 	reed#reedloden.com                  639        6.39%
5 	pascal.chevrel#mozilla-europe.org   567        5.67%
6 	fwenzel#mozilla.com                 551        5.51%
7 	nelson#wordmaster.org               524        5.24%
8 	mkaply#us.ibm.com                   481        4.81%
9 	paul#glaxstar.com                   392        3.92%
10 	mgalli#mgalli.com                   364        3.64%
11 	morgamic#mozilla.com                311        3.11%
12 	dougt#meer.net                      307        3.07%
13 	michael.koch#enough.de              179        1.79%
14 	ahajdukewycz#mozilla.com            173        1.73%
15 	shaver#mozilla.com                  162        1.62%
16 	reed#mozilla.com                    158        1.58%
17 	abuchanan#mozilla.com               112        1.12%
18 	dougt#mozilla.com                   105        1.05%
19 	eshepherd#mozilla.com               101        1.01%
20 	erik#raincitystudios.com            97         0.97%
21 	mfinkle#mozilla.com                 89         0.89%
22 	robert#accettura.com                89         0.89%
23 	smalolepszy#aviary.pl               82         0.82%
24 	oremj#mozilla.com                   76         0.76%
25 	tim.babych#gmail.com                58         0.58%

My numbers here (wclouser#mozilla.com) are inflated because I did a lot of the branching/tagging on our projects. If you throw my number out as an anomaly you’ll see that no single person has committed more than 7.5% of the code in SVN. That’s a great community hard at work!

Thanks for all your help!

Whether you can attend in person or just visit online it should be able to offer something for anyone interested in the open web. If you’ve got some free time at the end of this week, get involved.

UTF-8 is your friend. Use it.

The big step from ASCII to Unicode was the potential to use multiple bytes to represent a single character. With ASCII, each character was given a number between 0 and 255 and that’s all the programmer could use. If another character needed to be shown, the numbers were reused and a different font was loaded. If people didn’t have the same fonts, they got errors or undefined results.

Enter Unicode and UTF-8. With the creation of UTF-8, characters from all over the world are assigned numbers between 0 and 1,114,111. This is fantastic news, because you can store text in many different languages without having to worry about specific encodings. This also means you can support language fall back for sections of your web page. If you have 80% of your page translated for a specific language, you can fall back to an alternative, all while using the same encoding. (an aside: Be sure to use lang=”" attributes in your HTML tags if you’re changing the language mid-stream).

Don’t concatenate strings

Code like this makes me sad, and will make your localizers cry (or quit):

$item = "toast";

// Example one - This is bad
echo _("Sometimes I eat")." {$item} "._("and sometimes I don't.");

// Example two - This is better
echo sprintf(_("Sometimes I eat %s and sometimes I don't."), $item);

In the first example, chances are good the localizer will get the list of strings to translate and the two separate calls to _() will look like two different sentences with no context around them. Firstly, the phrases by themselves make no sense, and secondly, a localizer needs to be able to look at an entire sentence (and sometimes more) to understand how to translate it most effectively.

The second example uses the printf() standard %s to let the localizer know you’ll be substituting a string into the middle of the sentence. This is the current best practice for creating sentences with variables. Depending on what the string is, they may still be upset, but that’s out of scope for this tip (here’s a hint though).

Don’t use machine translation

In recent years great progress has been made towards programmatically translating documents from language to language. That said, it is far from being an acceptable replacement for a fluent translator. The edge cases and “what ifs” on the technical/logical side of the translation are enough for me to say that, but when you start talking about potentially offensive translations (that’s the next tip) this is a definite requirement. Just look at an example of an automated German to English translation. It’s readable but it’s far from polished – not something you want as a first impression to your site.

Be culturally sensitive

If you’re not very familiar with your target culture ask for an opinion from someone who is (or hire a localizer who is). Seemingly innocent words, phrases, and images could be misunderstood by another culture. If you use terminology that is only understood in your region or culture, the best case you can hope for is that a visitor to your site just won’t understand and will ignore it, but it really reduces your credibility and the overall enjoyment of visiting your site.

Use multi-byte functions

This may be a little PHP specific, but it’s good to be aware of it in any language. PHP has string functions and multibyte string functions. The latter functions support characters that fill up more than one byte (ie. UTF-8 characters). This is essential when manipulating strings with letters outside of the Latin alphabet. If you’re not using PHP, at the least, verify your programming language will manipulate multi-byte strings correctly.

Separate your views from your logic

I’m a fan of MVC separation, but there are plenty of other architectural patterns. Depending on what process and software you use for localization you may be giving template files to localizers. If that’s the case, the simpler the better – you don’t need a bunch of complex code around the strings waiting to be translated. Even if you’re using a method that doesn’t require giving template files to localizers, updating strings is easier, and whoever does maintenance on your software in the future will thank you.

Use (meaningful) placeholder text

This one might be a little controversial and is gettext specific. The documented and recommended way to use gettext is to pass an English string to the gettext() function. This serves two purposes: It lets the localizer see the complete English string when they are translating, and it let’s gettext fall back to English if a translation isn’t available.

I’m suggesting using a substitute string in place of the English string. For example, instead of _(“Error: Your cart is full!”) I would use _(“error_cart_full”). English translations are done in the .po file, just like every other locale. By following this rule, it’s possible to change the English text, without affecting the other translations. Using the documented method means that even adding a comma means changing every locale’s .po file and then recompiling them all. If you’ve got localizers watching for changes on their files (through a shared repository) this means they have to check and verify any changes – it’s a hassle and it’s time consuming for everyone involved.

The first purpose I mentioned, seeing English strings, can be duplicated by running `msgattrib –set-fuzzy $file1 | msgmerge -NUs $file2` where $file1 is the updated en-US .po file, and $file2 is the outdated .po file from another locale. This will merge the English strings into the other locale, but will mark them as fuzzy, so gettext will ignore them until they are translated.

The second purpose can be addressed just by making sure the strings you’re trying to use are available. If you need to use a new English string on the site, and the localizer is unavailable, you can temporarily move the fall back logic into your code:

// This is a temporary fix!
if (_("string_to_translate") == "string_to_translate") {
  // Print the English string
} else {
  // Print the translated string
}

While we’re on the subject of .po files, useful comments should be added to the file wherever appropriate to help provide context and hints for localizers.

Be aware of word length

Words in different languages have different lengths – words in Asian languages generally have fewer characters than English, and German words, more. When designing the layout for your site, bear this in mind. Don’t hard code widths to elements holding text – the words should be able to flow and expand as necessary. This can be tough with today’s complex sites, but CSS will go a long way to help. Also, when accepting user input, don’t put unneeded arbitrary length restrictions on the input.

Don’t use graphics as text

This is just a good idea in general, but it makes even more sense when localizing pages. Creating images is time consuming and has more potential for error. Using an appropriate encoding and employing CSS should get close to the same effect (with an extra point for accessibility). If you need to use an image, be prepared to accept localized strings and make the image yourself – localizers may not have the time, skills, or software they need to create the images.

Be aware of how changing the locale can affect strings

Setting the LC_ALL variable doesn’t just change the formatting of strings – it also changes currency formatting, time/date formatting, how things are sorted, and what symbols represent numbers/lettters/etc. Some Examples:

setlocale(LC_ALL, 'fr_FR');
$num = 1.5;
var_dump($num); // Prints 1.5
echo $num; // Prints 1,5

Internally, the decimal is represented by a period, and all the php functions will recognize that (eg. /[0-9.]+/ matches, whereas /[0-9,]+/ does not). However, if you need to print the string to pass it to another library or page (into a mysql query, passing to javascript, etc.) it’s going to become a comma. Another example:

preg_match('/\w/', 'ホーム'); // Will never match, regardless of LC_ALL

Using regular expressions on UTF-8 data can be risky. The \w and [[:alpha:]] character escapes only ever match single byte values (ie. characters with values up to 256) with the preg functions. The PCRE Documentation says:

“This remains true even when PCRE includes Unicode property support, because to do otherwise would slow down PCRE in many common cases. If you really want to test for a wider sense of, say, “digit”, you must use Unicode property tests such as \p{Nd}.”

If we need to match UTF-8 strings with regular expressions in PHP, we can use:

mb_regex_encoding('UTF-8');
mb_ereg('\w+', 'ホーム', $match);
print_r($match); // Prints: Array ( [0] => ホーム  )

By setting the internal regular expression encoding to UTF-8, and using the mb_ereg() function, we can match multibyte characters with regular expressions. Realize though, that this has the performance issues the PCRE documentation mentioned.