Unicode In A NutShell

Here is a great article about Unicode and how it affects Web developers and programmers. Here’s an excerpt:

The Single Most Important Fact About Encodings

If you completely forget everything I just explained, please remember one extremely important fact. It does not make sense to have a string without knowing what encoding it uses. You can no longer stick your head in the sand and pretend that “plain” text is ASCII.

There Ain’t No Such Thing As Plain Text.

If you have a string, in memory, in a file, or in an email message, you have to know what encoding it is in or you cannot interpret it or display it to users correctly.

Almost every stupid “my website looks like gibberish” or “she can’t read my emails when I use accents” problem comes down to one naive programmer who didn’t understand the simple fact that if you don’t tell me whether a particular string is encoded using UTF-8 or ASCII or ISO 8859-1 (Latin 1) or Windows 1252 (Western European), you simply cannot display it correctly or even figure out where it ends. There are over a hundred encodings and above code point 127, all bets are off.

For the article, go to: http://www.joelonsoftware.com/articles/Unicode.html

Profiles in Unicode

The New York Times has a great profile of Michael Everson, one of the many architects of Unicode. I hope some day there are more profiles like this about the many people devoting great chunks of their lives to the Unicode cause, people like Mark Davis, Asmus Freytag, and John Jenkins (to name just a few).

The article did a good job of describing Unicode — something I find extremely hard to describe:

A more technical explanation of Unicode is this: When Mr. Everson sends e-mail in ogham, his computer isn’t sending ogham letters through the ether. Instead, strings of 0’s and 1’s are transmitted, and when they arrive on a friend’s computer, they generate on its screen the same ogham letters that Mr. Everson typed. Unicode is the master list that resides in both computers and translates individual letters and symbols into strings of 0’s and 1’s and back again. Most current software is Unicode-compliant, which means that this master list of all the world’s writing systems has been built into operating systems, browsers and software.

Even though Unicode includes more than 50 different writing systems. it is far complete. I was surprised to learn that there are nearly 100 more writing systems left to be included, which means that we will likely not be around to see Unicode completed. I guess it’s kinda like Boston’s Big Dig.

Want to learn more about Unicode? Visit the Web site.