Live from Austin!

I upgraded to Apple iLife 09. My main motivation was to get the new version of iPhoto, which adds support for face recognition and GPS integration.

When I fired up the new iPhoto, I discovered that the GPS data for my images in JPEG format had been read in correctly; but that all of my Canon raw CR2 files had been ignored, even though the photos of Hamburg mostly had GPS data.

I investigated whether iPhoto was scriptable. If I could add the GPS coordinates to the existing photos via AppleScript, I could take the GPS track logs and build an importer. Unfortunately, though, there’s no AppleScript interface to the GPS metadata, according to iPhoto’s AppleScript dictionary.

Experimenting further, I discovered that if I rooted around inside the iPhoto library (using right click and "Show contents"), copied the CR2 files out, and imported them into iPhoto again, that the GPS data would be processed correctly.

There was a snag, though. If I did that, I would lose all the names and descriptions I had entered in iPhoto. This was a solvable problem, though: iPhoto remembers the original filename, and has AppleScript properties that let you read and write the description/comment and name/title of any image.

The next small snag was that I don’t know AppleScript. I’ve tried to learn it, but I find it very annoying and syntaxy. I considered giving it another go, but I did a few quick Google searches and determined that AppleScript lacks hashes, arrays, and most other useful data structures that you might use to store and retrieve a set of captions for a large number of photographs indexed by filename.

Ruby to the rescue. When Apple created AppleScript, they did it by layering it on top of a lower level system called OSA or Open Scripting Architecture. This is a language-independent API, which means that you can glue any scripting language to it, and instantly be able to do anything in that language that you can do in AppleScript. There are a couple of projects which link OSA to Ruby; I picked RubyOSA , though a brief exploration of rb-appscript (a port of a Python bridge, spit ) suggests that it would have done the job too.

A quick sudo su and gem install ruby-osa , and I was ready to script iPhoto using Ruby. I built two scripts. The first, iphotodump.rb , dumps out all the titles and comments for the selected photos, storing them in data files using Ruby’s standard marshalling (persistence) mechanism. The second script, iphotorestore.rb , reverses the process: it reads in the data files, then scans through the selected images and restores the appropriate metadata by matching up the filenames of the original image files.

I ran iphotodump to dump out all the metadata for my Hamburg photos, deleted them from iPhoto, re-imported the CR2 files, then ran iphotorestore to restore the metadata. Problem solved.

I imagine there are other people out there with similar iPhoto-related metadata problems, so hopefully my scripts can point the way at solutions. With iPhoto glued to Ruby, there are all kinds of possibilities–for example, you have the power to do a full regular expression search and replace, with Perl-like regular expression power. I think I’m also going to write something to append the photo’s keywords to the description, so I can then go on a keyword deleting spree but still be able to find photos using search.

Problem:

You are trying to run Ruby on Rails on OS X, and all you ever get is

Rails requires RubyGems >= 0.9.4. Please install RubyGems and try again: http://rubygems.rubyforge.org

When you check gem --version you find that you already have RubyGems 1.1.1, or some other version higher than 0.9.4 that Rails ought to be happy with.

Possible explanation:

At some point you installed MacPorts. MacPorts built and/or installed a redundant version of ruby, even though you didn’t ask it to.

To confirm that this is the problem, type which ruby and see if the answer has /opt in it. If so, yes, MacPorts hosed your Rails/RubyGems. Solution: sudo port uninstall ruby

(I’m pretty confident that MacPorts is at fault here because I installed it for the first time this evening to build bzr, which involves no Ruby, and indeed is from a bunch of Python programmers who would probably sooner drink raw sewage than require Ruby for anything. So thanks, MacPorts.)

WordPress 2.3 is out, with official tag support. I’ve just finished upgrading, and tags now work properly. I had to hack together some SQL + Ruby to convert everything, but it should all be done now.

Atom feeds and OpenID support should hopefully work as before; let me know if you notice anything strange. I’m going to test by replying to this…

Update: It works. And excitingly, you no longer have to hack code to get OpenID support working to and from LiveJournal.

I got tags working via a plugin.

Since I was messing with the site anyway, I hacked together some Ruby code to pull all the content out of the database and perform automatic keyword extraction via naïve bayesian analysis.

It spat out a file of SQL commands, consisting of the subject of each posting and the first line of text (in comments), followed by the commands to add the tags. I ran through the file in vim deleting here and adding there, then executed the result. So now pretty much everything should be tagged, right back to the start. How cool is that?

As you have probably noticed, I’ve just gone through a major software migration for my web site.

I was using typo. It was OK, but had a few problems. While its web site describes it as “lean”, that isn’t really the reality. It also relied on a combination of Apache, LigHTTPd and FastCGI that tended to break down without explanation.

The biggest reason for change, though, was that typo’s authors’ idea of what was important functionality was diverging from mine. The wakeup call was when someone spent a bunch of time replacing the regular page templates with templates written in HAML.

For those lucky enough not to know, HAML is a stupid and inexplicably trendy idea in the Rails community, comparable to LiveJournal’s S2 style system. Basically, instead of creating your page templates in HTML and CSS, which everyone can understand and for which there are a zillion useful tools, you instead write program code in a whole new language which has minimal documentation. The program then generates the HTML and CSS.

Of course, this destroys the entire point of template systems, which is to separate code from presentation and make the presentation layer editable by non-programmers using common tools.

I wouldn’t have minded the HAML idiocy so much if it wasn’t for the fact that typo still lacked support for things as basic as user authentication for commenting. So I looked at other web content management software… and looked… and looked.

I tried Blojsom. Supposedly it’s what Apple uses. If so, I hope they’ve done a lot of work on their version, as it’s a major PITA to set up, and very complicated even when you get it working.

In the end, though, I knew the main feature I wanted: OpenID support. Hence, I found myself reluctantly herded towards Wordpress, which has a working OpenID plugin. (Or at least, it works for my OpenID account when I test it. I don’t think it has XRI support, though.)

I did entertain the idea of writing my own CMS. I even sketched out some design notes. But it really is a solved problem, I just didn’t like the technologies used to solve it.

Let’s be blunt about this: I hate PHP, and I hate MySQL. PHP is the Visual BASIC of web programming languages, a mess which grew with no planning out of a quick hack, a kitchen sink language known for its amenability to security holes. MySQL is a toy database, popular because it’s fast, fast because by default it doesn’t actually provide the basic ACID functions a database is supposed to provide. (Sure, you can turn those on, but once you do, today’s PostgreSQL is faster under non-trivial load.)

But I don’t believe in religion, especially not when it comes to software. I’m a strict pragmatist–whatever it takes to get the job done, even if it may offend a few aesthetic sensibilities and fall far short of perfection.

I spend most of my time at work developing using IBM Lotus Notes and Domino. Every time Notes is mentioned on Slashdot, a bunch of people will rant about how bad its UI is. They miss the point utterly. Believe me, the poor UI of Notes is only the most glaringly obvious defect it has; there are far worse problems underneath that the average end user is blissfully unaware of. But you know what? It works. It is sufficient. It lets you build groupware applications and dynamic web sites with fine-grained security in days, not weeks. That is why people use it. The only other tool I’ve found which comes close is Ruby on Rails, and that’s still too immature for me to want to use it on production systems. (That, and it’s surrounded by a community of people who think things like HAML are a good idea.)

So, here we are. I’m editing this in a nice AJAX WYSIWYG editor with spelling checker (an idea shot down by the typo developers), and you should be able to log in with OpenID to comment (an idea the typo developers seem utterly uninterested in).

It took most of Saturday hacking with Ruby, PostgreSQL and MySQL, but I believe I’ve managed to transfer not just all my data, but all your comments too. I think I’ve even managed to keep all the permalinks the same, and preserve all the timestamps. I’ve temporarily lost the tags functionality, but should be able to get it back with another plugin. Hopefully Wordpress will prove more reliable than Typo, and hopefully the OpenID stuff will interoperate correctly with LiveJournal. If not, pray that I inexplicably become independently wealthy and have the time to write something that does the job properly.

We don’t need a Google Summer of Code; we need a Google Summer of Documentation.

Rails 1.1 broke typo. My web host suggested a fix to freeze the Rails version at 1.0, but that didn’t fix the problem. It did, however, break rake so I couldn’t un-freeze.

I decided to migrate to the trunk version of typo, which works with Rails 1.1. So I downloaded that using svn, and set it up with a clean database to reduce the number of possible sources of error. It didn’t work, but creating a new blank rails app and copying over the dispatch scripts and .htaccess file fixed it.

Next problem was to migrate my data. There’s a rake migrate command which is supposed to do this. Hey, guess what, it didn’t work! So, I ended up typing in raw SQL directly to psql to update the schema of the old database to something compatible with the new one.

That done, I dumped the old data with pg_sql --attribute-inserts, and imported it back into the new database.

After learning how to reset serial numbers, I was up and running again.

Last night I wrote my first program in Ruby. So far I like it, a lot.

I’d been intending to learn something better than Perl for a long time. Perl is very useful, and with CPAN you can get a great deal done in very little time. (Thanks, Jarkko—how’s it going?) However, it’s a really crufty and syntaxy language, and the way it supports object-oriented programming is so butt-ugly and riddled with pitfalls that I had never bothered to get to grips with it enough to write my own classes. People complain about regular expression code in Perl being write-only, but to my mind it’s the OO code that’s the real problem. Not just that it’s ugly; as the tutorial says, “Yes, this is a lot of work”. Using OO shouldn’t be a lot of work, that’s the whole point of it!

I think that the Perl motto of “There’s more than one way to do it” has become something of a convenient excuse for not making difficult decisions. I don’t need two or three different syntaxes for method invocation, particularly not if they’re all ugly. I don’t need a variety of different types of scoping, and any language where the simplest and cleanest scoping options still require a lengthy tutorial needs to be taken out and shot.

I read the Apocalypses. I wanted to know if Perl 6 was going to become the language I wanted, in much the same way that Perl 5 had become a tolerable version of Perl 4. It’s clear that Larry Wall understands how awful Perl’s OO system is, in that he adds a whole bunch of fundamental problems to my rather superficial ones. Unfortunately, the proposed new clean OO for Perl 6 looks to have a lot of the ugly and arbitrary syntax that is a familiar feature of Perl—like $.x for a public variable and $:y for a private one, *%_ for a variable list of arguments, all the way up to abominations like %::*::($package)::method. And still “There are too many ways to do it”, for example in regular expressions you have a whole new syntax, plus you can use m:p5/ and use all the old Perl 5 syntax. Or hey, why not use both?

And Perl 6 isn’t even out yet. Who knows what kinds of compromises will be made before it becomes real? And it’s clearly going to have a major learning curve for Perl 5 users, so much so that some people are grumpily saying they’ll stick with Perl 5. Me, I’m always ready to learn a new language—but if I’m going to spend a few months of my time learning Yet Another Programming Language, I’d like it to be one that’s (smaller|simpler|easier|more consistent) than the ones I know already, not one that needs a big wall chart to summarize its operator precedence rules.

Ruby isn’t perfect, of course. I wish it had support for variable declaration, with an equivalent of use strict. Unfortunately, the Ruby zealots seem to have an unjustified religious objection to allowing variable declaration, and I was bitten by the lack of declarations twice last night alone. Still, I was writing proper OO code with recursion inside a couple of hours, and that has to count for something.

The other two big problems with Ruby are performance, and Unicode. It kinda lacks both. However, once Parrot has matured a bit, Cardinal should solve both problems at once.