Burning inside: CD-R and archiving data

Some people may wonder why my web site was left unchanged for over a year. Well, I’m engaged in a lengthy project to digitize my entire photo collection, using a Nikon film scanner to produce 3000×2000 scans direct from the negatives.

Some of the images are decades old, and often the film has deteriorated and needs careful restoration. Color film in the 70s really wasn’t very stable, and these negatives haven’t been particularly well cared for either. My plan is to scan them, fix them, and archive them onto digital media.

Of course, this requires some care—five years ago, it might have seemed like a sensible idea to archive onto Syquest cartridges, after all. Who’s to say what will be around in another decade? Will we even be able to read most of today’s file formats? (How many art programs read NeoPaint files?)

A lot of people use TIFF. Few of them realize it, but TIFF is a really ugly file format originated by Microsoft. I say it’s ugly because I’ve read the specification. It has a zillion variations, including different byte ordering on different platforms. I’ve seen graphics packages which both claim to read and write TIFF, but won’t read each other’s files. So for archiving, TIFF is a definite no-no.

PNG is an open standard, it’s lossless, and it gets better compression than practically every comparable format, including TIFF. Because it uses no patented algorithms, it’s likely that every graphics program will at least have code to read it. Because Open Source implementations of the algorithm are available, I know that if the worst comes to the worst I can always write my own program to read PNG and write it into whatever’s the appropriate format in ten years’ time. So it’s PNG for me.

Anyway, after months of work my hard drive was getting dangerously full, so this weekend I bought a CD burner. Of all the data storage media out there, I think CD is the one most likely to still be readable in a couple of decades. I’m planning on using the Kodak pro-grade gold CD-Rs, which have a rated life of 100 years.

CD is a bit of a bitch to use, however. You have to burn the discs, verify them afterwards, and so on. On PCs, this generally involves a lot of dicking around with flaky driver software; the ThinkPad at work refuses to boot if the CD burner is plugged in, so you have to boot first, then plug and pray, and about half the time it’ll then recognize the drive. Assuming that worked, you can then try and burn a disc, which works about 80% of the time. The rest of the time the CD burning software hangs while updating the catalog at the end of the burn, and you have another coaster.

I was determined not to have similar experiences at home. Of course, I have a Mac at home, so that was a good start. Then I picked out a CD burner which was Firewire, so (a) I wouldn’t have any buffer underrun problems, and (b) I wouldn’t have to dick around with SCSI or USB drivers and termination problems.

Next, I narrowed my selection down to CD burners which were approved by Adaptec (who now want to be called Roxio), who make the Toast software used by practically everyone who burns CDs for a living.

Finally, I picked a drive which had the latest BurnProof technology. This is a hardware feature where if the drive stops receiving data fast enough—say, because Internet Explorer chokes while you’re browsing the web—the laser stops in a controlled fashion, marking how far it had got so it can continue when the data flow resumes. Which means fewer coasters, and the option of burning CDs while doing something else.

That’s the theory. Of course, no matter how careful and prepared you are, the universe has a way of screwing you over. In this case, I managed to get a faulty CD burner, and wasted most of yesterday trying to coax it into working properly. It would act just like it was working, but the CD would never verify and would be full of random (but sonically interesting) flipped bits.

Fortunately, I foresaw even this eventuality. Rather than trying to save $50 by buying online, I had decided to slum it and buy from CompUSA. So instead of paying two sets of shipping charges and waiting several days for a replacement, I picked up another burner this morning. The new one works fine. Rips at 40x, burns at 14x. Sweet!

I’ll carry on using DVD-RAM for day-to-day stuff, as it’s just vastly more convenient than CD. But now when everything’s finalized and annotated and cataloged, I can burn it on gold for keeps.

The CDRW drive I picked was a QPS Que! and in spite of the initial problems, I’m happy with it on balance.