Kindle and Amazon and the DMCA

Amazon have misused the DMCA to demand takedown of a file called azw-0.1.zip. Since I have both the archive file and a Kindle, and have used both together, I can explain what’s really going on. Hopefully this will clear up some of the misinformation floating around.

The code in the disputed zip file is written in Python. It calculates the Mobipocket PID for your Kindle, based on the serial number written on the back. You can then provide this PID to any e-book store that sells e-books in DRMed Mobipocket format. They can sell you encrypted Mobipocket e-books, and you can then run a second Python script which flips a flag in the e-book file, making it readable on your Kindle. (The flag is just one that says “This is encrypted for Kindle”; no encryption is broken.)

This works because Amazon bought Mobipocket a few years ago, and used their DRM scheme and e-book format as the basis of the Kindle’s e-book format. The basic Mobipocket format is pretty simple. It’s HTML inside a Palm OS PDB database. That’s it. The DRM just adds a layer of encryption.

So, why are Amazon upset about this?

One theory is that they don’t want Kindle owners buying books anywhere other than Amazon.com. Well, if that’s the case, they’re playing a losing game, because Fictionwise (recently purchased by Barnes & Noble) sells e-books in DRM-free Mobipocket format, which you can just drag-drop onto your Kindle.

A second theory is that Amazon don’t want people to be able to create DRM-encumbered e-books for Kindle themselves, bypassing whatever fees Amazon may be charging for the service. I don’t know how true that may be, as I have no interest in creating DRM-encumbered anything, so I’ve never investigated how much Amazon charges.

My personal theory is that the real reason Amazon don’t want people finding out their Kindle’s Mobipocket PID is a fear that people will then find out how to decrypt their DRM-encumbered Mobipocket books.

And indeed, there is a completely different set of Python scripts floating around on the web that will decrypt a Mobipocket e-book given the PID used to encrypt it. This shouldn’t be a surprise to anyone; DRM is fundamentally flawed. Clearly the e-book reader software has to have all the information necessary to decrypt the book so that it can show it to me. That being the case, it’s inevitable that the decryption code will be reverse-engineered if enough people are motivated enough to do so.

But make no mistake: the azw-0.1 files do not break any copy protection or reveal any secret codes. They just calculate the PID of your Kindle, based on the serial number that’s written right on the back of the device in plain sight. They are interoperability tools, and the DMCA explicitly allows for interoperability tools. I suspect that the EFF could take on this case and win easily.

While I’m writing, here’s a quick summary of a few Kindle myths that I see repeated a lot in coverage of the story:

  • The Kindle traps you into buying everything from Amazon.

    Not true. Even if the azw scripts were illegal, you could still buy as much DRM-free content as you liked, load it directly onto the Kindle via USB, and never use the wireless connection to Amazon at all. As mentioned above, you can buy DRM-free e-books in Kindle-ready Mobipocket files from Fictionwise.

    It’s like the iPod: you may be stuck with a single vendor for DRM-encumbered content, but you can buy your DRM-free content from anywhere. Personally, I intend to buy as little DRM-crippled content as possible, and hope that Amazon gets the message.

  • The Kindle uses proprietary e-book format.

    As mentioned above, the Kindle’s native format is a trivial variation on Mobipocket format, which is HTML inside a Palm PDB database. The open-source mobiperl tools will pack and unpack .mobi and .azw files.

    As Mobipocket’s FAQ points out, the HTML extensions and metadata are based on an open industry standard.

    Also, there are free tools from Mobipocket for creating e-books. They’re Windows-only, however, and don’t seem to work under WINE.

  • You have to get all your content onto your Kindle by sending it to Amazon.

    Wrong. The Kindle mounts as a hard drive, using Storage Class USB. No drivers are required on Windows, Mac or Linux. Your library of books appears in a folder called “documents”. They’re just .azw and .mobi files. You can drag more books into the folder in Mobipocket or ISO-8859-1 text format, and the Kindle will display them.

    If you want to read PDFs, you have three options. One is to e-mail the PDF to your Amazon Kindle e-mail address; Amazon will convert it and it will appear wirelessly on your Kindle, at a cost of 10 cents. The second option is to e-mail the PDF to your free Kindle conversion e-mail address, and have Amazon e-mail it back in mobi/azw format for you to load onto your Kindle via USB. The third option is to use free tools to convert the PDF to mobi yourself, in which case Amazon need never see what’s in your PDF.

    From my own experiments, it appears that Amazon are using the open source pdf2edit on their back end as the conversion tool. Either that, or they’re using something which has exactly the same formatting conversion quirks.