The goal of this post is to simply document the extension and show how to create new works with it. I’m not going to debate whether it’s a good thing for the ecosystem here.

Install Phone Disk

Anyone working extensively with iBooks EPUB development should first grab a copy of the invaluable Phone Disk. This allows Mac and Windows computers to browse most of the iOS device filesystem as a normal drive, including opening and inspecting EPUB books.

Download a fixed-layout EPUB sample from the iBookstore

iBooks samples do not have DRM and so can be opened and read normally. A good bet is to download one of the “Children’s Picture Books”; these tend to be fixed-layout books.

Find the display-options file

The key component of the extension is the addition of the file META-INF/com.apple.ibooks.display-options.xml. This file tells iBooks that the book is the fixed-layout type, and typically looks something like this:

<?xml version="1.0" encoding="UTF-8"?>
<display_options>
    <platform name="*">
        <option name="fixed-layout">true</option>
    </platform>
</display_options>

There are a number of options here which are detailed in the iBookstore Asset Guide (v4.5), including how to define two-page spreads.

The first thing I tried with one of the samples was to validate it with the latest epubcheck 1.1:

Epubcheck Version 1.1
WARNING: /Users/liza/foo.epub: item (iTunesMetadata.plist) exists in the zip file, but is not declared in the OPF file
WARNING: /Users/liza/foo.epub: item (iTunesArtwork) exists in the zip file, but is not declared in the OPF file

These are just warnings; the fixed-layout book is valid. You can try opening them in other EPUB readers, with varying success depending on how wild the book designer got with the fixed-positioning.

Inspect the OPF and NCX

One immediate surprise to iBooks developers is that you’re likely to find an embedded font (now supported in any iBooks EPUB). Otherwise there’s nothing added to the OPF.

The NCX examples I found did not include all of the XHTML pages; I’m not sure whether that’s just a function of the book being a sample.

Structure of a fixed-layout book

In a typical EPUB, each chapter or major division of content is a single XHTML file. In the fixed-layout model, each XHTML file corresponds to a single physical page. So while it’s typical to name most EPUB XHTML files like chapter01.html, chapter02.html, in the fixed-layout book you’re apt to see a layout like:

cover.xhtml	font		package.opf	page001.xhtml	page003.xhtml	toc.ncx
css		images		page000.xhtml	page002.xhtml	page004.xhtml

XHTML content

The one unusual element in the fixed-layout content will be this HTML, which is required:

<meta name="viewport" content="width=1000, height=1000"></meta>

The specific width/height is per-book and based on the desired dimensions of the page. iBooks uses this information to scale the screen size appropriately (more on this later).

Otherwise the XHTML should look relatively normal (other than being very short). In fact the XHTML will be exceptionally readable since these books are by necessity all hand-coded at the moment.

CSS

Here’s where things get interesting:

The body element must be defined with an absolute width/height that matches the viewport above:

body {
	width: 1000px;
	height: 1000px;
}

Images and text will often be absolutely-positioned. z-index and other advanced positioning techniques are also supported. Font sizes are specified in pixels, not ems (users can zoom fixed-layout books, but the text will not reflow). Background colors are useful in this model as the XHTML content is full-bleed and does not usually have the goofy fake book margin.

Since this is all standard CSS 2.1 (or CSS3), these books don’t technically require a specialized reading system; any one based on a modern browser, like the mobile version of Ibis Reader, could potentially consume these books if the reading system could scale the pages appropriately.

Interactivity

It’s been noted that my original JavaScript in EPUB example no longer works in more recent versions of iBooks. This had nothing to do with the JavaScript parts themselves; instead the problem was that Apple began squashing the positioning information needed to align the popups.

In recent work for publishers I’ve found ways to get around those limitations, but they’ve been quite awkward and difficult to author. Would the “freedoms” of positioning in the fixed-layout books allow the original sample to work again?

Yes!

I took my original file and made only three changes:

1. I added the com.apple.ibooks.display-options.xml file just as shown above.
2. I added the viewport directive to the XHTML file and set it (arbitrarily to 600 pixels wide, 800 pixels tall).
3. I modified the CSS body directive to match the viewport.

The result:

Download the example EPUB (Creative Commons Attribution-Noncommercial-Share Alike 3.0)

Obviously the layout could use some work; there’s no longer any default margin added by the ereader, so the text bleeds out. Also my 600×800 dimensions don’t exactly match the iBooks aspect ratio, so there’s more fake book border. But you can see how this could be improved.

Note that the vast majority of EPUB books should not be formatted this way. This isn’t a one-way ticket to CSS Town. If the text on a particular XHTML page overflows the bottom of the single page, it will get cut off. You cannot mix fixed-layout and reflowable pages in the same EPUB file.

The arrival of iBooks this year helped many people understand that ebooks were already capable of interesting experimentation, multimedia, and nuanced, thoughtful design. Many of iBooks’ strengths come directly from the fact that they wisely chose to base it on WebKit, the rendering engine core used in desktop browsers like Safari & Chrome and mobile browsers for the iPhone, iPad, Android, and now Blackberry browsers. This makes it an attractive test platform for folks developing new and innovative content in EPUB, but many have been frustrated with the time and effort it takes to update EPUB content in iBooks on an iPad using the iTunes sync. This post outlines an easier approach for iteratively developing and testing changes to EPUB XHTML and CSS files in iBooks (1.1.2) without using iTunes.

Here’s a tweak to the XHTML of a chapter and the CSS used for h1s that I did quickly without having to make a new EPUB or use iTunes:

The key to this hack technique is the free (as in beer) iPhone Explorer application, which exposes the contents of your iPad like a USB drive (including resources for Apps). It has both a Mac and Windows version, but I’ve only tested these instructions on a Mac. Before starting, connect your iPad to your computer and iTunes. You should not be using automatic syncing (“Sync Books” is not checked).

To get started playing with a particular EPUB, sync the file onto your iPad, open the book once in iBooks, then close iBooks. Now you’re ready to start moving the files back and forth (iPhone Explorer is all drag and drop).

1. Open iBooks to your Library (not a book).
2. Open iPhone Explorer (or hit Refresh if it was open, but this is occasionally flaky).
3. Expand [Your] iPad->Media->Books.
4. Find the [XXXX].epub you’re interested in and expand it until you find your XHTML content or CSS.
5. Drag the XHTML file or CSS you want to experiment with somewhere on your computer (perhaps your Desktop in the Finder) to edit.
6. Edit the file using something you like.
7. Drag the XHTML or CSS file back from your computer to the same location in iPhone Explorer.
8. Say “OK” to the “Overwrite?” dialog.
9. Open the book in iBooks to see the changes.

Now you’re ready to use steps 5–9 (closing the book in iBooks in between) to continue experimenting with the EPUB’s design or content.

It can occasionally be non-obvious which filename iBooks has assigned to your EPUB. Curious folks will want to play around with some of the other resources exposed in addition to the exploded EPUBs (check out Books.plist).

iBooks supports JavaScript-based interactivity in EPUB content.

I took some content from Cooking with Booze by James Bridle (a.k.a. George Harvey Bone). It’s released under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 license, which means that this derivative work is as well.

I took one recipe and marked it up in EPUB:

I added some slightly useless annotations and illustrative images from Flickr (also under an Attribution license). I then added some JavaScript-based event handlers. If you tap on the hyperlinked items in the recipe, the annotation will fade in as a popup. Tap on the annotation to dismiss it.

There are two kinds of annotations available: image popups and textual notes. Here’s a text note:

The JavaScript here is extremely simple, though I needed to use webkitTransform rather than normal positioning code because iBooks does not recognize position: absolute and the like:

function popup() {
  var p = document.getElementById(this.id + '-popup');
  var top_of_el = this.offsetTop;
  p.style.opacity = 1;
  var new_y = p.offsetTop - top_of_el - 30;
  p.style.webkitTransform = 'translateY(-' + new_y + 'px) translateX(' + this.offsetLeft + 'px)';
}
function dismiss() {
  this.style.opacity = 0;
  var el = this;
  setTimeout(function () {
    el.style.webkitTransform = null;
  }, 1000);
}
/* Register the events */
var terms = document.getElementsByClassName('term');
for (var i=0; i < terms.length; i++) {
  terms[i].addEventListener('click', popup);
}
var popups = document.getElementsByClassName('popup');
for (var i=0; i < popups.length; i++) {
  popups[i].addEventListener('click', dismiss);
}

Here’s a sample of the XHTML:

1 wild <a class="term" id="turkey">turkey</a> breast

Some other nerdy details:

1. The fade-in effect uses CSS3. I think the iBooks implementation may not be hardware-accelerated like Mobile Safari — it’s less smooth than the transitions on Ibis Reader, for example.
2. Inline script blocks do not work, which is not a bad thing.
3. Scripts are disallowed from accessing remote URLs. This means that ebooks that interact with remote data, like the geo-aware ebook, do not work. Interestingly, this is one of the restrictions that Baldur Bjarnason suggested might allow for safe JavaScript use in ebooks in a recent comment on our post Practical Interactivity and Shaping the Future of EPUB.
4. iBooks caching is really annoying isn’t it?
5. I haven’t tried this on iBooks for iPhone.
6. I have no idea if JavaScript is allowed in the iBookstore…
7. …not that you need Apple’s permission to sell a DRM-free enhanced EPUB that users could load into iBooks themselves.

Download the sample EPUB book. All content in the book is licensed under a Creative Commons license of some kind; the whole bundle should be considered Creative Commons Attribution-Noncommercial-Share Alike 3.0.

That said, I encourage commercial publishers to explore interactivity in EPUB, and it goes without saying that we’d love to help, whether it’s to provide glosses and annotations like those shown here, or interactive quizzes, or animation, or…

We’re making the geo demo featured in the Interactivity in EPUB talk available for download under the MIT License.

About the demo

In the demo, the geolocation capability of the web browser transmits your latitude/longitude. The code then queries the Geonames database to get an English placename.

After that, the code will continually poll for your current location. If you remain still, text will be displayed to indicate that the main character is bored and waiting to set off on her adventure. When your lat/long changes, it displays new text.

What’s in the demo

This is a ZIP file bundle of the following resources:

• The geo code itself, in JavaScript, CSS and XHTML
• A copy of the Monocle ereader by Inventive Labs
• A copy of geo-location-javascript, which provides a simplified API to the HTML5 geolocation feature
• The embedded font IM Fell English Pro, converted to SVG format for use on iPads/iPhones.

The book content is in the form expected by Monocle, not EPUB, though if someone wanted to produce an EPUB version I would be happy to link to it. It should be possible to produce a valid EPUB file though very few ereaders would be able to run it.

Download the source code.

Try it now

You can also try it out in a geo-aware browser. This should work in Chrome, Firefox and Safari 5, as well as on iPads and iPhones (and probably Android), though the user interface will need to be changed to support smaller devices.

Keep in mind that if you’re on a real computer, you’ll never see the “you’ve moved” state change since your location is based on your internet service provider. Even on a mobile device, it can take quite some distance to update your position as GPS does not discriminate very finely.

In the video you can enjoy the dramatic moment when the lights inexplicably dim and I think I’m being thrown off stage.

(Slides for ‘Designing ebooks for ePub reading engines’)

Update April 19, 2010: Like many others, I was unable to fly to London and will not be attending LBF.

1. As noted earlier by Dave Thomas, when an ePub file is added to iTunes, the file is modified in place to add a file called iTunesMetadata.plist. This is really obnoxious, but it doesn’t render the files invalid. You can look at the file to see the values that iTunes has derived from the metadata of the epub; initially it won’t be readable — change the file permissions, if that’s something you know how to do.

   <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
   <plist version="1.0">
   <dict>
           <key>artistName</key>
           <string>Doctorow, Cory</string>
           <key>book-info</key>
           <dict>
                   <key>package-file-hash</key>
                   <string>47CC1BA0C40608D65C4F815788FBD10F</string>
                   <key>publisher-unique-id</key>
                   <string>urn:uuid:9bde0872-1ad5-11dd-b5b9-0018f369440e</string>
                   <key>unique-id</key>
                   <integer>336337745614362820</integer>
           </dict>
           <key>genre</key>
           <string>lcsh: Novels</string>
           <key>itemName</key>
           <string>Little Brother</string>
           <key>playlistName</key>
           <string>Little Brother</string>
           <key>releaseDate</key>
           <string>2008</string>
           <key>sort-artist</key>
           <string>Doctorow, Cory</string>
           <key>sort-artist-status</key>
           <integer>3</integer>
   </dict>
   </plist>
   

2. 

   As expected, you can add ePubs that won’t actually be readable in iBooks. Most of the cases are totally understandable: missing container files and other severe problems. It also seems that omitting a dc:title is a fatal error. All errors are reported generically as “This book cannot be opened,” so check with the ePub validator to look for the real cause.

3. One nice feature of Feedbooks content is the recommendations for related titles, with links directly to those ePub files. In Ibis Reader, accessing an ePub file link will prompt you if you want to add that book to your library, and let you start reading right away. In iBooks, it will open Mobile Safari and then return an error.
4. Hyperlinks, either internal to the book or external to the web, otherwise work normally.
5. Flash video does not work (of course). HTML5 <video> also does not work, though that’s outside of the spec and therefore unsurprising. I don’t know if it’s possible to embed multimedia into an iBook ePub using some other method. [Edit: subsequently Apple has released updates to support HTML5 video through use of the video tag; note that this will produce an invalid EPUB file at this time.]
6. [Edited to remove incorrect statement about justification; does seem to be under CSS control.]
7. CSS support seems good, as expected from a WebKit-based reading engine. I haven’t rigorously tested it but I assume it will support CSS outside of the required ePub subset, just as a normal web browser would.
8. Embedded fonts don’t appear to work. This is consistent with @font-face not being supported on Mobile Safari. It was also what I expected.
9. Extremely long books with limited breaks seem to fail. At least, Ulysses does not work at all: page numbering is random, and reading the book is basically impossible.

10. Text that cannot be line-broken (like code) runs off the screen. This is true of both portrait and landscape mode:

    

11. Tables sometimes break in the middle of text when paginating. (In Ibis Reader we try to ensure that text doesn’t get cut off even when it occurs inside a table.)
12. My favorite test book looks fine:
    

Overall I’d say iBooks is pretty good for a first-generation ePub reader. The biggest concern is of course that once you purchase books from iTunes, you’re locked in to only reading them in iBooks.

I also hope to see an ePub style guide from Apple to help publishers and designers understand more fully what is and isn’t supported in iBooks.

Update: Liz Castro has some good observations too.

I quickly ran through all of the invalid books to see how iTunes would behave. Note that it’s too early to say how or if these will display in iBooks.

1. Contains Adobe DRM – Allowed. Surprising since these won’t be readable.
2. No container.xml – Allowed, but no visible metadata. Clearly an error; this book wil not work and should not be addable.
3. Invalid character sets (non-UTF8) – Allowed, but this is often acceptable in ePub software.
4. No findable NCX, or no NCX at all – Allowed. This book should not actually work as there will be no useful TOC.
5. No namespaces in container.xml – Allowed, and metadata is still captured. ADE can also read this, but it shouldn’t be possible as the XML is invalid.
6. No spine – Allowed. This book should not be readable.
7. No dc:title – Allowed, uses filename. This is the one place where Bookworm and Ibis Reader are more strict than other readers. I feel that if there’s not even the most basic required metadata, we can’t guarantee that we can display it. But other ereaders do allow this, so not surprising.
8. Extremely invalid XHTML (no body) – Allowed, though not surprising since iTunes has no reason to process the XHTML content yet.

In fact, all 38 books in the test suite were addable. This isn’t actually a good thing, especially when the DRM ebooks were accepted too. iTunes should be notifying users quickly that their books can’t be read in the ecosystem, rather than waiting until they’re synced.

We’re on track now to resume consulting and custom publishing software development alongside Ibis Reader, so please feel free to contact info@threepress.org for:

• Ebook strategy
• EPUB consulting
• Help getting books onto the iPad
• Challenging ebooks: enhanced books, video, audio, interactivity, custom fonts
• HTML5 and mobile development
• Custom Bookworm development
• White labeling Ibis Reader
• XML workflows
• DocBook and other schema development
• Speaking and training

If you access the site on an iPad today, you get a different block of content on the home page:

The link provided is the same one that we direct all HTML5 devices at. Here are some screenshots from the experience:

List page

Book table of contents

Reading page

Obviously there are some stylistic changes we’ll want to make:

• Making the margins in reading pages wider
• Enlarging the size of cover images
• Making better use of space on list pages

Tech notes

Here are a few things we had to change to get it working:

Event handling

On Android and the iPhone, touchstart and touchend events are fired when the user touches the application, followed by onclick. We preferentially use the touch events when reading a book because of some quirks in Android.

The iPad doesn’t seem to automatically fire the onclick event after a touch, so I had to make it manually trigger that for anchor links.

Orientation changes

The iPad simulator doesn’t fire the orientationchange event when the simulator is rotated. We don’t know yet whether that will be true of the device, so at this time Ibis Reader won’t correctly re-paginate after an orientation change.

Viewport width/height

The iPhone doesn’t report its height/width correctly, and doesn’t report it differently when rotated. We have these values hard-coded in the app.

Happily, the iPad does seem to report the correct value, both when the URL bar is visible and when it’s not (because the app has added to the home screen). We don’t yet know about rotation since we’re unable to detect that.

HTML5 database size

There’s a bug in the current iPhone OS (3.1) in which the user won’t be prompted to increase an HTML5 database after the webapp has been added to the home screen. This means that Ibis Reader users on the iPhone can’t ever add more than the default 50MB worth of books. This bug appears to have been fixed in the iPad OS.

(Android does not prompt the user to increase a DB, but does increase it automatically if there is space available on the device.)