Threepress Consulting blog

Threepress will be fully represented at Tools of Change 2011. Keith Fahlgren will be part of the panel All About EPUB3. I’m not speaking this year but I’m attending; a relief for me as I’m really looking forward to spending the time listening rather than rehearsing.

The following is a list of talks that I’m either planning on attending personally or recommend, based on topic or speaker:

Workshops

This year’s workshop program is really strong. Attendees will not be disappointed by any of:

• EPUB Finishing Touches with Joshua Tallent.
• Standards in Flux with Bill Kasdorf et al.
• Designing iPad Apps with Jennifer Brook, William Couch, and Craig Mod. Craig’s thoughts on digital books are worth the price of admission.
• Two of my favorite people are presenting Metadata Boot Camp: Laura Dawson (now with Firebrand) and Adam Witwer (O’Reilly Media).
• I’m quite interested in HTML5: A Workshop for Publishers as it’s given by the author of the HTML5 Google Pac-Man, though I haven’t met him.

Sessions

Tuesday

• Is There Such a Thing as a Good Business Model for Publishing These Days? with Louis Rosenfeld (Rosenfeld Media, LLC), John Oakes (OR Books)
• Value-Added Apps: A Conversation with Top App Makers on How to Create Apps with Craft, Care and Polish with Neal Hoskins (WingedChariot), Gus Balbontin (Lonely Planet), Anil Dash (Activate), Michel Kripalani (Oceanhouse Media), Pete Myers (BirdsInTheHand, LLC). This is a really strong lineup.
• Data-driven Marketing and Product Development with Emily Sawtell (McGraw-Hill International Publishing Group), Brett Sandusky (Kaplan Publishing)
• Bookselling in the 21st Century with Kassia Krozser (Booksquare.com), Lori James (All Romance/OmniLit/ARe Cafe), Jenn Northington (WORD), Kevin Smokler (Booktour.com), Jessica Stockton-Bagnulo (Greenlight Bookstore), Malle Vallik (Harlequin Enterprises Ltd)
• All about EPUB3: An Update with Robert Kasher (Book Masters), Keith Fahlgren (Threepress) and others to be announced.
• eReading Survey Findings and Research: A Look behind the Numbers with Sarah Weinman (DailyFinance), Matthew Bernius (Open Publishing Lab at the Rochester Institute of Technology), Kelly Gallagher (RR Bowker), Peter Hildick-Smith (Codex-Group LLC), Jennifer Manning (Nielsen)
• From Startups to Publishing Companies Ripe for Expansion: What Are Investors in the Publishing Sector Looking For? with Jeff Shelstad (Flat World Knowledge), Linda Holliday (CEO Semi-Linear, Director New York Angels), Mark Jacobsen (O’Reilly AlphaTech Ventures), Stephen Koenig (Interweave), Brian Rich (Catalyst Investors), Rick Richter (Ruckus Media). I run a publishing startup, so I may be more interested in this than you.

Wednesday

• What Do eReading Customers Really, Really Want? An In-depth, Research, and Data-driven Exploration of Reading Behavior, Content Consumption, and Consumer Attitudes Toward eReaders and Multifunction Devices with James Cathey (Qualcomm MEMS Technologies, Inc. ), Michael Tamblyn (Kobo). Any talk with Michael Tamblyn is worth attending.
• Solving the Digital Loan Problem: Can Library Lending of eBooks be a Win-win for Publishers AND Libraries? with Heather McCormack (Library Journal Magazine), Katie Dunneback (East Central Library Services), Ruth Liebmann (Random House, Inc.), Micah Bowers (Bluefire). This is a topic I’m particularly interested in right now.
• Open, Webby Book Publishing Systems with Hugh McGuire (LibriVox.org/Iambik.com/BookOven.com), Kirk Biglione (Oxford Media Works), John Maxwell (Simon Fraser University). All of the cool kids will be in this session.
• What’s in HTML5 for Publishers? with Philippe Le Hegaret (W3C).

Keynotes

Do not be caught schmoozing in the Exhibit Hall during these keynote speeches:

• The Condition of Music by James Bridle
• Making the Book Truly Accessible by Jim Fruchterman (Benetech)
• Context First: A Unified Field Theory of Publishing by Brian O’Leary. I have seen this talk and it is one of the best presentations in publishing you will ever see.

I presented at Digital Book World 2011 about using EPUB3 to produce multimedia and interactive ebooks that will be compatible with multiple devices and software ereaders.

Anyone who’s ever been to a digital publishing conference knows that there is always a bewildering array of products that create and display enhanced content: ebooks that contain video, audio, animation, or other forms of interactivity. Generally each new platform requires an entirely different way of preparing the content and all the multimedia assets. While the production of static ebooks has become relatively standardized (using EPUB2), publishers are still struggling with how to produce more digital-native products without tremendous expense and constant re-tooling.

I’m optimistic that EPUB3 will provide significant headache-reduction in this area. While creating high-quality static, pure-text ebooks still has challenges, things are a lot better today than even a year ago. If we’re to see similar consolidation in the enhanced ebook space in the same timeframe, publishers are going to have to be firm that they will only (or at least primarily) produce standards-based multimedia books.

Slides from DBW

EPUB3 interactivity demo in iBooks

The talk includes a video demo. I had created an EPUB document several months ago for a research project, using Abroad by Thomas Crane as source material (edition from the Internet Archive). I wanted to emulate the accelerometer-driven animation of the Alice for iPad book using only the tools available in EPUB3.

Since the source was a heavily-illustrated children’s book, EPUB’s reflow capability was not really an asset. This quality made the book perfect for use in Apple’s fixed-layout mode. The video below shows the book running in iBooks on an iPad:

Accelerometer

In the first example, I use the accelerometer interface in JavaScript (some sample code), to cause the ship to “sail” back and forth as the user tilts the device.

Animation

In the second example, if the user touches any of the birds on the title page, they fly away. I modified the jQuery Path library to use CSS3 transforms to efficiently animate along a curve. JavaScript performance in iBooks appears to be somewhat worse than Mobile Safari; optimizing using CSS3 rather than DOM position updates is recommended.

Canvas, Processing.js, HTML5 audio

I’m a big fan of the Processing visual programming language. In the last example, the train appears to give off puffs of steam; each time the steam comes out of the train the puffs are in slightly different positions.

I embedded Processing.js code (which is itself then interpreted by JavaScript) and rendered the puffs to a transparent canvas. The puffs appear in a slightly random position, fade out as they reach the top of the page and then spawn anew.

Finally, there’s also an event bound to the invisible canvas: tapping on it will fire an HTML5 audio (a sample of a train).

(At this time we’re not able to share the demo EPUB file itself.)

We’ve just released an updated version of Ibis Reader with some new features, performance enhancements, and many bug fixes. The new features are all in the mobile version, though some bug fixes affect the desktop web experience as well. Here are some highlights:

Improved pagination and position-tracking

We’ve changed the method of moving from page to page and remembering where you left off. Books should load faster, and it should be easier to understand whether you’re moving forward or backward in the page.

Relative book length and reading position in My Books

In your list of books on the device, we now display some hints about the length of the book and how far you’ve read in it:

Also on this screen we tried to make it more clear that your “Online Bookshelf” is a link. Many people were confused by the old layout, which made that link seem more like a title. Clicking on the Online Bookshelf link will take you to the cloud library and allow you to pull down individual books to read on the current device.

Online Bookshelf cover images and metadata

For new books that you add starting with this release, you’ll be able to see cover images in your online bookshelf.

Improved iPad reading screen

We made the margins more generous on the iPad screen and added the page title (in portrait mode).

High-resolution (retina) screen support

We refreshed many of the icons and images to take advantage of the high-resolution screen in the iPhone 4 (and likely iPad 2).

Galaxy Tab support

We made some changes to ensure that Ibis works with the Galaxy Tab, as previously all our tested Android devices were phone-sized. We’ve chosen to give the Tab a phone UI rather than the iPad UI, in part due to screen size but mostly because Android simply does not support the smooth animations and transitions that iOS can.

Known issues

One consequence of the pagination change is that existing positions within a chapter have not been retained, so if you were reading a chapter before this update, you’ll be reset to the start of that chapter. We regret the inconvenience, but it will start position-tracking in the new method right away.

Some versions of Android don’t report their screen size correctly, either at all or after the device has been rotated. We’re going to be working on improving our Android UI in the coming weeks.

Getting the update

As always, just visit your installed Ibis Reader icon (iOS) or go to http://ibisreader.com/ on your mobile device. Due to the way HTML5 works, you may need to refresh more than once to see the update. You should receive a notification that your database was upgraded: close and restart Ibis and you’re ready to go with the new version.

Thanks

We received a lot of help in testing the new release from our friends and colleagues. Your support is always appreciated.

iBooks now supports an extension to EPUB that allows publishers to create books with precise layout using CSS. This is Apple’s own extension, not part of the EPUB specification itself (and not one that they suggested be included in EPUB3).

The goal of this post is to simply document the extension and show how to create new works with it. I’m not going to debate whether it’s a good thing for the ecosystem here.

Install Phone Disk

Anyone working extensively with iBooks EPUB development should first grab a copy of the invaluable Phone Disk. This allows Mac and Windows computers to browse most of the iOS device filesystem as a normal drive, including opening and inspecting EPUB books.

Download a fixed-layout EPUB sample from the iBookstore

iBooks samples do not have DRM and so can be opened and read normally. A good bet is to download one of the “Children’s Picture Books”; these tend to be fixed-layout books.

Find the display-options file

The key component of the extension is the addition of the file META-INF/com.apple.ibooks.display-options.xml. This file tells iBooks that the book is the fixed-layout type, and typically looks something like this:

<?xml version="1.0" encoding="UTF-8"?>
<display_options>
    <platform name="*">
        <option name="fixed-layout">true</option>
    </platform>
</display_options>

There are a number of options here which are detailed in the iBookstore Asset Guide (v4.5), including how to define two-page spreads.

The first thing I tried with one of the samples was to validate it with the latest epubcheck 1.1:

Epubcheck Version 1.1
WARNING: /Users/liza/foo.epub: item (iTunesMetadata.plist) exists in the zip file, but is not declared in the OPF file
WARNING: /Users/liza/foo.epub: item (iTunesArtwork) exists in the zip file, but is not declared in the OPF file

These are just warnings; the fixed-layout book is valid. You can try opening them in other EPUB readers, with varying success depending on how wild the book designer got with the fixed-positioning.

Inspect the OPF and NCX

One immediate surprise to iBooks developers is that you’re likely to find an embedded font (now supported in any iBooks EPUB). Otherwise there’s nothing added to the OPF.

The NCX examples I found did not include all of the XHTML pages; I’m not sure whether that’s just a function of the book being a sample.

Structure of a fixed-layout book

In a typical EPUB, each chapter or major division of content is a single XHTML file. In the fixed-layout model, each XHTML file corresponds to a single physical page. So while it’s typical to name most EPUB XHTML files like chapter01.html, chapter02.html, in the fixed-layout book you’re apt to see a layout like:

cover.xhtml	font		package.opf	page001.xhtml	page003.xhtml	toc.ncx
css		images		page000.xhtml	page002.xhtml	page004.xhtml

XHTML content

The one unusual element in the fixed-layout content will be this HTML, which is required:

<meta name="viewport" content="width=1000, height=1000"></meta>

The specific width/height is per-book and based on the desired dimensions of the page. iBooks uses this information to scale the screen size appropriately (more on this later).

Otherwise the XHTML should look relatively normal (other than being very short). In fact the XHTML will be exceptionally readable since these books are by necessity all hand-coded at the moment.

CSS

Here’s where things get interesting:

The body element must be defined with an absolute width/height that matches the viewport above:

body {
	width: 1000px;
	height: 1000px;
}

Images and text will often be absolutely-positioned. z-index and other advanced positioning techniques are also supported. Font sizes are specified in pixels, not ems (users can zoom fixed-layout books, but the text will not reflow). Background colors are useful in this model as the XHTML content is full-bleed and does not usually have the goofy fake book margin.

Since this is all standard CSS 2.1 (or CSS3), these books don’t technically require a specialized reading system; any one based on a modern browser, like the mobile version of Ibis Reader, could potentially consume these books if the reading system could scale the pages appropriately.

Interactivity

It’s been noted that my original JavaScript in EPUB example no longer works in more recent versions of iBooks. This had nothing to do with the JavaScript parts themselves; instead the problem was that Apple began squashing the positioning information needed to align the popups.

In recent work for publishers I’ve found ways to get around those limitations, but they’ve been quite awkward and difficult to author. Would the “freedoms” of positioning in the fixed-layout books allow the original sample to work again?

Yes!

I took my original file and made only three changes:

1. I added the com.apple.ibooks.display-options.xml file just as shown above.
2. I added the viewport directive to the XHTML file and set it (arbitrarily to 600 pixels wide, 800 pixels tall).
3. I modified the CSS body directive to match the viewport.

The result:

Download the example EPUB (Creative Commons Attribution-Noncommercial-Share Alike 3.0)

Obviously the layout could use some work; there’s no longer any default margin added by the ereader, so the text bleeds out. Also my 600×800 dimensions don’t exactly match the iBooks aspect ratio, so there’s more fake book border. But you can see how this could be improved.

Note that the vast majority of EPUB books should not be formatted this way. This isn’t a one-way ticket to CSS Town. If the text on a particular XHTML page overflows the bottom of the single page, it will get cut off. You cannot mix fixed-layout and reflowable pages in the same EPUB file.

Thanks to Hatgirl on MobileRead for this great tip on integrating Ibis Reader with Baen free books:

If you use Ibis Reader, you will have seen the “Get Books” link. This allows you to view OPDS catalogs (lists of web-accessible ebooks). The Feedbooks catalogs are pre-installed, and some of you may have set up a Calibre/Dropbox OPDS catalog of your own library.

On a whim, I used the “Add Your Own Catalog” link to add the WebScription Stanza link,
Code:

http://www.webscription.net/stanza.aspx?feed=free

It worked! I can access the Baen Free Library directly from Ibis Reader! In “Get Books” I click “Baen” (or whatever you named the OPDS link), then “Baen Free Library”, then the “Read” button next to the book I want to add to my Ibis Reader library. Yay!

Edited the above URL to go straight to the free books, bypassing the top-level catalog.

The Stanza catalog is an earlier form of OPDS, so some features like cover images won’t work in Ibis or other OPDS readers. If Baen updates their catalog to OPDS 1.0 then those features will be enabled, and we’d definitely consider adding it as a built-in catalog like the Feedbooks ones.

Please share any other OPDS catalogs that you’d like to see added (or at least listed as optional catalogs users can add themselves).

Read the original post.

Thank you to Roxanne McHenry for the opportunity to talk about ebooks, EPUB3, book apps, and Ibis Reader in a recent interview with The Unruly Guides.

I haven’t been checking each incremental release so it’s possible this landed earlier, but the current version of Apple Pages has improved the semantics of their EPUB export.

The original template now produces <p> elements instead of <div>s, an important update:

<h1 class="s7">Introduction to ePub</h1>

<p class="s6">This document will show you how to use paragraph styles to create a publication that looks great and is easy to navigate in ePub readers like iBooks... </p>

Presumably this affects the degree to which Pages-outputted styles are visible in iBooks (as user preferences may override), but it’s great to see more semantically-correct output. This corrects my earlier reservation about using Pages for EPUB creation.

We’re very excited to have a new team member for Ibis Reader. Ned Batchelder will be working with us in the weeks ahead to update and improve Ibis, and to add some features that have been requested by current and prospective software licensees.

First up is internationalization. One of the key advantages of an HTML5 webapp versus a native app is that they can be installed in any country on a supported device, even in countries that don’t yet have an Apple App Store. But first the application needs to be localized for the regional language.

Ibis uses the Django application framework and Django comes with native internationalization support. We also used this when internationalizing Bookworm with the help of some great volunteers.

Before translators can get to work, the application needs to be readied: each bit of text in the site needs to be marked for translation. This step can be time-consuming and difficult to test.

Automating translation testing

Ned has posted some code on his own blog that he used to help validate the internationalization process. Some good comments point to more advanced tools that take a similar approach. Recommended reading for anyone doing internationalization work.

The tool fakes a translation by randomly capitalizing all of the letters in the site text. If done properly, you should get a result like this:

The book title Middlemarch and the chapter name look normal; those should not be translated. The remaining words are directions to the user and will need to be in a regional language. If any of the site text, like “Next” or “Previous”, showed up with normal capitalization, we would know that we missed a step.

We won’t be rolling out translated versions to ibisreader.com in the near future, but we will be posting some performance enhancements and feature updates very soon.

Following my post Developing an EPUB Linking specification a couple of months ago, a subset of the EPUB3 Working Group formed and has been actively researching and discussing the problem of EPUB to EPUB linking and EPUB identifiers in general. Based on feedback from others inside and outside that group, I’ve decided that many aspects of my initial proposal are flawed, but we’re now much closer to consensus on a proposal (or two) for submission to the larger EPUB3 Working Group.

You can see the specifics of the current proposals from the subgroup as they evolve on the EPUB3 wiki at PURL-based EPUB Identifiers and URI Links Proposal. Please comment there or on the epub-revision mailing list rather than here.

The aggressive publication deadlines for EPUB3 (fast approaching in Q1 2011) mean that this work cannot expand too much in scope, but I suspect that’s good motivation: we should try to understand and solve the most pressing problem(s) for EPUB to EPUB links and then get out of the way and watch the implementors & content creators.

Note: These early drafts on the wiki don’t represent the consensus of the EPUB3 Working Group.

epubcheck is a valuable tool for anyone working with EPUB documents. Many beginners choose to start using it via the online validator at http://threepress.org/document/epub-validate, but it’s a mistake to use that too often. More explicitly:

If you’re making more than 3 EPUBs this year, you must setup epubcheck on your computer

Why setup epubcheck locally? If you have a local copy, you can:

• run it even if the online validator is down (it was this morning)
• run any version of epubcheck you want (useful when new versions are near release)
• run it more quickly
• run it on a lot of documents quickly (with a little command-line-foo)

Warning

This is a brief description of how you might be able to run epubcheck locally. This isn’t a tech support site, so please don’t post system questions in the comments.

Setup for both platforms

1. Ensure java is installed and in your PATH (Google is your friend if not)
2. Download the “binary” release you’d like to use (probably the featured one) from the epubcheck downloads page
3. Unzip the downloaded .zip file to somewhere on your computer
4. Find the file epubcheck-[something].jar in the newly unzipped folder (and keep this window open)
5. Find the EPUB file you’d like to validate (and keep this window open)

Each time you’d like to run epubcheck, you’ll need to ask the java command to run the epubcheck program (packaged as a .jar file) against your EPUB file. Happily, you can get drag and drop to help you with most of the filenames.

Windows

1. Open the “Run…” menu using [Windows-key]+r, Start Menu->Run…, or turning it on for Vista.
2. In the “Run…” menu, type cmd.exe.
3. In the new Command Prompt window (the “prompt”), type java -jar (note trailing space after “-jar”).
4. Drag the epubcheck .jar file from the unzipped folder into the Command Prompt window (the location of the file ["path"] should be copied).
5. Type another space in the terminal window after the .jar filename.
6. Similarly, drag the EPUB file from wherever into the Command Prompt window (it’s path should also be copied).
7. Press Enter.
8. Note the results in the Command Prompt window (No errors or warnings detected means it passed).

Mac

1. In the Finder, to Go->Utilities, then find the Terminal application (or do a Spotlight search for “Terminal”) and start it.
2. In the new Terminal window (the “prompt”), type java -jar (note trailing space after “-jar”).
3. Drag the epubcheck .jar file from the unzipped folder in the Finder into the Terminal window (the location of the file ["path"] should be copied).
4. Type another space in the terminal window after the .jar filename.
5. Similarly, drag the EPUB file from wherever in the Finder into the Terminal window (it’s path should also be copied).
6. Press Return.
7. Note the results in the Terminal window (No errors or warnings detected means it passed).

Linux

Ha, you’re on your own! (But do see the good comment from Toby Stevenson, below.)

Other tools

Now that you’ve mastered the above, you can try to run the epubpreflight checking tool in the same manner. It helps check if files are too big for the older Sony Reader devices, for example.