To match the licenses of other content in the presentation, the PDF is available for download under a Creative Commons Non-Commercial ShareAlike.

There are a few videos in the presentation which I’ve pulled out separately:

Using an accessible HTML5-based EPUB with VoiceOver

This demonstrations uses The Restless Universe from The Open University with a screenreader. The publication (somewhere between EPUB 2 and EPUB 3) uses the HTML5 canvas element to present visualizations, and includes alternate versions for the visually-impaired. Though this demo was run on a computer, the EPUB works in iBooks. (Software tools to build these publications available at http://sourceforge.net/projects/bookbind/.)

For each example, I show the visual presentation first, then the alternate screenreader-friendly one.

Creating a screen-readable interactive publication with WAI-ARIA

WAI-ARIA roles and states are used here in this demo (running on Firefox, Windows XP, NVDA screenreader) showing that JavaScript-driven interactive publications can also be accessible.

You’ll be able to hear the screenreader announcing that the word “delighted” has a “submenu”. When I use the keyboard to navigate I can find the link-like region through the auditory landmarks, select it, and then hear the popup read to me. WIthout using the ARIA roles, such JavaScript-driven interactivity would be invisible to the screenreader. See the slides for more details on the ARIA markup used here.

Interactive storytelling for the blind

My presentation included an excerpt from the documentary Get Lamp by Jason Scott; the relevant bit begins here in this presentation at a Google Tech talk.

Anyone who’s ever been to a digital publishing conference knows that there is always a bewildering array of products that create and display enhanced content: ebooks that contain video, audio, animation, or other forms of interactivity. Generally each new platform requires an entirely different way of preparing the content and all the multimedia assets. While the production of static ebooks has become relatively standardized (using EPUB2), publishers are still struggling with how to produce more digital-native products without tremendous expense and constant re-tooling.

I’m optimistic that EPUB3 will provide significant headache-reduction in this area. While creating high-quality static, pure-text ebooks still has challenges, things are a lot better today than even a year ago. If we’re to see similar consolidation in the enhanced ebook space in the same timeframe, publishers are going to have to be firm that they will only (or at least primarily) produce standards-based multimedia books.

Slides from DBW

EPUB3 interactivity demo in iBooks

The talk includes a video demo. I had created an EPUB document several months ago for a research project, using Abroad by Thomas Crane as source material (edition from the Internet Archive). I wanted to emulate the accelerometer-driven animation of the Alice for iPad book using only the tools available in EPUB3.

Since the source was a heavily-illustrated children’s book, EPUB’s reflow capability was not really an asset. This quality made the book perfect for use in Apple’s fixed-layout mode. The video below shows the book running in iBooks on an iPad:

Accelerometer

In the first example, I use the accelerometer interface in JavaScript (some sample code), to cause the ship to “sail” back and forth as the user tilts the device.

Animation

In the second example, if the user touches any of the birds on the title page, they fly away. I modified the jQuery Path library to use CSS3 transforms to efficiently animate along a curve. JavaScript performance in iBooks appears to be somewhat worse than Mobile Safari; optimizing using CSS3 rather than DOM position updates is recommended.

Canvas, Processing.js, HTML5 audio

I’m a big fan of the Processing visual programming language. In the last example, the train appears to give off puffs of steam; each time the steam comes out of the train the puffs are in slightly different positions.

I embedded Processing.js code (which is itself then interpreted by JavaScript) and rendered the puffs to a transparent canvas. The puffs appear in a slightly random position, fade out as they reach the top of the page and then spawn anew.

Finally, there’s also an event bound to the invisible canvas: tapping on it will fire an HTML5 audio (a sample of a train).

(At this time we’re not able to share the demo EPUB file itself.)

Slides from EPUB Evolutions are posted but it may be more helpful to hear the full presentation, which I’ve re-recorded.

Topics covered in the talk include:

• The current state of the EPUB3 working groups and overall schedule.
• Enhancements to worldwide language support, including vertical text.
• Native multimedia and HTML5 video/audio.
• Interactivity, and the UI and accessibility challenges it imposes.
• Improved styling and layout via CSS3 and media-query.

Here’s the executive summary of the talk:

• You can add interactivity to an EPUB book using either the <object> or <script> elements.
• Currently object has the best support, especially using Flash in Adobe Digital Editions.
• The script element is specifically discouraged (though not disallowed) in EPUB 2.0.1, and no major reading system supports it. However, all browsers do!
• The primary advantage of interactivity using script is that the content creator has the ability to manipulate every part of the ebook: all of the text, its layout, even potentially the user interface of the reading system itself. With object you can apply interactivity to just an arbitrary rectangle.
• There is a great deal of interesting animation and even interactivity possible using CSS3, which is allowed in EPUB 2.0.1 and is supported by iBooks and other WebKit-based ereaders.
• Interactivity is also possible using JavaScript combined with the HTML5 canvas element, though canvas is not part of EPUB 2.0.1.
• With access to JavaScript on mobile devices, ebooks can potentially gain access to information about the reading device itself, such as the location of the reader in the physical world, or the device’s orientation, or even add photos and video from the device’s camera.
• Any ereader that allows JavaScript provides the ability for ebooks to access live data on the web.
• The primary blocker for extremely rich interactive ebooks is ereader support, but by embedding browser-based reading systems like Monocle, content creators can distribute such ebooks today.
• It remains to be worked out how to handle JavaScript in larger ereading systems both safely and with proper fallbacks to non-interactive content.

I’ll also be posting some of the example files used in the tutorial in a forthcoming post.

In the video you can enjoy the dramatic moment when the lights inexplicably dim and I think I’m being thrown off stage.

(Slides for ‘Designing ebooks for ePub reading engines’)

To try to shed some light on some complicated work, I’ve pieced together a very incomplete introduction to some of the new things that HTML5 and related specifications might bring (remember: they aren’t finished or implemented everywhere).

Seven concrete areas of opportunity:

New semantics

HTML5 adds a bunch of new elements that might help publishers represent their content more clearly.

• section, article, header, and aside should be very useful wrappers for real-world content.
• Older web browsers will safely ignore unknown tags, but some ereaders that aren’t based on browsers may exhibit unexpected behavior. Also, typically, semantic tagging alone won’t drive a lot of sales.

Drawing

HTML5 adds a canvas element, which allows for (really snazzy) complex drawing and animation. Mark Pilgrim’s chapter in Dive into HTML5 offers not only the hairy details but also an example of how embedded HTML5 can change instructional materials (view it in Safari).

• This might replace some of what we think about when we say ‘Flash’ casually. It might allow for more interactive elements to be included in content.
• There are no easy-to-use tools that generate canvases — they must be coded by hand (Although it sounds like some basic tools are close).

Video

HTML5 adds a video element, which offers the first standards-based way to embed video in a web page. Similarly, an audio element has been added.

• Seamlessly embedable video may offer new ways of assembling and delivering multimedia content.
• Licensing for the actual encodings of the video content (the way that they’re compressed and stored) is unbelievably messed up and getting worse.

CSS3

A related set of updates to the Cascading Stylesheets specifications, CSS3, is often discussed alongside HTML5.

• CSS3 may bring a range of delightful updates for content creators, from better support for font-faces on the web to animations and transitions that may (along with canvas) allow alternatives to Flash for flashy stuff.
• Licensing solutions for fonts on the web are still in-progress. Authoring tools non-existent; support not complete across browsers.

Geolocation

Updates to a JavaScript API alongside HTML5 allow for users to reveal their physical location.

• There are probably a lot of opportunities to connect reading, readers, booksellers, and where people actually are.
• Very little support on desktops or more basic phones.

Offline Applications

A set of specifications related to HTML5 make it possible to create web applications that run without an active internet connection.

• Offline web applications can offer many of the features that standalone iPhone and Android Apps have and do not have to go through any App Store. They may also be more portable, as they don’t
  have to have a separate set of software for each platform. [Explicitly: This is the part of HTML5 that allows Ibis Reader to behave just like a normal App on iPhones, iPads, and Android phones.]
• Offline web applications are typically slower than standalone apps, although this will become less important as faster devices like the iPad become more common. Support on platforms is not uniform. Firefox doesn’t seem interested in supporting the current database specification.

Microdata

Another related specification that provides way of adding machine-readable annotations to content.

• One use of microdata might be to embed content licensing and other details inside pages.
• Yawn. Some of these features have been around in microformats for years and haven’t really taken off, although this could be very important for specialized content with regularly-structured content (cookbooks would be an easy example).

Sadly, HTML5 support (on any of the above) in Internet Explorer (even the unreleased IE9) is woefully incomplete and IE is still the most ubiquitous browser. Look at the number of red Xs in the right-hand columns of this support chart: http://www.findmebyip.com/litmus/

Does the above inspire you to try out some of these new opportunities now that you can pierce some of the HTML5 hype? Please let us know, as we’re actively seeking publishers and authors interested in innovating and experimenting with actual content to develop short- or long-form examples that take avantage of these new possibilities for enthralling, educating, and entertaining readers.

Want to explore in more detail? Both http://html5doctor.com & http://diveintohtml5.org have great details on the state of the above technologies.

Please enjoy my despair at the beginning as all the wireless-enabled readers interfere with the microphone.

It might be useful to follow along with the slides in the latter half of my talk, when the camera doesn’t show them.

This screenshot really says it all, but I just love this 1981 report on how news of the future will be transmitted.  I suspect Richard Halloran didn’t live long enough to see the “tele-paper” become ubiquitous.

Despite the tough economic times facing publishing in general and newspapers in particular, I’m pretty glad it doesn’t take 2 hours and cost $10 to browse nytimes.com every day.

Full YouTube video follows:

There are numerous in-jokes referencing the material in the books. You’ll get them if you’re smarter than me.

This Is Where We Live from 4th Estate on Vimeo.

Be sure to check out the making-of videos. In this day and age I was amazed that they used real time-lapse photography. It’s a nice nod to the physicality of the printed books themselves.