Interactivity in EPUB using JavaScript, HTML5 and CSS3: BEA/IDPF video posted
by Liza Daly
I re-recorded my talk at IDPF Digital Book as a video. Getting the audio synced properly was no fun so I apologize for a few production issues.
Here’s the executive summary of the talk:
- You can add interactivity to an EPUB book using either the <object> or <script> elements.
- Currently object has the best support, especially using Flash in Adobe Digital Editions.
- The script element is specifically discouraged (though not disallowed) in EPUB 2.0.1, and no major reading system supports it. However, all browsers do!
- The primary advantage of interactivity using script is that the content creator has the ability to manipulate every part of the ebook: all of the text, its layout, even potentially the user interface of the reading system itself. With object you can apply interactivity to just an arbitrary rectangle.
- There is a great deal of interesting animation and even interactivity possible using CSS3, which is allowed in EPUB 2.0.1 and is supported by iBooks and other WebKit-based ereaders.
- Interactivity is also possible using JavaScript combined with the HTML5 canvas element, though canvas is not part of EPUB 2.0.1.
- With access to JavaScript on mobile devices, ebooks can potentially gain access to information about the reading device itself, such as the location of the reader in the physical world, or the device’s orientation, or even add photos and video from the device’s camera.
- Any ereader that allows JavaScript provides the ability for ebooks to access live data on the web.
- The primary blocker for extremely rich interactive ebooks is ereader support, but by embedding browser-based reading systems like Monocle, content creators can distribute such ebooks today.
- It remains to be worked out how to handle JavaScript in larger ereading systems both safely and with proper fallbacks to non-interactive content.
I’ll also be posting some of the example files used in the tutorial in a forthcoming post.
Comments
[...] (last week). In this talk she discusses a number of Epub features you might see in the future. ThreePress Consulting Blog View more videos from lizadaly. A couple of thoughts She mentions that none of the mainstream [...]
Liza,
Have you tried the improved Epub with calibre? I’d bet that all the features will work.
I’ll try it if you want. Send me an email.
you said:
> now, in the current version of .epub — .epub 2.0.1. —
> the specification says that,
> while the script element is valid,
> and can be included in .epub books,
> reading systems *should*not* execute it.
> now that’s a very technical “should not”. that means…
> it’s different from meaning “can not”.
> they’re discouraged from doing so, but they’re not prohibited.
actually, i believe it means that, although they are not prohibited,
they’re strongly discouraged from doing so. it’s a question of focus.
> and the specific language here _may_ change
> in the next revision of .epub,
> to make script support less frowned-upon.
> but that will have to be determined
> by the working group, in the coming months.
has the working group committed to a time-frame
for making this decision? because otherwise it
might take _years_ for them to act, not _months_,
as you state.
and could you address the concerns about security?
allowing script support might well open the door
for a whole host of nasty attacks by the black hats.
-bowerbird
The Working Group is meeting in person for two days in two weeks. At that point it will be more clear what the pace is going to be, but the next revision of EPUB is scheduled for essentially 1 year from now, so no, it won’t be multiple years.
As for security, browsers already do a good job of managing JavaScript but I’d look to the activity around HTML5 and @sandbox: http://blog.whatwg.org/whats-next-in-html-episode-2-sandbox for specific details on the issues involved with importing foreign scripting.
liza said:
> but the next revision of EPUB is scheduled for essentially
> 1 year from now, so no, it won’t be multiple years.
thanks for that information.
so even if the working group approved the change next week,
it won’t formally become part of the spec until a year from now?
and even then, some viewer-programs might not support it
right away? and some might not support it at all, especially
if it’s not actually _required_ by the spec?
-bowerbird
> so even if the working group approved the change next week,
> it won’t formally become part of the spec until a year from now?
Correct, though the change is purely a linguistic one rather than (say) adding support for the HTML5 video element. There’s no reason for reading systems to wait until next year to support something that’s already not explicitly prohibited.
It’s unlikely to be a requirement that reading systems support scripting in any case; I’d just like to see the language changed to something like MAY rather than SHOULD NOT.
Really interesting talk, Liza. Completely chuffed to see Monocle used in the video, thanks!
You know I take a different view of this stuff — in fact, in the development branch of Monocle on GitHub, script tags are stripped from the source. For us, this stuff is still in flux as we try to figure out how to do safe and fast embedding within the domain security model of browsers. Even without the possibility of books containing scripted behaviours, that’s a pretty thorny question. So the security question you mention as a ‘missing piece’: I would underline it, twice, with a red pen (although I must admit I’d scarcely considered it until Keith asked me about it in March — I’d always planned to strip scripts). I have read the various propositions regarding sandboxes and I can’t see any of them as a solution for this; I think they’re intended for another purpose entirely. And an implementation of any of them seems like it’s years off.
The other missing piece, which for me is probably more significant, is how ‘interactivity’ in epubs affects the interaction model of the reader. Both Ibis Reader and Monocle expect a pointing device or touchscreen, but that’s not the case for many readers. And even then, object-based Flash movies aren’t always clickable in Monocle right now, because a ‘contact event’ will usually turn the page. To get around this, iBooks makes only the margin a page-turning interface, but while that’s suitable for a large iPad screen, it’s not ideal for a small mobile screen. I worry that ceding too much interactive freedom to the book will make it harder for the reading system to have a really awesome interaction model, and right now most reading system interfaces are unfriendly enough.
Anyway, it’s all quite thought-provoking; I have much more to say, but perhaps I will formalise my thoughts a little first.
[...] making the geo demo featured in the Interactivity in EPUB talk available for download under the MIT [...]
hi Liza
Thanks for the great demo. videos are playing fine on ipad we are having problems with java scripts could please mial or post an example of epub with java script.
[...] in ebooks. Liza Daly of ThreePress Consulting has just posted the first demo based on the ideas in her talk. This one is a geolocation demo. The ebook will use Javascript to query your current location every [...]
[...] there seems to be one existing format that does a lot of that, and that’s ePub. Apparently, it has vast potential in terms of interactivity, through javascript, flash, CSS3, and HTML5. Again, I wish I had more technical knowledge so I [...]
You mentioned epub 2.0.1 a lot, and I’m pretty sure the maintenance group was supposed to finish in April. Has epub officially been updated to 2.0.1 yet? I haven’t seen any indication of that on the IDPF website (which is terrible, by the way). Thanks.
EPUB 2.0.1 was finished on schedule and no it has not been posted on the site yet (in part because the site itself is undergoing a major revision).
I see. Thanks a lot Liza.
Hi Liza
I have some questions about videos that could play good on the iPad.
What frame rate is the best ? Can I shoot HD and will that show on the device?
File format, does it have to be m4v?
What are my limits ?
and… What ePub editor is the best to deal with embedding the video
thank you
Jose, at this time there are no published specifications for video in iBooks as it’s not officially supported.
Hi Liza,
Great blog and info. Thanks to your example and a little editing in Adobe’s PDFXML I have managed to get our on-line magazine running as an e-book, complete with the videos we normally incorporate.
I’m reading it on my Ipod using iBooks. When the video plays, if I turn the Ipod through 90 degrees, iBook reverts to showing the magazine page after the video, rather than rotating the video player and showing the moving image. The iPod continues to play the actual video, but there appears to be no way of getting it back on the screen with the controls.
Regarding Jose’s question. The Ipod and the Ipad have technical specifications for their video, as long as you honour these specifications then videos will play if you incorporate them using the method you describe.
For the iPod they can be found here.
http://www.apple.com/ipodtouch/specs.html
For the iPad, they can be found here.
http://www.apple.com/ipad/specs/
Thanks
Martin
Thanks for the info, Martin — that’s interesting.
Apple has included a lot of features in iBooks (either intentionally or just as a consequence of using WebKit) that aren’t 100% there — video, audio, and interactivity are still at a sort-of-working state.
I’m glad that people are trying to use them and posting their results, since that’s a signal to Apple that these are valuable features. On the other hand, I don’t want anyone going nuts trying to find edge cases that do work reliably, since I’m sure these features are going to evolve rapidly.
[...] came across a fantastic blog post and presentation Liza Daly of Threepress about adding interactivity to ePub. This was really very interesting as I guess with my very superficial knowledge of it in the past I [...]
[...] came across a fantastic blog post and presentation byLiza Daly of Threepress about adding interactivity to ePub. This was really very interesting as with my very superficial knowledge of ePub in the past I [...]
[...] to a small box on the screen. Compared to web pages, this is a major limitation. (Thanks to the Threepress Consulting blog for this [...]
The question of JavaScript in EPUB is about vendor DRM and control
rather than any perceived security issues e.g. vendors don’t really
want you loading scripts to disable the inevitable advertising popups.
If you want to create interactive books then sooner or later you
require programmability — JavaScript is the obvious choice, it’s been
specifically designed to work safely with web pages. We need to move
away from proprietary readers, the sooner web browser vendors cotton on
to EPUB and start shipping a native ‘eBook Reader Mode’ the better.