Just in the last few days, details emerged of two new JavaScript ePub readers, rePublish from Blaine Cook (@blaine) and JSEpub (screenshot) from August Lilleaas (@augustl). These two new readers join @liza’s epubjs, which will be a year old on Tuesday. An improved version of epubjs powers the ePub Zen Garden, which helps “dispel the myth that digital books can’t also be crafted works of visual design.”

Why are JavaScript ePub readers interesting? They’re interesting to me for three reasons:

• JavaScript is the most popular programming language in the world and it might be the best way to get more developers interested in creating and tweaking ePub readers.
• JavaScript ePub readers start challenging publishers, developers, and book readers to start thinking about what’s most important in delivering a compelling reading experience in a browser. We’ve spent a lot of time thinking about these choices while developing Ibis Reader, which will launch later this month, so I’m eager to see more opinions.
• Building a pure-JavaScript ePub reader requires unzipping in JavaScript, which had no open source implementations until just recently. August has written about and open sourced his critical breakthrough for unzipping files in JavaScript. [Edit: Oops! I was wrong about this one. See the comments for more details.]

Colin Hazlehurst has also published some impressive introductions, tutorials, and code for the .NET/C# crowd at his InsideEpub project and on his blog.

Do you know of other techies making waves with ePub? Please let us know!

(And if you’re one of those publishers who is looking for help, contact us.)

We’re starting to share early betas of the Ibis Reader mobile UI for iPhones, Nexus Ones, and other Android devices with a limited group of testers. If you’re interested in joining the beta program and testing on other phones, tablets, and laptops, please email info@ibisreader.com. You may be asked to sign a non-disclosure agreement.

More info on our upcoming ereader is available in our announcement post.

But I don’t think it’s minimal enough. If I could go back in time and be involved with ePub and its predecessors, here are the choices I’d make:

Make the NCX optional

Many books or book-like publications have no chapters. In this case ebook authors are forced to create useless one-item NCX files and invent fake chapter titles, like “Pages.” Reading systems should be able to rely on the opf:spine alone to order OPS documents without defined, named chapters.

Simplify the NCX

Good progress is being made in the EPUB Working Group towards clarifying and simplifying the NCX requirements. (Making playOrder optional is an especially useful step.)

But because the NCX is derived from the DAISY specification, there’s still some useless overlap, like the duplicated title. NCX is great for complex hierarchies, but I wish its features were simply a part of the OPF file, leaving only one file with publication-specific metadata.

Drop container.xml and replace with a required name and location for the OPF

I don’t personally understand the choice that was made here: there’s a file with a required location and name (META-INF/container.xml) whose sole purpose is to point to another file which may be named arbitrarily. Perhaps this is a historical artifact, but why not simply require there to be a content.opf file at the top level of the publication and be done with it?

(When combined with my first change, this would result in ePub requiring half as many files, which to me is a good thing.)

Support any valid form of XHTML

XHTML 1.1 was obviously a mistake, as it’s used (as far as I know) nowhere else, and is a dead-end as far as web technologies go. Few automated HTML tools generate it, and the changes from XHTML 1.0 are simply annoying rather than useful.

I’d prefer that ePub support XHTML 1.0, which is simply HTML 4.01 with an XML vocabulary. HTML 4 is the dominant form of HTML on the web (and will remain so for some time) and common automated tools like Tidy can clean up “street” HTML 4 into XHTML 1.0 quite well. Tidy won’t, however, produce XHTML 1.1.

I also don’t want to put an upper bound on the XHTML supported: XHTML 5 should also be okay, and the rules for a reading system which don’t support later tags should be the same “ignore and move on” that has worked well on the web.

Minimize or eliminate any ePub-specific styles and markup

I dislike the existence of special style properties like the oeb-* styles where equivalent CSS3 properties exist. I know CSS3 is a mess, but I’d rather use the same vocabulary as will eventually be found on the web. (This was discussed but isn’t happening.)

Support MathML as a first-class document type

This will happen eventually, but who knows the timeline. At least the fallback system means that MathML documents are currently allowable; it’s just more of a hassle and many people don’t realize it’s possible.

How would you change the spec if you could magically make it so?

Authorship

Two elements describe authorship of a work: dc:creator and dc:contributor. As you might guess, dc:creator designates the primary authorship, though there can be more than one.

Each author should be in their own element.. Don’t do this:

<dc:creator>James F. Smith, Thomas Jones</dc:creator>

Instead use:

<dc:creator>James F. Smith</dc:creator>
<dc:creator>Thomas Jones</dc:creator>

We recommend that anyone who had any kind of significant authorship role be marked as a dc:creator. Most reading systems use this element to discover or search for the author name, so ensure that individual authors in a multi-author work are so tagged.

A contributor designation is appropriate to describe individuals who had a non-primary role in the work. For example, if a multi-author work had multiple editors, or you want to capture additional staff like proofreaders, illustrators or the like, then use dc:contributor.

<dc:contributor>Alice Proofreader</dc:contributor>

File maker

ePub allows ebook creators to indicate how the values in these elements should be sorted. Name sorting, especially internationally, is a surprisingly difficult problem.

We recommend listing the author names in display order (usually First Middle Last) in the element text, and then setting opf:file-as to Last, First Middle, or whatever is appropriate in your source language.

<dc:creator opf:file-as="Smith, James F.">James F. Smith</dc:creator>
<dc:creator opf:file-as="Jones, Thomas">Thomas Jones</dc:creator>

Role playing

ePub extends Dublin Core by providing the opf:role attribute. This allows ebook creators to more precisely describe the contributions made by individuals listed in dc:creator and dc:contributor. It’s always a good idea to fill this out, even if not all reading systems display it:

<dc:creator opf:role="edt" opf:file-as="Smith, James F.">James F. Smith </dc:creator>
<dc:creator opf:role="edt" opf:file-as="Jones, Thomas">Thomas Jones</dc:creator>
<dc:creator opf:role="aut" opf:file-as="MacNeil, Lisa">Lisa MacNeil</dc:creator>

The recommended list of values for opf:role come from the MARC code list.

page-map

Adobe was motivated to provide a solution to this problem in Digital Editions before other reading systems, and they came up with a mechanism called page-map. The Adobe EPUB Best Practices Guide describes the issue:

There is no inherent linear navigation indicator which could be used for the same purpose that page number is used in the printed document world.

There is no way for an eBook to incorporate page number information for the printed edition of the same book.

Adobe developed an extension called page-map, documented in the sme Best Practices Guide, that provides a solution. To implement page-map, the creator includes a special page-map file in the ePub and references that file in the OPF metadata. This file gives a (page #) name to each pointer to a specific location within the content.

Here’s an example (note that many of the content files in the preface don’t span more than one page):

<page-map xmlns="http://www.idpf.org/2007/opf">
  <page name=""  href="strandedwithaspy_cov.html"/>
  <page name="" href="strandedwithaspy_intro.html"/>
  <page name="1" href="strandedwithaspy_fm01.html"/>
  <page name="2" href="strandedwithaspy_fm02.html"/>
  <page name="3" href="strandedwithaspy_tp01.html"/>
  <page name="5" href="strandedwithaspy_adc01.html"/>
  <page name="5" href="strandedwithaspy_ata01.html"/>
  <page name="6" href="strandedwithaspy_ded01.html"/>
  <page name="" href="strandedwithaspy_con01.html"/>
  <page name="7" href="strandedwithaspy_fm03.html"/>
  <page name="8" href="strandedwithaspy_fm03.html#page8"/>
  <page name="9" href="strandedwithaspy_fm03.html#page9"/>
  <page name="10" href="strandedwithaspy_ch01.html"/>
  <page name="11" href="strandedwithaspy_ch01.html#page11"/>
  <page name="12" href="strandedwithaspy_ch01.html#page12"/>

Pages in Action

How does page-map actually look in Digital Editions?

The catch? page-map is (intentionally) an extension to ePub and adding a page-map file to your ePub will make it invalid. On top of that, unless the reading systems is based on Adobe software (like Digital Editions and the Sony Reader), page-map will have no effect.

Pages in Digital Editions (without a page-map)

So, why do you always see these pages in Digital Editions, regardless? An un/fortunate feature of Digital Editions is the addition of the page-map-like display of pages, even if the ePub doesn’t include any page-map file. Here’s Adobe’s Best Practices again, describing how it chunks any content into a regular size, then labels each chunk a page:

When page map is not available in the document, Adobe Digital Editions will synthesize a page-map based on the document content. The approach used is the following:

Determine a compressed byte length of each resource which is referenced in the spine, subtracting any known encryption overhead (IV size)

Assume that there is a page for each 1024 bytes…

Some people want to see more of this, while others wish it could be turned off.

pageList

The NCX spec provides an alternative to the page-map extension, the pageList. This aptly named container for pagination information, provides a mechanism for giving a label (navLabel) to a point in the ePub (the pageTarget).

Here’s an example from an Internet Archive ePub:

  <pageList>
    <navLabel>
      <text>Pages</text>
    </navLabel>
    <pageTarget type="normal" id="pagetarget000006" value="6" playOrder="6">
      <navLabel>
        <text>6</text>
      </navLabel>
      <content src="part0000.html#page-6"/>
    </pageTarget>
    <pageTarget type="normal" id="pagetarget000007" value="7" playOrder="7">
      <navLabel>
        <text>7</text>
      </navLabel>
      <content src="part0000.html#page-7"/>
    </pageTarget>

The catch? While the OPF spec (part of ePub) says reading Systems must support NCX. (and mentions pageList offhandedly), it’s not at all clear how much of NCX is supposed to be supported. This ambiguity has meant that no reading systems (to my knowledge) have implemented support for pageList, so it’s appeal is primarily aesthetic. Unlike page-map, adding a pageList won’t make your ePub invalid. The EPUB Standards Maintenance Working Group is trying to clarify the NCX issue.

¹ What percentage of people actually have both media in front of them at the same time? It’s unquestionably better when trying to tell someone on the phone about the hilarious double-entendre on page 294, but I wonder how often this happens as well. Why not tell them to search for would you like to check my figures?. Finally, in STM & educational content, where this sort of thing might come up often, the headings are often numbered, and serve as a better guidepost (because they don’t rely on the same trim/pagination for various international editions, etc, etc).

0. Use InDesign CS4

It’s critically important to use the InDesign CS4 rather than an earlier version. CS4 included a number of updates and improvements to ePub output, so buy/upgrade at least one seat before starting to work on ePubs from InDesign rather than wasting your time on CS3 bugs that have been fixed. On top of that, make sure you’ve gotten all of the latest CS4 updates from Adobe (which have resolved some other ePub output bugs).

From your CS4 .indb file, select Export Book for Digital Editions…, an export filename, and start working through the three-paneled set of output options:

1. General

The first section of General options have to do with the metadata that InDesign will include in the OPF. Because too many ePubs have too little metadata, it’s worth always checking that option & filling out some basic metadata. Adding the name of the publisher (mapped to, surprise, <dc:publisher>) is an obvious choice. In addition, it’s worth filling out some of the fields in File→File Info… on the book’s Style Source .indd file to provide InDesign with a bit more metadata to work from.

The next set of choices around CSS is more flexible. Defined Styles should be your default choice, especially for internal testing. It (sometimes) provides a reasonable set of CSS without being bloated with overly-specific styles. While it sounds tempting to let InDesign obsessively copy every element of your design from print to CSS, it often degrades the experience on ePub readers other than Adobe Digital Editions and on tiny screens.

If you’re working on a series of titles with similar styles, you may find it worthwhile to write your own master CSS file that modifies or overrides what InDesign suggests and then manually update the CSS in the ePub with your master. In that case, either start with the CSS generated by Defined Styles & pick and choose what to override or choose Style Names Only, which will give you a CSS skeleton (all the selectors but no declarations).

Map to … Lists under Bullets and Numbers will get you the widest interoperability in most cases. If you have complex numbered lists, it can be hard to get list numbering to continue after a break (in XHTML 1.1 in general), but try to avoid the Convert to Text options unless you’re really in a bind. Map to Static Ordered Lists isn’t a good choice because it produces invalid ePub.

I don’t recommend Adobe’s font embedding option because it (intentionally) creates invalid ePubs. Liza offered some advice on how and when to include embedded fonts.

View eBook after Exporting just opens Adobe Digital Editions with the exported ePub file.

2. Images

The set of Images options probably deserves more testing because manuscript setups are so diverse. If you’ve setup your InDesign files with Links to web-ready images scaled to a reasonable size, choose Copy Images: Original. Because that usually isn’t the case, you’re probably better off choosing Optimized and experimenting with the Image Quality JPEG export options. I always choose JPEG over GIF or Automatic as the Image Conversion¹. Choose the highest Image Quality you can without making your ePub unreasonably large in file size.

3. Contents

The final set of options controls the schema of the OPS documents that InDesign will output (which will be either XHTML or DTBook). The control also provide some options for the declarative table of contents (NCX). While there are times when DTBook output is encouraged or required, XHTML-based ePubs are much more common and will work well in more ePub readers.

The choices surrounding the Table of Contents depend on the way you’ve setup your InDesign files, but I’ve had the best luck with Include InDesign TOC entries, which tries to map the heading hierarchy from your files into the ePub, and Suppress Automatic Entries for Documents. As ePub readers evolve, I’m finding they include increasingly attractive displays of rich NCX data, so it’s worth the time to test out this option.

¹ Although I always choose JPEG instead of Automatic, I was never able to make InDesign output a GIF under the Automatic option, so it may be a non-issue.

In ePub, you can’t.

I was surprised to discover that the subset of CSS supported by ePub only includes the rtl or ltr values in the CSS 2 direction property. Rendering vertical text is covered in the CSS3 Text Layout module, which is still in the editorial draft stage. But it does have some unusual browser support already.

There are multiple methods to specify directionality, but the simplest is the shorthand writing-mode property:

writing-mode: tb-rl; /* Chinese */
writing-mode: lr-tb; /* English */

Curiously, writing-mode is only supported by one browser: Internet Explorer. I doubt any other ePub rendering systems besides Bookworm viewed with IE will actually display such content properly (nor are they required to, as writing-mode isn’t in the specification).

Here’s a modified Project Gutenberg book with the English boilerplate and Chinese content rendered with different writing-mode values, as shown above:

I’d like to see future revisions of ePub specifically support writing-mode, even if the CSS Text Layout module isn’t fully ratified at that time. (I may add an issue to the EPUB Maintenance list if that seems warranted.)

This Project Gutenberg edition of Hunger by Knut Hamsun has text in Hebrew. It also has text in English. Ideally we’d like it to run in the correct direction for each language, but Bookworm shows it all running LTR (note the ragged-right margin):

Maybe it’s a Bookworm bug; if I open the XHTML file directly in my browser, I get the Hebrew text RTL, but the English text is aligned right, which looks weird:

Adobe Digital Editions doesn’t have a Hebrew font, and also the directionality is wrong:

(This is a case where font embedding is appropriate.)

This particular ebook is coded with the directionality marked using the dir attribute in the <body>:

<body dir="rtl" xml:lang="he">

But that’s not really accurate, since the first block of content is in English (it’s the Project Gutenberg boilerplate text). We need to mark up the ebook to properly describe the directionality of all parts of the text.

CSS provides the direction property for this. It’s pretty simple; you can have:

direction: ltr; /* Left to right */
direction: rtl; /* Right to left */

So a better way to mark up Hunger might be to define the default as rtl and then mark up any English text with a .english class:

body {
  direction:rtl;
}
.english {
  direction:ltr;
}

And add the .english class where necessary:

<div class="pgheader english" id="pgepubid00000"><p>
<br/>
<br/>The Project Gutenberg EBook of Hunger, by Knut Hamsun
<br/>
...</p></div>

Now Bookworm is happy:

And later on that same page…

(The fact that Bookworm didn’t respect the initial dir attribute is a bug, which I opened with a test case.)

The three ePub specifications (OPF, OPS, and OCF) include a lot of detail on ebook metadata and markup, but do not include a technique for describing covers. Despite this omission (a fix is being discussed), there are some widely adopted approaches for marking up covers that will make the cover display correctly in most (but not all) ePub reading systems.

Ensuring the widest interoperability for your cover means including an XHTML cover page (as the first “flow”), a cover image, and bits of OPF metadata to refer to these items using consistent names.

Cover as XHTML

To get started, here’s an extremely minimal XHTML cover page that serves only to reference the cover image. This file should be named cover.html:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <title>Cover</title>
    <style type="text/css"> img { max-width: 100%; } </style>
  </head>
  <body>
    <div id="cover-image">
      <img src="the_cover.jpg" alt="Title of this Thing, Second Edition"/>
    </div>
  </body>
</html>

Cover as image

Next, make sure the cover image itself, probably in JPEG (it’s a photo) or PNG (it isn’t) format, is scaled to less than 1000px in width and height. Choosing the perfect size for ebook images is still an open question given the growing range of screen sizes, but shrinking to 1000px should ensure that you don’t trigger a (historical) bug that crashed a popular mobile reading system by giving it a cover too large in size.

Covered in metadata

The real fun¡ is in the OPF file. First, make sure you have references to both the image and XHTML covers in your OPF manifest. The id attributes below are important to note:

<item id="cover"       href="cover.html"    media-type="application/xhtml+xml"/>
<item id="cover-image" href="the_cover.jpg" media-type="image/jpeg"/>

Using the same value as the id attribute for the XHTML cover, add an initial itemref to the spine element that refers to the cover page but is not linear. This tells reading systems (like ADE) what position the XHTML cover should have in the document (must be first to trigger cover behavior), but asks that it not be displayed to the reader as part of the “linear reading order of the publication.”

<spine toc="ncxtoc">
  <itemref idref="cover" linear="no"/>
  <itemref idref="normal-first-content"/>

The cover image also gets a special bit of metadata: a reference as a meta element inside the OPF metadata block. It’s critical that this use name="cover" as a signal to reading systems that would prefer to show covers as images rather than covers as XHTML:

<meta name="cover" content="cover-image"/>

Finally, it’s a good choice to include an OPF guide reference to your XHTML cover as a final hint to reading systems of what’s going on:

<guide>
  <reference href="cover.html" type="cover" title="Cover"/>

If you’ve gone to all this trouble to include a cover in your ebook, please don’t forget that readers expect it to be nice & in color.

Other approaches

Adobe’s EPUB Best Practices recommends a SVG-based approach to cover images. While it’s an interesting technique that can integrate well with Adobe Digital Editions, I haven’t found it to be widely supported.

It doesn’t tell you anything about the information encoded in those angle-brackets (really called elements). If the element is called <i>, does that mean “put this text in italics”? Or “indent”? Or even, “The following text is about me”?

In order to know what an XML document actually means, you need to know its schema. A schema is a kind of dictionary that defines all the names of the elements and to some extent, what they mean. It also describes the grammar of the document: for example, we might say that a <chapter> can be inside a <book> but not the other way around.

You can make up your own schema, and that’s often advisable when modeling a unique business practice. But books and other kinds of literature are well-understood, and there’s already been a huge amount of thought put into how to properly model them in XML. If you’re in digital publishing, these are the three schemas you’re most likely to come across when modeling written works:

DocBook

Originally designed for technical books, DocBook has emerged as an excellent general-purpose book schema. Because it’s in wide use, there are a lot of modern tools that understand it (including the excellent oXygen XML editor), and it’s trivial to generate other formats, including PDF and HTML, from a DocBook source.

Here’s a really simple DocBook document, in this case describing an article rather than a whole book:

<?xml version="1.0" encoding="utf-8"?>
<article xmlns="http://docbook.org/ns/docbook" version="5.0" xml:lang="en">
  <title>Sample article</title>
  <para>This is a very short article.</para>
</article>

TEI

The Text Encoding Initiative is also used to model textual works, but supports methods to encode historical and academic texts. TEI allows document authors to include revision history, extensive footnoting and cross-references, and provides a rich tagging mechanism for poetry, drama, and other forms of human literature.

TEI is frequently used in library digitization and archiving projects, and it can be used to encode texts that might seem otherwise impossible to render in XML. This project from the University of Maryland really shows off TEI’s power: In Transition: Selected Poems by the Baroness Elsa von Freytag-Loringhoven.

XHTML

In lots of ways, XHTML is wholly unsuited for use in book content. XHTML has almost no semantically-meaningful elements as applied to literature — there’s no built-in way to indicate a chapter, or footnotes, or dialogue versus description.

The advantage it does have is that it’s ubiquitous — thanks to the web — and many people who otherwise have no experience in XML or text encoding know at least a little HTML. Because of the web there are probably more works written in HTML today than in any other form in history.

By supplementing it with other forms of XML that do provide semantic structure, as in ePub, XHTML is demonstrably a useful and important commercial format.