Threepress Consulting blog » epub

Better support for Unicode characters in EPUB validator

Liza Daly — Tue, 02 Aug 2011 15:29:40 +0000

We’ve fixed a long-standing limitation in the online EPUB validator that prevented files with non-ASCII characters from being validated. (The bug was actually not in our code but at a lower level; I’ve posted a fix that simply works around it.)

If you had problems before receiving “Site error” when uploading an EPUB with accented characters, you should now be able to validate successfully.

Transforming NCX into EPUB 3 Navigation Documents

Keith Fahlgren — Tue, 14 Jun 2011 13:30:02 +0000

The first version of EPUB used the NCX format to describe an accessible, machine-readable Table of Contents. NCX had come to EPUB from DAISY’s DTBook standard and was a crucial navigational aide. Unfortunately, NCX was rarely understood and is not very human-readable. As part of the alignment with wider web standards, EPUB 3 has dropped the NCX format and encodes the same information in a specialization of XHTML, the EPUB Navigation Document.

Moving away from specialized ebook-only solutions was a big part of EPUB 3, so I am quite interested to see what these new EPUB Navigation Documents look like in the real world. It seemed like the easiest way to create a lot of them was to transform NCX files into the new format, so I’ve written an open-source (BSD) stylesheet to do just that:

ncx2end-0.1.xsl

Note: This ncx2end-0.1.xsl is alpha-quality software in the worst possible way—it probably won’t work correctly on your documents and is hard to use. It does produce apparently-valid output for the 100+ NCX files I had around, but I would not put much faith in that today. Enjoy!

If you find this tool useful enough to discover an error, please submit a bug report and make sure to attach your NCX file to improve the test suite.

EPUB 3 does strongly encourage Reading Systems to support EPUB (including NCX), but it would be nice to start seeing more experiments with EPUB 3 files to help encourage meaningful adoption.

Here’s an example of the input and output, just for show and tell. The EPUB Navigation Document looks normal and straightforward: that’s the point.

NCX in



  
    
    
    
    
  
  
    Moby-Dick
  
  
    Herman Melville
  
  
    
      
        Cover
      
      
    
    
      
        Title Page
      
      
    
    
      
        Original Transcriber’s Notes:
      
      
    
    
      
        ETYMOLOGY.
      
      
    
    
      
        EXTRACTS (Supplied by a Sub-Sub-Librarian).
      
      
    
    
      
        Chapter 1. Loomings.
      
      
    
    
      
        Chapter 2. The Carpet-Bag.
      
      
    
    
      
        Chapter 3. The Spouter-Inn.
      
      
    
    
      
        Chapter 4. The Counterpane.
      
      
    
    
      
        Chapter 5. Breakfast.
      
      
    
    
      
        Chapter 6. The Street.
      
      
    
    
      
        Chapter 7. The Chapel.
      
      
    
    
      
        Chapter 8. The Pulpit.
      
      
    
    
      
        Copyright Page

EPUB Navigation Document out


  
    Moby-Dick
  
  
    
      Contents
      
        
          Cover
        
        
          Title Page
        
        
          Original Transcriber’s Notes:
        
        
          ETYMOLOGY.
        
        
          EXTRACTS (Supplied by a Sub-Sub-Librarian).
        
        
          Chapter 1. Loomings.
        
        
          Chapter 2. The Carpet-Bag.
        
        
          Chapter 3. The Spouter-Inn.
        
        
          Chapter 4. The Counterpane.
        
        
          Chapter 5. Breakfast.
        
        
          Chapter 6. The Street.
        
        
          Chapter 7. The Chapel.
        
        
          Chapter 8. The Pulpit.
        
        
          Copyright Page

A more complicated book might use a lot more nesting, but it’s essentially turtles the whole way down:


  III. CSS Page Layout
  
    
      11. Introducing CSS Layout
      
        
          Types of Web Page Layouts
        
        
          How CSS Layout Works
          
            
              The Mighty <div> Tag
            
            
              Techniques for CSS Layout
            
          
        
        
          Layout Strategies
          
            
              Start with Your Content
            
            
              Mock Up Your Design
            
            
              Identify the Boxes
            
            
              Go with the Flow
            
            
              Remember Background Images
            
            
              Pieces of a Puzzle
            
            
              Layering Elements
            
            
              Don't Forget Margins and Padding
            
          
        
      
    
    
      12. Building Float-Based Layouts
      
        
          Applying Floats to Your Layouts
          ....

Thanks to Dave Cramer for producing one of the first EPUB 3 test files.

Validating EPUB 3 experiments

Keith Fahlgren — Mon, 13 Jun 2011 13:30:45 +0000

EPUB 3 is tricky to experiment with today. Like any brand-new specification, there aren’t many of the resources we often take for granted, from books to software to validation tools. However, if you’re already comfortable getting your hands dirty you can get meaningful validation for your EPUB 3 documents now. In the future, we’ll probably have a dedicated EPUB 3 validation tool (modeled somewhat on epubcheck, although with quite a few changes, I hope), but I’d like to start working today. This post outlines how.

Note: I’m going to give examples using a number of bare-metal tools available on Mac OS X. These are probably portable to Linux and even Windows if you were motivated, but I’m not going to explain how to install them or set them up (here or in the comments). Google is your friend.

To get started, download all of the EPUB 3 schemas (I put them in an epub30-schemas/ directory), install the absolute latest version of the RELAX NG validator Jing (jing-20091111/ for me), download the Schematron tools at iso-schematron-xslt1.zip is for XSLT1 processors (iso-schematron for me), and make sure you’ve got access to both xsltproc and java. Finally, save this as svrl_as_text.xsl:



  

  
    
  
  
    FAILURE: 
    
    :

Layout of the EPUB 3 schemas

All of the schemas for EPUB 3 are available as RELAX NG and sometimes Schematron. Each one has specific strengths, so we use both schemas whenever possible to get a complete list of all the validation issues. The EPUB 3 schemas are broken into separate files for each type of document inside an EPUB 3. You should notice that there is a RELAX NG Compact .rnc file for each type:

epub30-schemas/epub-nav-30.rnc        # EPUB Navigation Document
epub30-schemas/epub-svg-30.rnc        # SVG Content Documents
epub30-schemas/epub-xhtml-30.rnc      # XHTML Content Documents
epub30-schemas/media-overlay-30.rnc   # Media Overlay Documents
epub30-schemas/ocf-container-30.rnc   # META-INF/container.xml
epub30-schemas/ocf-encryption-30.rnc  # META-INF/encryption.xml
epub30-schemas/ocf-signatures-30.rnc  # META-INF/signatures.xml
epub30-schemas/package-30.rnc         # Package Documents

Unsurprisingly, you use a RELAX NG validator with epub30-schemas/media-overlay-30.rnc to validate a Media Overlay document.

A few of these documents also have a Schematron schema with the same prefix but ending with .sch, which is used to express other requirements that aren’t possible in RELAX NG:

epub30-schemas/epub-nav-30.sch
epub30-schemas/epub-svg-30.sch
epub30-schemas/epub-xhtml-30.sch
epub30-schemas/media-overlay-30.sch
epub30-schemas/package-30.sch

There are some standalone Schematron validators, but we’re actually going to roll our own tool for more human-readable output.

There’s a third file extension too, .nvdl, which is short for Namespace-based Validation Dispatching Language. These files are supposed to wrap these two schemas together for unified validation tools, but there isn’t good software support for NVDL today. Ignore the .nvdl files for now.

What to validate

I’m currently interested in the EPUB Navigation Document, a reformulation of EPUB’s NCX document as XHTML, so these are the examples we’ll use. However, this approach should work for any of the other document types if you go through the same setup.

Here is a valid, if short, EPUB Navigation Document:



  
    EPUB Navigation Document Example (Good)
    
  
  
    
      
        Contents
      
      
        
          
            Introduction
          
          
            Chapter 1
          
          
            Copyright Page
          
        
      
      
        Guide
        
          
            Table of Contents
          
          
            Begin Reading
          
          
            Copyright Page

And here is one with a few errors that should be reported as invalid:




  
    EPUB Navigation Document Example (Bad)
    
  
  
    
      
        Contents
      
      
        
        
          
            Introduction
          
          
            Chapter 1
            
            
          
          
            Copyright Page
          
        
      
      
        Guide
        
          
            Table of Contents
          
          
            Begin Reading
          
          
            Copyright Page

RELAX NG validation with Jing

Once you’ve got jing setup, it’s pretty straightforward to validate our files (above) against the appropriate .rnc. We’ll be using the epub30-schemas/epub-nav-30.rnc schema.

When you run jing against a file and it passes, you get no output (good) and an exit code of 0. I’m calling jing as java -jar jing-20091111/bin/jing.jar, passing the -c flag to tell it to expect a Compact version of RELAX NG, and then the schema filename followed by the filename of the document to validate:

$ java -jar jing-20091111/bin/jing.jar -c epub30-schemas/epub-nav-30.rnc good.nav.html
$ echo $? # What was the exit code?
0

Unlike earlier versions of jing, the latest versions have much clearer error reports on invalid documents (we also saw this improvement in epubcheck 1.2 thanks to George Bina from oXygen):

$ java -jar jing-20091111/bin/jing.jar -c epub30-schemas/epub-nav-30.rnc bad.nav.html
bad.nav.html:21:20: error: element "span" not allowed here; expected the element end-tag or element "ol"

…and the exit code is not 0, just as expected:

$ echo $?
1

We can take apart that first bit out output, bad.nav.html:21:20, to know which file had the error (we could run it on many at once) and also the line number (21) and character on that line (20). Line 21 has just what we would expect given the error message (a span instead of another ol or the end of this one), but for other errors it can be quite illuminating:

$ sed -n -e 21p bad.nav.html

Note: For really large documents, you may get an java.lang.OutOfMemoryError or other exception. Find out how to give jing more “heap space”.

Schematron validation with XSLT

Validating the Schematron schemas is a little more involved, but it catches some validation errors than jing and RELAX NG just cannot find. First we turn the .sch file into a re-usable XSLT stylesheet that produces Schematron Validation Report Language (SVRL). We can then run that stylesheet on any document of that type inside an EPUB 3 file to produce SVRL, which we then transform into something human-readable.

First we create our validation stylesheet, epub-nav-30.sch.xsl, from the epub30-schemas/epub-nav-30.sch Schematron schema:

$ xsltproc -o epub-nav-30.sch.xsl iso-schematron/iso_svrl_for_xslt1.xsl epub30-schemas/epub-nav-30.sch

Now we can use epub-nav-30.sch.xsl on any EPUB Navigation Document:

$ xsltproc epub-nav-30.sch.xsl bad.nav.html



  
  
  
  
  
   ... many more lines ...

…but we rarely want to read the SVRL it outputs directly (although sometimes it is worth it for the extra detail it contains), so we need to send it through another stylesheet (svrl_as_text.xsl from above) to get a human-readable output:

$ xsltproc epub-nav-30.sch.xsl bad.nav.html | xsltproc svrl_as_text.xsl -
FAILURE: failed-assert: Exactly one 'toc' nav element must be present
FAILURE: failed-assert: Spans within nav elements must contain text
FAILURE: failed-assert: nav elements other than 'toc', 'page-list' and 'landmarks' must contain a heading as the first child

These are completely new issues that jing could not catch. Note that the issue about the span is actually distinct from the one above, which said it was in the wrong place, whereas this says that it has the wrong content (none at all, in fact).

Unlike jing, we don’t get meaningful exit codes. Although that is not too hard to add, it’s slight tricky to get all of the errors and exit codes rather than just exiting on the first one, which can make you think your document is less invalid than it really is. We still get no output for valid documents:

$ xsltproc epub-nav-30.sch.xsl good.nav.html | xsltproc svrl_as_text.xsl -
# no output

I’m certain to have made lots of mistakes in the examples above. If you spot some, please let me know in the comments and I’ll correct the post.

Copy Editors in EPUB 3

Keith Fahlgren — Tue, 07 Jun 2011 13:37:11 +0000

The new flexibility in metadata for EPUB 3 is a strength. Instead of locking down the set of permitted metadata schemes and schemas (like EPUB), it allows us to use a range of definitions. While this flexibility comes with some dangers (see the discussion in the comments here about what to do when there isn’t yet a good external definition), I’m generally not worried about giving the masses power tools (most folks are smarter than you think).

In particular, I’ve always been annoyed that in EPUB’s metadata I could not give proper, specific credit to the many people that make a contribution to a title, even if they are not on the cover. Here’s the cover of Responsive Web Design:

but here’s the set of contributors in the EPUB’s metadata:

Ethan Marcotte
Jeffrey Zeldman
Jason Santa Maria
Mandy Brown
Dan Cederholm
Krista Stevens
Neil Egan
Threepress Consulting Inc.

As you can see, there are a lot of people credited in the metadata, and we’ve credited each one as a Dublin Core contributor with a reference to a MARC Relator code. Specifying these contributors using these external definitions (both of which have huge amounts of both consensus and uptake behind them) means that a lot of software can understand it mechanically (including software and people outside of the US or English-speaking world, with some limitations). However, consensus sometimes leaves us impoverished: in this case there are three people all labeled as Editor, but the copyright page from the book itself shows us their true roles:

Publisher: Jeffrey Zeldman
Designer: Jason Santa Maria
Editor: Mandy Brown
Technical Editor: Dan Cederholm
Copyeditor: Krista Stevens
Compositor: Neil Egan

With the ability in EPUB 3 to reference a different source for these editor definitions, we can actually differentiate Dan Cederholm’s role as Technical Editor and Krista Stevens’ as Copy Editor:

Dan Cederholm
Cederholm, Dan
technicaleditor
http://www.docbook.org/xml/5.0/rng/docbook.rng

Krista Stevens
Stevens, Krista
copyeditor
http://www.docbook.org/xml/5.0/rng/docbook.rng

In the above example, we’ve described a few different things about these folks using the new EPUB 3 metadata syntax. First, we’ve referred to them as a contributor:

Dan Cederholm

Next, we’ve specified how to sort their names with a file-as property:

Cederholm, Dan

Finally, we’ve accomplished our goal of saying that Dan Cederholm’s contribution role was as a Technical Editor, as defined by DocBook 5.0:

technicaleditor
http://www.docbook.org/xml/5.0/rng/docbook.rng

For some reason, Copy Editors and Technical Editors have long been defined by DocBook, but are omitted from the ONIX Contributor codes, the MARC Relator codes, the PRISM codes, the Z39.86-201x Structural Semantics Vocabulary and every other metadata standard I could find.

Note/Warning: If you were going to make an EPUB 3 title today, you’d probably include both the “old” style EPUB metadata for these contributors AND the new EPUB 3 metadata (with the prefer attribute for each set). This would give you the benefit of both backward-compatibility with existing EPUB reading systems and better metadata for the future when EPUB 3 reading systems are more commonplace. Eventually you may just need the EPUB 3 version, and that’s the world I’m talking about here (partially as a small attempt to get us there faster).

Can an author create an EPUB using normal tools? Part 2: Scrivener

Liza Daly — Thu, 02 Jun 2011 15:59:41 +0000

This is part two of a series on using author-friendly word-processing tools with native EPUB export. Part 1 was about Apple Pages.

Part two: Scrivener

I first became aware of Scrivener’s support for EPUB export in November of 2010, when they released a special edition for NaNoWriMo (and the less said about my novel-writing experiment, the better). EPUB export is now available in the latest major release.

Though Scrivener is primarily a Mac application, there are beta versions for Windows and Linux. I have no idea if they support EPUB export — please comment if you know either way!

The sample document

Scrivener’s interface can be intimidating if you have only worked with relatively straightforward word processors. It’s less of a document editor than a book authoring platform (or the way I would look at it, it’s Visual Studio, not emacs).

For this test I used the “Manuscript” style, which provides many of the utilities that an author of a novel-length book might want, rather than starting with an entirely blank document (analogous to using the EPUB sample document in Pages). For one thing, the Manuscript layout already provides obvious support for structured blocks like chapters and parts. In fact Scrivener segments documents even further into “scenes” which can be relocated easily through the document. Programmers love structured content!

As in the previous post, I set up the sample document to include all of the obvious stylistic elements that I wanted to test. I didn’t attempt to exactly re-create the Pages sample as that would be soul-crushing.

Finding the EPUB export feature can be challenging; it’s actually listed as “Compile.”

Export options

No complaints here. There are dozens of options for EPUB configuration from the menus.

EPUB-specific output options:

Select which content segments are included in the book, and in what order:

My favorite pane. Look at all that metadata!

Output

Scrivener’s output is variable in quality. Some things are nice. It’s valid, for one thing, and I was able to produce this valid output without any special handling — I didn’t change any of the menu items beyond adding the metadata, which indeed trickles down nicely into the OPF file:

       Creating an EPUB sample document
        urn:uuid:A54DCD13-A6E0-4ACA-8B31-4E0EBA0624EF
        en
        Liza Daly
        Non-fiction
        An EPUB test of Scrivener's output
        Threepress Consulting Inc.
        http://creativecommons.org/licenses/by-sa/2.0/deed.en
        2011-05-29

Whitespace is exported as significant. While this makes the web developer in me cringe (I’d rather see CSS margins used here), it’s acceptable, especially since whitespace is semantically relevant in fiction, as when used for scene breaks:

Hello world!

I have lists and tables!

Unlike Pages, lists are generated as the correct list elements:


  Numbered lists are numerous.
  We often have more than one item.


  Make love
  Not bullets

Tables are fine, though I’d like for these tools to have a “table header” style that could ultimately map to th elements. This is a little more readable than the Pages output.

Headers are curiously not optimal:

First, a header.
A subheader.

Why the inline margins and font? Why the b? And like Pages, no apparent way to generate strong and em:

Back to body text. But with strong and emphasis?

Drag-and-drop images and internal and external hyperlinking all worked well:

Hello dog.

Inline URL: http://www.flickr.com/photos/beinecke_library/5166412915/in/set-72157625240109163/

A hyperlink to Chapter One. A hyperlink to a website.

I pulled the image out of the Pages EPUB output – Pages renamed it to droppedImage.png, and Scrivener retained the filename, which is a nice touch.

Room for improvement

Scrivener seems to generate one CSS file for each XHTML file too many CSS files by default. Uncool! This could potentially generate huge unwieldy EPUB books. There’s no reason to have more than one CSS file in an automatically-converted document, and the styles should be normalized across XHTML files.
Like Pages, there was no obvious way to add alt text values to images.
Also like Pages, I’d like to be able to customize the elements and class names in the outputted XHTML using simple menus.

A pleasant surprise to me was that Scrivener has a lot of options for power-users under the hood, including integration with version control tools like Subversion, use of a text-based markup format for better serialization to other formats, or low-level customization of the CSS or even XSLT: Scrivener advanced topics. This means it’s potentially a better choice for an ambitious digital-only publisher; you can start with the basic WYSIWYG layer, and gradually customize the output by diving into the lower level. This will be especially true once Scrivener is fully cross-platform.

Recommended with reservations (but mainly because of the CSS file issue).

Download the sample EPUB file and Scrivener file (unzip first). (Released under a Creative Commons Attribution license)

Can an author create an EPUB file using normal tools? Part 1: Pages

Liza Daly — Wed, 01 Jun 2011 14:00:40 +0000

Yes, but it may require a Mac.

The IDPF board met on the last day of the Digital Book 2011 conference at Book Expo America. One of our topics for discussion was what the IDPF as an organization should do to further the adoption of EPUB. I brought up an issue that’s been concerning me for some time: the lack of digital-native authoring tools aimed at authors, not publishers.

If publishers are struggling to produce high-quality EPUB files either via InDesign, XML workflows, or strategic outsourcing, authors are in an even worse place. This is especially true for authors with an ambition to self-publish, or to start a micro-publishing outfit, and yet still retain some creative control over the look of their digital product. InDesign (especially CS5.5) is a great solution for small- to medium-sized publishers who produce both print and digital books, but its feature set is inappropriate for digital-native publishing, and its price and complexity are unsuited for self-publishers.

I’m aware of two document creation tools right now that have native EPUB support and are available for my platform, Mac OS X: Pages, and Scrivener. (The only product I know of on the Windows side is Atlantis. Linux users have to make do with plugins for OpenOffice — judging from the comments in the issue tracker, EPUB export is not a priority, to say the least.)

This post will cover Apple Pages. A subsequent post covers outputting EPUB with Scrivener.

Pages

Apple’s Pages was the first major commercial word processor to include EPUB export. I reviewed the initial EPUB support in August 2010, but it’s been through some updates since then, and I wanted to dive into the semantics of the outputted code more closely.

The sample document

I started with the Apple-provided EPUB template (more on that later) and added a number of new elements and semantic tests. In particular, I added:

Chapters and headings
Emphasis and strong text (rendered in Pages as italics and bold)
Numbered and unnumbered lists
Hyperlinks both internal to the document and external to the web
Inline images (by dragging and dropping)
A cover page with an image
All available metadata in the export pane
Tables

In all cases, I used only styles available in the style drawer; I did not change any font sizes or font weight via the toolbar buttons.

The EPUB output

As in my previous test, this produced a valid EPUB 2.0.1 document according to EPUBCheck 1.1. Hooray!

Headers and subheaders

The semantics are much-improved from my first test. Paragraphs are now wrapped in

elements, for example, and headers are headers:

  
    
      Chapter Two: The Chaptering
      This chapter has an introduction. Hello!
      I’m a subchapter or section under that. 
      Don’t hold it against me. I just have a lot of things to say.

However, the white-space:pre-wrap style is curious: the property is meant to specify that whitespace inside the XHTML is significant, meaning that the ereader/web browser should retain it. That is emphatically not a best practice in general text; on the other hand, there was no whitespace in the output at all, so I’m unsure of its purpose. If I were post-processing this EPUB file, I would remove that style.

I used the “Chapter” style to generate the chapter heading. This header should be an h1 rather than an h3, but at least the subheading is also a header and one step down.

Go boldly

I completely failed to find a way to output strong and em rather than b or i.

Listless

I used the list styles provided in the template, but these are not the lists you’re looking for:

      Chapter Three: Lists
      Reasons why people love lists, in order.
      1.Lists are neat.
      2.It’s cool to let the computer fill in numbering.
      3.Yessir.
      Other reasons that people like lists, in no particular order:
      •Sometimes they have bullets
      •Not real bullets.
      •Those are scary.

This must be fixed.

Tables

A little verbose markup-wise, but basically fine:

Not sure how to make headers

Without just using colors

Not semantic

....

Images, covers, and links

Creating an image is as easy as dragging it in. I’m not sure if it’s possible to add alt text to the image — I believe document creation tools should prompt users to add descriptive text by default.

Only images styled as “inline” will be exported; Pages will warn you that the image was discarded if it had a floating or fixed style. I tried to select a page with an inline image as the cover page but Pages gave me a warning that it was being discarded. Then it actually shows up in iBooks anyway.

It would be nice if the original filename were preserved (it was not “droppedImage.png”), and the empty style attribute should be discarded on output.

Chapter Five: Hyperlinks

This is an internal link to chapter one. This is an external link to photos of kittens.

The empty span here is for the purpose of creating a back-link. A similar one was auto-added to Chapter One. Adding an internal hyperlink requires an initial step of creating a Pages “bookmark”, and then linking to that bookmark, which was a little confusing; I should be able to target any point in the document using the hyperlink feature.

I didn’t test HTML5 video output, but I’ve been told that video can be successfully embedded and output such that the video will work in iBooks (it will use HTML5 tagging).

Metadata

Both the OPF and the NCX were perfectly well-formed. The EPUB export dialog should optionally request richer metadata than the current list of author/title/subject, though.

The dreaded sample document

The EPUB export function is next to useless on large documents unless you start with the sample template, or import its styles later and tediously update yours to match. The EPUB styles are completely opaque — I have no idea why they have magical properties, or what I would do to my own styles to emulate them. Since the Pages native file format is binary, there’s nothing for me to inspect to reverse-engineer the styling. The Pages file format is zipped XML, so it may be possible to inspect it directly — thanks Steve!

The native header/paragraph/list styles in the blank template should output useful semantics in the XHTML. It is unacceptable to force users to import an external document to produce a half-decent EPUB file. At the very least, an EPUB-friendly template should be one of the default choices available when creating a new document.

Improvements

The list styles should generate lists. They should be ordered or unordered as appropriate to the style.
EPUBs should be importable as well as exportable. It’s understandable that they won’t magically re-constitute into the original Pages document, but a conversion pipeline is entirely reasonable.
It should be possible to export chunked EPUBS (with multiple XHTML chapters) without having to use the sample template.
It should be possible for a power user to understand how to create styles that will have specific behaviors.
It should be possible to customize the XHTML serialization (“I want the style named ’strong’ to output strong elements with the classname ‘foo’”).
There should be much more metadata allowable in the OPF file.
Images should require or at least prompt for alt attribute values.
Bold and italic buttons should output strong and em with the appropriate CSS styling in all cases. I would say this is actually true of any EPUB output tools — it’s unreasonable to ask users to create named styles (as in InDesign) when those tempting bold and italic buttons are available.

I don’t expect Windows/Linux versions of Pages to ever exist, which means that Pages will remain a marginal tool in the publishing ecosystem, but it’s perfectly adequate for an individual Mac-only user.

Download the sample EPUB file and sample Pages document, (released under a Creative Commons Attribution license).

Subtitles in EPUB 3

Keith Fahlgren — Tue, 31 May 2011 13:15:57 +0000

Update: I should have been clearer that the “bare” use of subtitle without referencing a scheme is really suboptimal. Using already-defined properties/definitions and referencing them explicitly (like the ONIX codelist example below) is a much better technique, but I did not come up with a compelling property for subtitle, hence the call for better options. See the comments below for a good discussion of those options.

There’s also a Bulgarian translation.

In a previous post, Thomas Rasche asked:

How are book subtitles best dealt with in Epub3? They could be included with a meta data property, but is there a recommended way so that a reader recognizes them, to display them sensibly (eg subtitles display in smaller font under the main title etc)?

EPUB 3’s metadata model is powerful and flexible. It is a big improvement over the limited metadata permissible in EPUB, but there are three real costs associated with EPUB 3’s metadata:

Because of backward compatibility with EPUB, metadata in EPUB 3 duplicates at least three required metadata elements in both an old and new style: title, identifier, and language (probaby creator too).
Greater flexibility in metadata schemas means that EPUB 3 metadata sometimes seems quite verbose (but you’re generating your OPF using software with robust metadata stores anyway, so this isn’t much of a complaint, right?).
Different metadata schemas describe the same thing with different words, so there are often many ways to accomplish the same thing and we don’t yet know what the convention will be.

That last issue of choosing a schema is the central issue in our subtitle problem. The EPUB Publications 3.0 specification provides a number of examples on how to include specific metadata but doesn’t exactly prescribe how a subtitle should be included. Instead, we’ll include two titles in a row, declare one as the subtitle, and indicate a display order.

Here’s the markup, using Dreams from My Father: A Story of Race and Inheritance as the example:


  Dreams from My Father
  Dreams from My Father
  1

  A Story of Race and Inheritance
  A Story of Race and Inheritance
  subtitle
  2
  …

What’s going on here?

In this markup, we do a range of things. We state both the title and subtitle using the legacy EPUB markup for backward compatibility and we do it in order, which is important for EPUB, but we also tell EPUB 3 reading systems to use the new syntax with the prefer attribute:

  prefer="dcterms-title">Dreams from My Father
  …
  prefer="dcterms-subtitle">A Story of Race and Inheritance

We also include the titles using the new markup for EPUB 3:

  Dreams from My Father
  …
  A Story of Race and Inheritance

Next we specify that the subtitle, is, in fact, a title-type subtitle. We do this by adding another meta element that refers to the subtitle with the about attribute:

  property="title-type">subtitle

Finally, we specify the display order of the two titles with meta elements and display-seq

  property="display-seq">1
  …
  property="display-seq">2

Other options

EPUB 3 is new enough that we don’t know which practices will become adopted by content creators and reading systems (so we don’t know about a smaller font, like the original question). The subtitle markup suggested above is a decent approach, but other options might be using http://open.vocab.org/terms/subtitle (not widely used in publishing today) or ONIX’s element (very hard to reference cleanly).

If you’ve got a better suggestion for the markup above please leave a comment and I’ll update the post if a winner emerges.

Colons don’t make subtitles

While it may seem tempting to split title and subtitle on a :, don’t, ever: slide:ology.

If there are specific EPUB 3 questions you have that you’d like to see us write about, please leave a note in the comments—Thanks!

Subtitles in EPUB 3 by Keith Fahlgren is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License.

Covers in EPUB 3

Keith Fahlgren — Thu, 26 May 2011 13:36:31 +0000

Now that EPUB 3 is an IDPF Proposed Specification it seems like a good time to update some of our EPUB blog posts. One of the most popular was Best practices in ePub cover images, which helped clarify how to include a cover image for most ebook readers. While cover image markup wasn’t specified in EPUB, I’m happy to report that EPUB 3 clearly explains how to include a dedicated cover image.

Note: If you’re new to the EPUB 3 specifications, the best place to get a clear introduction to the four parts of the specification is the EPUB 3 Overview. Marking a specific image is a metadata problem at the publication level, so we’ll be referring to the EPUB Publications 3.0 specification. As expected, it includes “Publication metadata — mechanisms for including and/or referencing metadata applicable to the entire Publication and particular resources within it.”

Without further delay, here is the markup to mark an image as the cover in EPUB 3:

properties="cover-image"
      href="the_cover.jpg"
      media-type="image/jpeg"/>

The new properties attribute is added to an otherwise “normal” OPF item and is all that’s required to mark an image as the cover. While including a cover image is still optional in EPUB 3, you may only mark one image as the cover.

Most of the metadata improvements in EPUB 3 are based on this properties attribute. Its uses range from noting that content includes MathML or JavaScript to indicating the type of publication.

If there are specific EPUB 3 topics you’d like to see us write about, please leave a note in the comments—Thanks!

Using Ibis Reader and Apple’s VoiceOver

Keith Fahlgren — Fri, 11 Mar 2011 15:30:54 +0000

Liza and I have long been interested in making ebooks more accessibile. Both Bookworm and Ibis Reader reflect that goal (with limited success—accessibility can almost always be improved). However, our focus has always been on improving the web version of Ibis Reader rather than the installable HTML5 App that many people use on their iPhones, iPads, and (increasingly) Android devices. Some of our users wanted to try out the installable App, which has the benefit of working offline, using Apple’s VoiceOver on their iOS device. Happily, a recent performance update (switching to columns-based layout) means that Ibis Reader is much more usable in VoiceOver, so I wanted to document how to use the two systems together.

Note: Many users may still prefer ebook applications that offer a more straightforward interface to VoiceOver than what I’ve described—I’m just happy we have a starting point from which to improve.

[Update: The reading mode has just been updated to have more useful “Next” and “Previous” links, as I describe in Reading, below.]

As a sighted user, I’d love to get feedback on how to make these instructions clearer and how we can improve the accessibility of Ibis Reader.

VoiceOver and iOS

Apple’s VoiceOver screen reader is available on both the Mac and on iOS devices. There’s a good web overview and a much longer Accessibility section in the iPhone User Guide [PDF].

You’ll need to turn on and get used to the VoiceOver gestures before trying the steps below.

Installation

Start the installation process by opening the Mobile Safari browser as usual. Next, visit the URL m.ibisreader.com, where you’ll immediately be prompted to “Increase Database Size?”. Select the “Increase” button (on the right) to give Ibis Reader some space for storing ebooks on your device. Next, you’ll install it.

“Installing” an HTML5 App like Ibis Reader is mostly just getting it onto your Home Screen:

Select the “Utilities” (VoiceOver’s name for it) button from the center of the bottom row of buttons. It used to be a plus symbol in previous iOS releases and is now a box with an arrow to the right.
Select the second button in the Utilities menu, “Add to Home Screen”. This will give you an “Add to Home” form with an Ibis Reader icon (just like any other App from the App Store) and an editable name (the default is “Ibis Reader”, unsurprisingly).
Finally, select the “Add” button from the top right (it’s one left-swipe “before” the editable name, where the focus starts).

You’ve now got an Ibis Reader icon on whatever page of Apps iOS fancifully decides to place it. Discover and open it in the usual way.

When it opens for the very first time ever, Ibis Reader has a brief introduction screen that pops up. You’ll need to follow the “Start Reading” link, which is the third or fourth item on the page. After that you’ll be sent to the “Get Books” section, which I describe below in Finding Books, but I’d actually sign in or register first.

When we first launched Ibis Reader, we were focused on making it as straightforward as possible to get started reading with a minimum of hoops to jump through. That means that we let people browse for new books before logging in and made the login and register screens exactly the same (they’re both tiny, with just an email and password field). However, if you’re just starting with the App, go ahead and follow the “Sign in” link first (it’s the link in the top right corner of the screen or the third item from the top of the screen). Fill out the fields as normal and you’ll be sent to the My Books section (which will be empty).

Basic Interaction

Ibis Reader is, at its heart, a web application. That means that you should be able to use most of the VoiceOver gestures and navigation techniques you’ve learned from browsing the web in Mobile Safari inside the Ibis Reader App. However, there are few parts of the Ibis Reader interface that are based on “touch” events rather than HTML links, so you’ll need to get used to the double-tap+hold gesture (accompanied by a rising series of three tones), which switches you between VoiceOver’s gesture mode and the “native” one. [Update: We’ve improved the UI to make these “touch” events unnecessary for VoiceOver users.]

For much of the App, we have a header bar with three relatively big buttons showing the navigation choices. On an iPhone-sized screen, these take up roughly one third of the screen each, but on an iPad two are huddled together on the left with the third on the right. The navigation choices are:

My Books, Get Books, or Sign out (or in): this is the Home of the App, where you browse your library, find new titles, and where you go if you select “Home” (surprise!) from another part of the interface
Close, Read This Book: these are the options when you’re looking for new books
Home, Book Info, or Settings: this is the Reading Info section of the app, which shows your options while reading a book (this typically won’t be used by VoiceOver users)
Back (top left corner): this is how you return from the Book Info screen to the book text

If you’ve started reading a book, Ibis Reader will remember where you left off and open right to that page when you start the App the next time. Unfortunately, because VoiceOver users won’t need to “turn” any pages, you’ll start back at the beginning of the section each time. I’m not sure what the best workaround is for this.

Finding Books

In the “Home” section of the App, the second item on the page is “Get Books” (roughly in the middle of the top), where you can browse popular Public Domain and Original content from Feedbooks. If you haven’t used Ibis Reader before, browse or search the titles in “Feedbooks: Popular Public Domain” (the first item on the list on the page) to find something interesting. VoiceOver will read you the title and author of each book. When you’ve found one you want, select the one you want (it’ll probably repeat “[the title], link”. If you have VoiceOver read from the top of the page, it will give you the books details and a description (if present). To download the book and start reading it, select either “Read This Book” link (there’s one before and after the description). After selecting it, be prepared to wait until it has downloaded an loaded (might be a while depending on your connection. VoiceOver should read the first page when it’s ready (maybe “Cover image” or similar). Now on to the description of Reading.

If you’ve already used the (more accessibile) web version of Ibis Reader at ibisreader.com, you’ll have titles in your library in the cloud that you can download to your device by following the “Browse your Online Bookshelf” link (about the fourth element on the page) from the My Books section. Navigating your uploaded books is exactly the same as navigating the Feedbooks content (they’re both delivered as OPDS Catalogs under the hood.)

Reading

The experience of reading works great in Ibis Reader. The normal “Two-finger flick up: Read all from the top of the screen.” gesture works well and will read a whole section of text without having to turn the page (although others looking at your screen may be confused). Unfortunately, switching between sections of the text isn’t as accessible as it could be, as it relies on both a very good TOC from the publisher inside the EPUB itself and some “native” tap gestures. [Update: Switching between sections is easy too: just choose one of the “Next” or “Previous” links from the very top or bottom of a section (these links are only visible to screen readers). Remember: You can use a “four-finger tap at top of screen” to select the first item on the page, like the “Previous” button, or a “four-finger tap at bottom of screen” to select the last item on the page, like the “Next” button. Note: For existing titles, you may have to delete the book from your device and re-download it from the “My Online Bookshelf” link to get these buttons.]

To get out of the Reading mode and back to the My Books section, select the “Home” link between the “Next” and “Previous” buttons.

My Books

This part of the “Home” section lists the books you’ve downloaded to your device for reading. Because Ibis Reader stores everything in the cloud, you don’t need to worry about deleting titles here because we’ll keep a copy at the main website if you ever need to get it back onto your phone or tablet again. Like the Get Books list, these are just a list of the title and author(s) for each book, and selecting one will open it to your last known reading position.

Cost-effective Development of Enhanced Content with EPUB3 (Digital Book World 2011)

Liza Daly — Mon, 31 Jan 2011 13:59:44 +0000

I presented at Digital Book World 2011 about using EPUB3 to produce multimedia and interactive ebooks that will be compatible with multiple devices and software ereaders.

Anyone who’s ever been to a digital publishing conference knows that there is always a bewildering array of products that create and display enhanced content: ebooks that contain video, audio, animation, or other forms of interactivity. Generally each new platform requires an entirely different way of preparing the content and all the multimedia assets. While the production of static ebooks has become relatively standardized (using EPUB2), publishers are still struggling with how to produce more digital-native products without tremendous expense and constant re-tooling.

I’m optimistic that EPUB3 will provide significant headache-reduction in this area. While creating high-quality static, pure-text ebooks still has challenges, things are a lot better today than even a year ago. If we’re to see similar consolidation in the enhanced ebook space in the same timeframe, publishers are going to have to be firm that they will only (or at least primarily) produce standards-based multimedia books.

Slides from DBW

Cost-effective enhanced ebooks with EPUB3

View more presentations from lizadaly.

EPUB3 interactivity demo in iBooks

The talk includes a video demo. I had created an EPUB document several months ago for a research project, using Abroad by Thomas Crane as source material (edition from the Internet Archive). I wanted to emulate the accelerometer-driven animation of the Alice for iPad book using only the tools available in EPUB3.

Since the source was a heavily-illustrated children’s book, EPUB’s reflow capability was not really an asset. This quality made the book perfect for use in Apple’s fixed-layout mode. The video below shows the book running in iBooks on an iPad:

Accelerometer

In the first example, I use the accelerometer interface in JavaScript (some sample code), to cause the ship to “sail” back and forth as the user tilts the device.

Animation

In the second example, if the user touches any of the birds on the title page, they fly away. I modified the jQuery Path library to use CSS3 transforms to efficiently animate along a curve. JavaScript performance in iBooks appears to be somewhat worse than Mobile Safari; optimizing using CSS3 rather than DOM position updates is recommended.

Canvas, Processing.js, HTML5 audio

I’m a big fan of the Processing visual programming language. In the last example, the train appears to give off puffs of steam; each time the steam comes out of the train the puffs are in slightly different positions.

I embedded Processing.js code (which is itself then interpreted by JavaScript) and rendered the puffs to a transparent canvas. The puffs appear in a slightly random position, fade out as they reach the top of the page and then spawn anew.

Finally, there’s also an event bound to the invisible canvas: tapping on it will fire an HTML5 audio (a sample of a train).

(At this time we’re not able to share the demo EPUB file itself.)

Threepress Consulting blog » epub

Better support for Unicode characters in EPUB validator

Transforming NCX into EPUB 3 Navigation Documents

NCX in

EPUB Navigation Document out

Validating EPUB 3 experiments

Layout of the EPUB 3 schemas

What to validate

RELAX NG validation with Jing

Schematron validation with XSLT

Copy Editors in EPUB 3

Can an author create an EPUB using normal tools? Part 2: Scrivener

Part two: Scrivener

The sample document

Export options

Output

First, a header.

A subheader.

Room for improvement

Can an author create an EPUB file using normal tools? Part 1: Pages

Pages

The sample document

The EPUB output

Headers and subheaders

Chapter Two: The Chaptering

I’m a subchapter or section under that.

Go boldly

Listless

Chapter Three: Lists

Tables

Reasons why tables are nice

Who feels this way

I can’t think of a third thing.

Images, covers, and links

Chapter Five: Hyperlinks

Metadata

The dreaded sample document

Improvements

Subtitles in EPUB 3

What’s going on here?

Other options

Colons don’t make subtitles

Covers in EPUB 3

Using Ibis Reader and Apple’s VoiceOver

VoiceOver and iOS

Installation

Basic Interaction

Finding Books

Reading

My Books

Cost-effective Development of Enhanced Content with EPUB3 (Digital Book World 2011)

Slides from DBW

EPUB3 interactivity demo in iBooks

Accelerometer

Animation

Canvas, Processing.js, HTML5 audio