Neither of those are suitable, because they are proprietary to Adobe and Apple respectively, and so they are not supported on all platforms. In order to support either of those, a device maker has to license software from Adobe or Apple. Most video playing devices don’t have the CPU power to run FlashPlayer or QuickTime Player, that is PC software.

There is a universal consumer video format called ISO MPEG-4 H.264/AAC. It succeeded the MPEG-2 found on the DVD and the MPEG-1 found on Video CD about 10 years ago. It’s vendor neutral, any device maker can use it in their device or software player on equal terms with all other device makers.

ISO MPEG-4 plays not just in both FlashPlayer and QuickTime Player on Mac/PC, which would be remarkable in itself, but also on iPod and other media players, iPhone and other smartphones, iPad and other tablets, set top boxes, game consoles, Blu-Ray Disc, and so on and so on. Every video on YouTube is stored as MPEG-4, and every video ever sold from iTunes is MPEG-4. Also, if you shoot video with a Flip or iPod or iPhone camcorder, that is MPEG-4. In the same way a DVD Player always has an MPEG-2 decoder, all of the devices that succeed the DVD Player have MPEG-4 decoders.

A key thing to understand is that devices other than Mac and PC do not have large general purpose CPU’s that can run any arbitrary codec as software. Instead, they have dedicated video decoder chips that can decode ISO MPEG-4 with high efficiency and very low battery use. That’s why the iPad can play 10 hours of video, it’s playing through a hardware device that is highly optimized, not through a software codec on a general purpose CPU, as it would on many PC’s. (Macs have H.264 decoders in their GPU’s.)

Another thing to consider is bandwidth. If you have a book with Flash video in it, and you replace that video with ISO MPEG-4 H.264, the weight of the video will be 20% of what it was. (And it still plays in FlashPlayer.) When you consider readers will be downloading your book over cellular connections, and there are limits in some cases where a fat book has to be downloaded over Wi-Fi, this is important.

As for quality, ISO MPEG-4 scales from YouTube to Blu-Ray HD. Flash video does not do HD.

So if you want universal playback, start with the universal video format. Include an image as fallback content for devices that can’t play video, and text for devices that can’t display images. But I would be very suspicious of doing any other audio video formats as that time is likely wasted, and likely doubles or triples the weight of your book for all readers for no good reason.

> since the iPhone doesn’t support flash video

There are no mobiles that support Flash video (zero, none, nada), it is not just the iPhone that does not support it. Flash video only plays in Adobe FlashPlayer, which as I write this (2010-03-10) runs only on Mac and PC platforms and requires a desktop-class CPU equivalent to a P4 2GHz, which is many times the computational power of a mobile. A version of FlashPlayer for mobiles has been promised by Adobe for over 3 years now but has still not shipped. FlashPlayer is Mac/PC software. Authoring books for FlashPlayer is like shipping your book as a Word document or InDesign document. Movies don’t ship in Final Cut, music doesn’t ship in Logic format.

Video in books has to be universal because many books really benefit from it as essential content, not just an add-on frosting layer. For example, tech books: a frozen shot of a cursor hovering over Photoshop or a wrench hovering in mid-air is just not nearly as good as a short video showing the entire technique. Obviously books about movies should have movie clips illustrating the text. So books benefit from standardized consumer video as much as music and video players have benefited.

element.scrollIntoView is an interesting idea too — thanks! (When I had a columns implementation it was basically the same as yours.)

My solution is probably peculiar to Monocle’s column-width/scroll setup, but I found that using element.scrollIntoView then checking scrollLeft gave me at least a useful x coordinate.

It’s a hack I’d love to dig out, though. Great news that it works in the latest nightly.