HTML5 audio and video

HTML5 audio and video has been on my to-do list for over a year now and a recent discussion on the HotPotatoes support Yahoogroup has urged me to write about the various things that are involved in migrating HotPotatoes exercises to (X)HTML5. The list of things I'm going to discuss here goes in two sections:

Section 1. HTML5 audio and video
• Server issues and MIME types
• Browsers, and what audio/video works where
• Encoding audio and video properly
• Recommended Encoders
• The (X)HTML5 code for
  • audio
  • video

Section 2. HTML5 audio and video in HotPotatoes
• (X)HTML5 audio and video in HotPotatoes
• (X)HTML5 HotPotatoes in Moodle with the HotPot and QuizPort modules

MIME types

Check that your server can play the files properly because some servers have not yet added the new file formats, for example .webm.

MIME types with cPanel

If your host has cPanel, find the MIME types in the menu and add the appropriate file extensions, like in the screenshot below. Input the MIME Type, input the file extension, click the Add button.

MIME types with .htaccess

An alternative is to add the appropriate mime types in the .htaccess file (or create one) in the root of your web site, which will affect the whole web site. On a Linux host this usually lives in the directory public_html. Here is what to put in the .htaccess file.

# Server can play audio
# MP3 audio
AddType audio/mpeg3 .mp3
AddType audio/mp3 .mp3
AddType audio/x-mpeg3 .mp3

# Ogg Vorbis audio
AddType audio/ogg .ogg
AddType audio/ogg .oga

# Server can play video
# Ogg Theora video
AddType video/ogg .ogv

# MP4 video
AddType video/mp4 .mp4

# Webm video
AddType video/webm .webm

Browsers-codecs-formats issue, or what audio/video works where

Now that nearly 3/4 (74% according to StatCounter ) of the browser market can play HTML5, Flash support for mobile devices will eventually disappear. Danny Winkour, Vice President & General Manager of Interactive Development at Adobe said November 9, 2011 :

We will no longer continue to develop Flash Player in the browser to work with new mobile device configurations (chipset, browser, OS version, etc.) following the upcoming release of Flash Player 11.1 for Android and BlackBerry PlayBook. [...] And, we will design new features in Flash for a smooth transition to HTML5 as the standards evolve so developers can confidently invest knowing their skills will continue to be leveraged.

The biggest challenge with HTML5 audio and video is still the fragmented support for audio and video formats. On desktops, the split is almost exactly 50/50, with Chrome/Firefox/Opera supporting the WebM format and Chrome/IE/Safari supporting the MP4 format. Firefox recently capitulated in the Web-video war for open web , said Stephen Shankland March 14, 2012:

High-ranking Mozilla staff, believing they've lost a fight to keep patent-encumbered technology off the Web, have concluded it's time to change course and support H.264 video technology.
[...] The patent issue led Mozilla to strongly endorse Google's alternative VP8 codec that's part of its royalty-free WebM project. But WebM just isn't catching on, and Google hasn't fulfilled a promise to remove H.264 support from Chrome to try to promote WebM.

There's one good thing for coders, the media sources are now only two: MP4 and WebM. Note that Ogg Theora is excluded from the video formats because it doesn't offer as high quality video. However, there's still some room for Ogg Vorbis on the audio front because of Opera and Firefox and optionally for Chrome.

So here's a comparative table of what browsers support what audio and video.

Browser / Device	Video Formats	Audio Formats	Multiple Sources
Chrome	MP4, WebM	AAC, MP3, Vorbis
Firefox	WebM	Vorbis
Internet Explorer	MP4	AAC, MP3
Safari *	MP4	AAC, MP3
iOS	MP4	AAC, MP3
Android	MP4	AAC, MP3
Opera	WebM	Vorbis

^* - Safari 5.1 (on Windows) plays if QuickTime is installed. This is fixed in Safari 5.2.

The table above brings us to the next two points, encoding and encoders.

Encoding audio and video properly

It's now clear that you need two files to cover nearly all HTML5 compliant browsers and devices: MP4 for Chrome, IE, Safari, iOS and Android, and WebM for Firefox and Opera. It's also important to make the right combination of audio and video codecs if you want to make sure that your audio and video will play properly in the different browsers. The right codecs need to be specified in the code in the <source> tag.

Encoders

The tools I use are the ones I recommend. For audio, get Audacity , and install the FFmpeg Library to be able to import/export to and from AC3 and M4A among others.

Encoding video is a different job, though and I'd recommend Miro Video Converter , especially for WebM. The presets is something you'll appreciate. Another powerful converter is Format Factory (Windows only), but you have to watch for the audio/video codecs combinations of your videos. Mac users will surely have some free and commercial converters/encoders at hand.

If you think you are now ready to kick off, hold your horses - there are some other details to consider.

The HTML5 code

Tag attributes support

The HTML5 video and audio tags support several attributes, the most important ones being poster, preload, autoplay & controls. To date nearly all (90%) are supported across the various browsers and devices. Had it not been for IE9, which ignores the preload=none attribute, desktop browsers would have scored 100%. In fact, IE9 always loads a part of the file, which is not that bad as a matter of fact.

The mobile browsers ignore preload (video is never preloaded) and autoplay (video is never played upon page load). This is quite logical and will save users bandwidth and hardware resources.

For HotPotatoes exercises, though, preloading may actually turn into improved user experience. Since users will either way have to listen to or watch the file, these will be ready to play instantly, rather than start loading when requested through the user's click.

Although the design of the video controls in each browser is different, all browsers provide the same functionality: a play/pause toggle, a time slider and a volume slider. Additionally, Firefox and Safari also render a fullscreen toggle. iOS and Android omit the volume slider (in favour of hardware buttons).

Check this page with screenshots to see how the HTML5 players look in the different browsers on Windows.

More on the preload attribute

I already mentioned this, but again here's a little more about the preload attribute. There are the four possibilities.

<audio preload />
<audio preload="auto" />
<audio preload="none" />
<audio preload="metadata" />

- As seen, preload can be used on its own, which is equivalent to preload="auto". To iterate, this may be good for HotPotatoes exercises since the user is required to play the file and that way the files will be already cached; the user will not have to wait to play the file, which provides better user experience.

- If preload="none" is used, IE will ignore it and will either way load a part of the file, as noted earlier.

Mobiles generally do not preload, which is also sensible. However, if you are making HotPotatoes exercise for mobile devices, consider using preload="metadata".

- Using preload="metadata" means that the browser will get the info about the file length, size and other meta information about the file; it will be ready to access the audio file when, and if, requested, but it won't load the audio/video into the browser cache. So, if the user doesn't request the file by clicking the play button, it won't load, thus saving bandwidth. This is quite reasonable for non-HotPotatoes pages.

Multiple sources

As you can see from the code below, multiple sources are allowed so that HTML5 audio and video is offered across the different browsers. If a browser can't play the first file, it goes on to try the second source (or the third, or the fourth if they exist).

Mobile-specific issues

iOS only recognizes the first source. The solution: MP4 must always come first (this is fixed in iOS 4.0). Another one on iOS, <video> is not recognized if the poster attribute is included, so don't include it. For pre-Android 2.3, you must omit the type on the MP4 source. And since it's the only format that Android recognizes, make sure the file extension is .mp4.

For more on the source tag see the W3C Specs .

HTML5 audio

<audio controls="controls" preload="metadata">
   <source src="audio-file.m4a" type='audio/mp4; codecs="avc1.42E01E, mp4a.40.2"' />
   <source src="audio-file.oga" type='audio/ogg; codecs="vorbis"' />
    No HTML5 audio playback capabilities. No Flash fallback either!
</audio>

HTML5 video

<video controls="controls" preload="metadata" width="320" height="240">
   <source src="ipad-video-file.mp4" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"' />
   <source src="webm-video-file.webm" type='video/webm; codecs="vp8.0, vorbis"'/>
    No HTML5 video playback capabilities. No Flash fallback either!
</video>

Now that you have the code for HTML5 audio and video, I believe you are impatient to give it a try, so let's move on to Section 2. HTML5 audio and video in HotPotatoes.

Related tutorials

HTML5 audio and video with the Custom reading hack