Now Hear This!

With all the talk about Flash versus HTML5 you might be forgiven for thinking that HTML5 is just about video. It’s not, of course. HTML5 is a major extension of HTML in many areas including media.

Today, I’d like to explore HTML5 audio, which is similar to HTML5 video in many respects, but fundamentally different in others. For starters, a video element is essentially just a TV set embedded in a webpage whereas audio can serve multiple purposes – four examples:

• Sound effects. A button can make a sound when it’s clicked. How about a swoosh sound when a division is toggled open or closed?
• Alerts and notifications. A little bell can sound when an AJAX request completes and updates a page section. Or, an audio “Uh Oh!” can play when you enter a malformed email address into a input field.
• Background sounds. Some web pages can benefit from background music. But say you have a photo gallery of birds. Wouldn’t it be nice to be able to click one and hear it tweet?
• Spoken content. Yes, we have podcasts, but we also need something simpler such as pronunciations of foreign words and phrases, or a click-to-have-this-post-read-to-you feature so you can go off and browse other webpages while I continue to read this one to you.

S i l e n c e.

Yo Dawg, I can’t hear your website!

This isn’t as dumb a statement as it first, um, sounds. Web developers have shied away from including sound in webpages because of several difficulties:

• Confusion – There are codecs and there are file formats. A codec is a method of encoding audio data. A file format is a container for the codec and optional metadata, such as a song title. Some file formats can accommodate multiple codecs; other formats are codec specific. For simplicity, we can narrow this discussion to the most popular file formats/codecs for the Web. These are:
  • wav – An uncompressed format. A wav file with 30 seconds of silence is the same size (megabytes) as a file with 30 seconds of symphonic music.
  • mp3 – A popular format for music that uses a lossy compression format resulting in file sizes typically 1/10 the size of a wav file with the same audio content.
  • ogg – An open-source alternative to mp3 that also uses a lossy compression format.
• Cross-browser incompatibility – Unlike images, where all major browsers support the three major file types—gif, jpg and png—all browsers do not support all audio formats. For instance: mp3 is a proprietary format and its decoding software cannot be included in GPL-licensed, open-source applications such as the Mozilla Foundation’s Firefox browser. Here’s a table showing which formats are supported by the current versions of the major web browsers:
  

  	IE	Firefox	Safari	Chrome	Opera
  wav	NO	YES	YES	NO	NO
  mp3	NO*	NO	YES	YES	YES
  ogg	NO	YES	NO	YES	NO

  (* IE8 does not recognize the audio element at all. The IE9 beta works only with mp3 files. )

• Plugin required – HTML4 Browsers do not support audio natively. A plugin is required and JavaScript must be enabled. While most browsers support audio with a distributed default plugin, web developers must still use the ugly and opaque object, embed and param elements to bring sounds to a webpage.
• Insufficient processing power – For the first time visitor to a page, it can take several seconds for an older or overloaded browser to load the player code, download, decode and play a audio file. This pretty much nixes using sound for mouseover effects, even simple click sounds for navigation buttons.

YouTube made Web video workable. No such service existed to popularize the use of audio in webpages. On the contrary, Napster.com and iTunes focused attention on downloadable music applications. Wouldn’t it be great if there was an easy way to incorporate sounds on a webpage for other purposes. Good News! There is. All major browsers now support basic HTML5 audio with the notable exception of Internet Explorer.

The HTML5 Audio Solution

HTML5 audio consists of the audio element and a JavaScript API for playing the sound it represents. The element is a container (it has both opening and closing tags) and what’s inside is fallback content for legacy browsers and search-bots. It can be written simply as:

<audio src="url" autoplay loop>
        Aw, snap. Your browser doesn't support HTML5 audio!
</audio>

The above would be perfect for playing background music on a webpage. it could be placed anywhere in the body of the document—heard but not seen. For creating a controllable audio player, the controls attribute can be added to the opening tag.

<audio src="url" controls>
        <a href="url">Download the audio file</a>
</audio>

Which, depending on your browser will create a player like one of these:*

IE	Download the audio file
Firefox
Safari
Chrome
Opera

(* Just pictures of the controls in different browsers. They don’t work here; neither does the link.)

Unfortunately, because of the differing support among browsers for the three file types, this approach won’t work in all browsers. That means having at least mp3 and ogg audio file versions and using browser detection to figure out which to load.

But wait. There’s an alternate form of the audio element that takes a different approach: try one format and, if it doesn’t work, try another. You still need at least two files but the syntax is much simpler.

<audio id="bgSound" loop>
        <source src="audio/bgSound.mp3" type="audio/mp3" />
        <source src="audio/bgSound.ogg" type="audio/ogg" />
        <-- lnsert fallback Flash Player code here --">
</audio>

The audio element must be in the body of the html document. I’ve given an id attribute to the above so it can be referred to by JavaScript statements. Place a script element right after the audio element to get its DOM object:

<script type="text/javascript">
  var bgSound = document.getElementById('bgSound');
</script>

Don’t put the JavaScript in the head of the document, the audio element must be defined before its DOM object can be referenced. The DOM object for the audio allows you to change its properties, start and stop the music. To play the audio as page background, add an onload attribute to the body tag:

<body onload="bgSound.play();">

As an alternative, use jQuery in the document head to call the play method at document ready time (when all the HTML is parsed but images may still be loading.)

<script type="text/javascript">
  $(document).ready( function () {$('#bgSound').play();} );
</script>

The jQuery $ function is similar to JavaScript’s getElementById( ) method. You’ll need the jQuery library for this to work. If it isn’t already loaded, placing the following script element in the document head will get it from Google.

<script type="text/javascript"
 src="http://ajax.googleapis.com/ajax/libs/jquery/1.3/jquery.min.js">
</script>

To play a sound effect when the user’s mouse is over a link, remove the loop attribute from the audio tag and add preload, then use the onmouseover attribute in the link. You can get the DOM object and play it in one move:

<audio id="thanks" preload>
        <source src="audio/thanks.mp3" type="audio/mpeg" />
        <source src="audio/thanks.ogg" type="audio/ogg" />
</audio> 

<a href="javascript:void(0);"
   onmouseover="document.getElementById('thanks').play();">A shout out</a>

Or, to assign a sound to a button like the one at the bottom of this post

<audio id="ella" preload>
        <source src="audio/swonderful.mp3" type="audio/mpeg" />
        <source src="audio/swonderful.ogg" type="audio/ogg" />
</audio>

<button onclick="document.getElementById('ella').play()">
        I like it!
</button>

The JavaScript API for HTML5 Media elements contains many more properties and features including a track property which can be used to segment a file into separate sounds of different durations. One audio element could be used to (pre)load dozens of different sound effects for a website. Not all advanced API features will be supported by all browsers in the near future, but eventually, using sound on a website will become as natural and as easy to do as using images.

A shout out to my friends at WWWAC who helped test HTML5 Audio on the IE9 beta release.

What do you think of HTML Audio?    I like it!

The BUTCH & harold website is a traditional HTML website. It’s nicely designed but unlike the blogs I’ve been working on recently, there are no templates or includes for global page elements. There are just a lot of HTML files. I didn’t build this site, but the Web designer/programmer who originally constructed it did a good job writing clean, modern code, making It easy to add content and make minor layout changes within the existing architecture. However, s/he did use some tricks with the navigation menu that gave me headaches trying to implement the drop-menu without rewriting the entire thing—and make it work in all browsers.

The remainder of this post is more technical. Skip it unless you’re really interested in how page navigation works. Or just visit the site and check out how I built the Stickr (frame) and Stickr (mini) pages. You’ll see (hopefully) some fun javascript in action.

Navigation menus have evolved since the early days of the Web. At first, websites were only able to feature a list of links; either vertically, as a list, or horizontally in a table. A link changed color when the mouse pointer hovered over it but all links on the page had the same color behavior. Designers wanting more choice than the common available fonts (e.g: Times, Helvetica and Courier) replaced the text with small images using designer fonts. However, unlike linked text, a linked image provides no visual feedback when you hover over it other than the mouse pointer changing from an arrow to a hand.

When Javascript came along in the mid-nineties, it provided a means to capture a limited number of page “events” and take action when they occured. These events have descriptive names, such as: onMouseOver, onMouseOut and onClick, and one of the actions availabe on the occurrence of such an event is changing the source of an image. Thus, instead on a single image used as a button, with Javascript, the button could have both “on” and “off” image that were swapped when you moved the mouse on and off of the link. This provided the interactivity designers wanted, but it came at a price. There was often a noticeable delay the first time a link switched between its off and on images as the browser waited for the alternate image to download from the server. Techniques were developed to “pre-load” the alternate images, which helped somewhat (as did faster Internet connections) but pre-loading images just moves the delay from the individual buttons to the initial page load.

Plus, there was the additional cost of complexity—If you had ten navigation links, that meant having 20 images to manage. Changing image-based navigation links involved expensive software tools, such as: Photoshop, Illustrator, DreamWeaver and ImageReady, and took much more time. This limited the ability of the client to explore and develop their online concepts from the bottom up as they learned the technology of Web publishing. It robbed them of the freedom to play with ideas.

But, Wait! It’s even worse than that. There are two other problems related to the 21st Century Web (aka: Web 2.0.) First, robots can’t read images. That is, the specialized automated tools that Google, Yahoo, and others use to visit, catalog and rank sites can’t interpret the content inside an image. From the robots perspective, a lot of information about the structure and flow of a Web site is invisible when you use graphic images for navigation links. Now let’s face it, Google has become indispensable to the way we use the Internet to market our products and services and you’re at a competitive disadvantage if your site isn’t robot-friendly.

The second problem with graphical navigational elements has to do with complexity again. You’ve heard it said that “Content is king.” What this means is that a static site, where the content hardly changes over time loses out to sites where fresh content is dynamically generated tailored to the customer’s needs. Publishing content on a regular schedule requires good content management systems. It’s also a lot easier if the overall structure of the site and its component HTML elements are as simple as possible.

Thus, there’s been a return to the use of text as navigation items and, indeed, a movement away from specifying layout constructs and style choices in the HTML, and toward using cascading style sheets (CSS) for these purposes. Google is a good example of this. Fortunately, the CSS recognized by today’s browsers is much more powerful and the designer has many more options to add interactivity than s/he did in the last century. For example, you are all familiar with simple, unordered lists, the kind with bullets. With CSS, you can eliminate the bullets and float the list items so that they appear in a horizontal line instead of using a table for the items. Also, nested lists are easier to manage than nested tables. And, instead of capturing a mouseOver event with Javascript, you can preset the hover state of any element and change text and background colors, even background images with the CSS without the pre-load problems.

This brings us back to BUTCH & harold where the original designer tried to have it both ways with the navigation menu—using text lables for the robots to read and pretty images with mouseOver effects for humans. It’s all done in the CSS. The navigation menu is an unordered list displayed horizontally with a unique on and off background images assigned to each list item. The images have a nice font (Bliss) for the labels and the “on” or hover states have the B&H logo above them. To keep the actual text out of sight for humans but still readable by robots, the designer set the text indentation in the CSS to -5000 pixels. The robots don’t care; they don’t bother reading a website’s CSS, but for humans, this moves the text off the left side of the monitor to somewhere in the next room.

Michelle and Ariane didn’t want the clutter or expense of new graphics for the drop-menu items; they’d be satisfied with text in the same color as the labels and a close match to the font. Making the drop-menu items appear in the right place meant setting the text-indent for the nested list to +5000 pixels to counteract the original setting that made the top level invisible. This worked well for Firefox and IE7 but not for Safari and, try as I might, I could not get the positioning right for all three.

I threw out the text-indentation hack and used the visibility property to hide the text labels from humans. This fixed the positioning problem with Safari, but made the entire “Shop” menu and the drop-menu invisible in IE6. IE6 is significantly different in that only recognizes a hover state for links whereas other browsers recognize a hover state for any HTML element. I did my research and found a solution using Javascript to capture a onMouseOver event on the top level list item (the Shop button) to dynamically change the CSS class of the nested drop-menu.

Wow, I make it sound simple but it was anything but. It took a lot of trial and error and I ended up rewriting almost all of the CSS to eliminate conflicts between the list styles used for the menus and the list styles used everywhere else.

In conclusion, I notice that about 1/5 of you are still using IE6 to browse the Web.* Please Stop! Make my life easier and your computing more secure and enjoyable. Upgrade to IE7 or download Firefox, Safari, Opera or one of the two new browsers: Flock or Chrome.

Google Chrome is a minimalist’s browser. It doesn’t offer a multitude of features, but it’s very fast and solid as a rock. It relies heavily on tabs when visiting different sites and each tab runs in its own process. This keeps the browser from crashing from a page error or some misbehaved plug-in, and prevents pages in the background from slowing down the foreground window. I like Chrome’s history function. When you open a new tab or window it displays thumbnails and links to your most recently vistited pages.

Chrome offers a single address bar for both URLs and keywords. Google is the default search engine, but it can be switched to others—Wikipedia, Aol, Yahoo, etc. The address bar has a fast auto-suggest function. For flying around the Web from site to site, Chrome is a jet fighter. However, if you mostly stay put on a collection of actively fed “home” pages, Flock may be the browser for you. It’s a mega-cruise ship.

Flock is Firefox on social media steroids. Flock takes a framed approach to visiting the Social Media Web, grabing feeds from places, such as: Facebook, Twitter and MySpace into sidebars and YouTube selections into a headband. Flock works best opened full-screen on a big, wide, LCD display. Beware! The shear number and variety of tool bars, control tabs, bookmarks, menus and search boxes can overwhelm.

Flock, by default, remembers the sites you had loaded when you last quit and reopens them. It also opens a generated page, “My World” that captures all your various feeds, messages and pings.  I like the way Flock integrates with Gmail, and find it useful as an “active desktop” – a one-stop site that I scan every so often to keep in the mix as I’m doing other work. Flock pops-up a standard blog editor when I want to post something and just about anything can be shared with a drag-n-drop action.

The online support documentation provided by the these two new browsers also provide a facinating contrast. Google wrote a Chrome comic book that teaches you how to use Chrome with a technical depth that’s actually readable. Flock has uploaded a series of youTube videos showing Flock’s Features in action. Check it out, you’ll get a better idea of what Flock is like and I won’t have to insert a screenshot here.

Google Chome is only available now for Windows. When it becomes available for the Mac, I’ll probably be using both Flock and Chrome more often than Firefox and Safari.

— Larry