There is sometimes significant rancor within the web development community as we transition from Flash to HTML5/JS for our audio-visual and interactive web content.

Let’s look at what went into a real-world game of Cut The Rope. While it was implemented as a promotional tool for the Windows 8 Store and IE9, ZengaLabs reports that it does work reasonably well on all HTML5-friendly desktop browsers.

First, we have the libraries. They chose jQuery 1.7, plus the FancyBox and the jQuery Easing plugins. To this, they added SoundManager 2 and Modernizr 2.0.6. These weighed in at just under 62KB.

Of course, no website is complete without tracking (Google Analytics in this case), as well as Facebook and Twitter integration, but those libraries are typically hosted remotely and are outside of our control.

This leaves us with a 69.7KB minified JavaScript file, ctr.js. This file contains significant swaths that look like

a.lineTo(32.7,183.4);
a.bezierCurveTo(31.8,176.3,31,168.9,30.3,161.3);

and

$("#resultStar3").removeClass("starEmpty").addClass("star");

What of the HTML? At a scant 4KB, it is largely boilerplate. Of particular interest is this snippet:

// html5 audio is unreliable in many browsers so its only enabled by default on
// IE9 or greater. You can force html5 audio by setting the querystring:
// http://www.cuttherope.ie/?html5audio=true
soundManager.useHTML5Audio = (isIE9OrGreater() || ZeptoLab.ctr.forceHTML5Audio);
soundManager.preferFlash = !soundManager.useHTML5Audio;
 
// initialize the game
ZeptoLab.ctr.run();

Note that we’re still resorting to browser-specific hard hacks. This is an understandable trade-off, though falling back to Flash doesn’t work if you also want to function in mobile environments. Our AV content is still forcing us to produce multiple formats.

It is also interesting to see the game effectively self-enclosed inside of a single object following a Java-like naming convention. This not only exhibits best practices, but also encloses it within a closure, making it harder for the code to be attacked.

The developers also reported another concern. Since the code can easily run before all of the required image and audio assets are loaded and the full game weighs in at 6MB, they needed a loader (much like what you have seen in traditional Flash environments). Happily, they released this original code as PxLoader under an MIT license.

Additional overhead is spent tracking the frame-rate to caution the user when performance suffers.

So what lessons can be drawn from this?

For the most part, once ActionScript developers become familiar with JavaScript development, the code they eventually produce will be fairly straightforward. Libraries abstract out most browser incompatibilities, although concurrent Flash and HTML5 development is still a fact of life unless you can be sure your users are on Internet Explorer 9+, Firefox 4+, Safari 5+, Opera 10+ or Chrome 10+.

It is unfortunate that patents on media formats are actually encumbering innovation among the people who would use them. Effectively, all media must be encoded in a minimum of 4 versions: traditional Flash, Chrome, Safari and MPEG. This is a drain on developer time and resources.

Finally, modern browsers do offer sufficient performance to implement complex interactive content natively within JavaScript. Plugins are no longer required to get work done.

We all know that code optimization is important, especially when processing large quantities of data. It is also common knowledge that adding layers of code to loops reduces performance. Except this maxim doesn’t always hold when it comes to JavaScript in browsers and even native methods can vary wildly in performance. In fact, in almost all cases, Underscore, jQuery and YUI outperformed any of the native methods. Interestingly, the newest implementation for native looping, forEach, consistently trumped a traditional for (var i = 0; i < 2500; i++) loop.

I set up a testing framework that looped through a normal array instantiated like so:

var data = [];
var v = {}; //This value doesn't seem to affect performance at the data size I chose
 
for (var i = 0; i < loops; i++) { //loops is defined elsewhere
   data.push(v);
}

I looped through this array using the following methods:

• for loop
• for … in … loop
• forEach loop (not supported in IE9 Quirks Mode)
• Prototype 1.7: Enumerable.each
• jQuery 1.7.1: $.each
• Underscore 1.2.3: _.each
• YUI 3.1.4: YUI.each

Each loop was passed the value of the array element and returned immediately, like so:

//From the native for and for ... in ... tests
(function(el){return;})(data[i]);

I eventually settled upon an array of 2500 elements and running the test 200 times to collect a reasonable number of samples. These numbers were not chosen arbitrarily. Prototype would time out on Firefox at 5000 array elements and Safari seemed to have wildly varying results on each run of the suite with fewer than 200 samples. With 200 samples, the variance in mean time between runs became a few hundredths of a millisecond, not enough to be of consequence. Disturbingly, while Prototype froze the browser with 5000 array elements and 100 samples, it had no issue with 2500 elements and 200 samples. Even so, with some tests averaging less than a hundredth of a millisecond, we will soon require much larger data sets to get meaningful results at all, which may preclude the inclusion of Prototype (and even native methods!) in future tests.

All tests were run in a freshly launched browser on an updated operating system (either MacOS 10.7.2 or Windows 7 Home Premium) running on a MacBook Pro with a 2.53GHz Intel Core 2 Duo processor and 4GB of 1067MHz DDR3 RAM. Firefox had all add-ons disabled and Internet Explorer had the Developer Tools open (to switch between Standards and Quirks mode).

So let’s look at some charts!

This was certainly an interesting start to the process. YUI, Underscore and jQuery all clocked in at 0.005 ms. Prototype would prove to be faster than native methods, with the exception of forEach, on all browsers. Underscore was only be outperformed on IE 9 (32-bit, Standards Mode).

Chrome was exceptional as well, with the traditional for loop actually running slower than for ... in .... This behavior was also seen on Chrome for Windows and Safari for Windows.

Since all of the major browsers are clearly pre-optimized for these libraries (as they themselves wrap the native looping language structures), let’s look at just the libraries:

Here we can begin to see that Prototype really is behind its peers in loop performance. While we are talking fractions of a millisecond, our apps are growing larger by the day and this could become a serious issue.

Hey! It looks like Firefox really is slow. Curiously, IE 9 is significantly faster, which flies in the face of most generalized JavaScript performance metrics.

Still, something looks odd about IE:

Your mileage varies wildly depending on whether you choose standards or quirks mode, and not always in obvious ways. In general, 64-bit is faster, but only marginally, and most people are not using it. (You have to explicitly launch it, as it is not the default, even on 64-bit Windows). Meanwhile, quirks mode apparently embeds some performance hacks. As a note to benchmark developers, this should be evidence enough that not all IE’s are the same and we need to start breaking down by rendering mode and compilation.

So far, we’ve seen the average times and most browsers are quite acceptable. What are the worst case scenarios?

Those averages hid some extreme variations within Internet Explorer. Fortunately, most of these are found using native language structures, which we know better than to use.

Please forgive the Simpsons reference in the title, but let’s face it, browsers eat up an enormous amount of screen real estate with menu bars, super bars, status bars, toolbars, scrollbars. And that’s all before you get to the white space and toolbars and menus your own app requires.

Sure, you can press F11 today in pretty much every browser and achieve that end, but how many of us actually do that? If we’re the power users and we don’t do that, what hope do we have for our users?

The Full Screen API will make it easy to do automatically. Better still, pre-release versions of this API are already supported in Firefox 9 and Chrome 15 (though disabled by default in Firefox 9).

To get you started, the Mozilla Developer Network has released its own guide to the interface.

In brief, there are a few key requirements:

1. The calling object for requestFullScreen() must be a DOM node
2. The DOM node may not be the child or parent of a plugin
3. If the DOM node is an IFRAME (or is the parent or child of one), the IFRAME must have an allowfullscreen="true" attribute

So what does real-world code look like? Let’s take a look at Mozilla’s own example:

function toggleFullScreen() {
  if ((document.fullScreenElement && document.fullScreenElement !== null) ||    // alternative standard method
      (!document.mozFullScreen && !document.webkitIsFullScreen) {               // current working methods
    if (videoElement.requestFullScreen) {
      videoElement.requestFullScreen();
    } else if (elem.mozRequestFullScreen) {
      videoElement.mozRequestFullScreen();
    } else if (elem.webkitRequestFullScreen) {
      videoElement.webkitRequestFullScreen(Element.ALLOW_KEYBOARD_INPUT);
    }
  } else {
    if (document.cancelFullScreen) {
      document.cancelFullScreen();
    } else if (document.mozCancelFullScreen) {
      document.mozCancelFullScreen();
    } else if (document.webkitCancelFullScreen) {
      document.webkitCancelFullScreen();
    }
  }
}

As you can see, the naming conventions aren’t quite there yet. Further, neither Firefox nor Chrome quite apply the CSS expectations of the developing standard.

When the calls fire successfully, either fullscreenchange, mozfullscreenchange or webkitfullscreenchange events will fire. Likewise, when it fails, either fullscreenerror, mozfullscreenerror or webkitfullscreenerror events fire.

Note also that in Firefox 9, you cannot use a form in fullscreen mode; pressing any key inside of an input seems to take you out of Full Screen mode. The browser provides no instructions for exiting fullscreen mode, so you will need to provide them yourself.

Confusingly, the ALLOW_KEYBOARD_INPUT parameter in the webkitRequestFullScreen() call seems to block the keystroke handler in the MDN sample, though forms once again work. It also provides instructions for exiting Full Screen mode, though it neglects to mention that Esc will do it as well, instead referring to Command + Shift + F on my Mac.

Still, Firefox 10 is expected to support the API out of the box, so we can start to see the development of true browser application environments. The possibilities for kiosk environments are also exciting.

HTML5 begat the AUDIO tag. Now that we have it, what do we do with it? And how do we accommodate older browsers?

Unfortunately, there are two incompatible standards and the W3C Audio Working Group "has not decided upon a specification".

The Web Audio API works on recent builds of Chrome and Safari. The API includes complex filters and allows developers to roll drum machines and visualizations. It’s active by default on Chrome 16, but not on Safari 5. Worse, Safari 5 reports that it is supported, but won’t run the samples to which I have linked. Adding insult to injury, it’s also slow and can randomly introduce skips and clicks.

Mozilla introduced the Audio Data API, but it only works in Firefox 4 and above. It’s also a very low level API and requires intermediate libraries to be truly useful for non-developer apps, of which there are a number. Like Google’s Web Audio, it’s slow and can randomly introduce skips and clicks.

Since these two technologies encompass only about half of web traffic, we clearly need fallback mechanisms.

There is SoundManager2 which uses both HTMl5 and Flash 8 or 9+ to bridge platforms. It relies on MP3/AAC/OGG/WAV sample files and supports panning, sampling and – most valuable – event support. Events like whileplaying and onposition make it easy to sync your audio and video. Beyond equalization, volume and start/stop positions, however, you can’t do much to filter the sound output.

You might also try Audiolet, which lets you generate your own waveforms in browser, giving you a full range of freedom, but it’s going to sound like 8-bit audio unless you use samples. (Hint: try combining it with MUSIC.js to make the most of that 8-bit arcade sound)

Then there is audiolib.js, which goes even lower than Audiolet and can operate server-side. It allows you to both manipulate and generate audio samples in PCM format, which it can then save for future use.

Audiolet starts quickly and is half the size of SoundManager2, but can click and skip, especially when the browser is scrolling or otherwise pre-occupied. SoundManager2 can bog down when playing too many sounds at once. Try activating +follow and then moving your mouse quickly over its Smashing Christmas Lights demo to see this in action.

Audiolib is extremely low level, but for generating short realistic waveforms, it is effective and fast, both client-side and server. Sadly, in Firefox 9 on my Mac, this example only plays the first bell while Safari 5 and Opera 11.6 played nothing at all. Chrome 16 played it correctly. However, when I had the page open in 3 browsers simultaneously, even though only one was generating noise, the load was causing skipping and clicking in Chrome. Your audio could easily suffer if the end user’s computer is otherwise busy.

In short, there is no good solution yet that is widely supported and flexible and performant, even within modern browsers.

Remember the glory days of 8-bit audio? Games today feature fully scored soundtracks records in dozens of high quality MP3′s, but as late as the 90′s, music was 8-bit frequency manipulation.

MUSIC.js is Greg Jopa’s library for taking notation and calculating its corresponding frequency. This can then be fed to audio generation libraries like audiolet.js to generate the sound.

Implementation is very simple. Take his sample of A from the fourth octave. Two lines is all it takes:

var n = Note.fromLatin('A4');
var freq = n.frequency(); // returns 440

Intervals are similarly straightforward:

var c = Note.fromLatin('C3');
 
// by semitone
var wholeStep = Interval.fromSemitones(2);
var d = c.add(wholeStep);
console.log(d.latin()); // "D"
 
// by interval name
var g = c.add('fifth');
console.log(g.latin()); // "G"

It even calculates chords and scales. Note that all of these calculations are uncached, so you will want to optimize your loops.

How do you share print music? Chances are you print out your compositions from your favorite editor and then make a lot of copies down at the office supply store.

With VexFlow, we can now render compositions in-browser, without library dependencies and support for both CANVAS and SVG (okay, Raphael is required for SVG).

It can render both classical notation and guitar tablature, all with code like this:

var renderer = new Vex.Flow.Renderer(canvas,
  Vex.Flow.Renderer.Backends.CANVAS);
 
var ctx = renderer.getContext();
var stave = new Vex.Flow.Stave(10, 0, 500);
 
// Add a treble clef
stave.addClef("treble");
stave.setContext(ctx).draw();

This is all a little low-level for someone focused on displaying music, however, which is why 0xfe gave us VexTab.

Now you can render music with notation like:

tabstave
notes (5/2.5/3.7/4) 5h6/3 7/4 |
notes t12p7p5h7/4 7/5 5s3/5

tabstave
notes (8/2.7b9b7/3) (5b6/2.5b6/3)v 7s0/4 |
notes (5/2.6/3.7/4)v

tabstave
notes (5/4.5/5)s(7/4.7/5)s(5/4.5/5) (5/4.5/5)h(7/5) |
notes t(12/5.12/4)s(5/5.5/4) 3b4/5 5V/6

The kicker is that with his other library, TabDiv, such notation would automatically be rendered if contained within a DIV with a class of vex-tabdiv

Microsoft has proudly announced that IE6 was used by only nine-tenths of the web surfers in the US in December 2011. The US joins Austria, Czech Republic, Denmark, Finland, Mexico, Norway, Philippines, Poland, Portugal, Sweden and Ukraine in their champions circle, nations with an IE6 penetration of less than one percent.

Belgium, Colombia, Indonesia, Netherlands, and Turkey are all poised to join them, at 1.0 to 1.2%.

Asia is still, regrettably, home to a great many IE6 users. Fully a quarter of Chinese web traffic is on IE6. South Korea is at 7.2%, Japan is at 5.9%, Vietnam weighs in at 5.5%, India is at 5.4% and Taiwan is only slightly better at 4.9%. If your website serves significant numbers in these markets, consider encouraging users to upgrade.

Happy New Year! I thought the first working day of 2012 for most of us marked a good occasion for reviewing what you, the web developer, are using to access this site (and presumably develop). Some of the numbers were surprising. From the return of IE, to the appearance of mobile devices in the top 10 display resolutions, to a spike in traffic from the Arab peninsula, you did the unexpected.

For the purposes of this comparison, I looked at the months of December 2010 and December 2011 using Google Analytics. All numbers are based on total visits to simplify the comparison.

Operating Systems: Windows… and Sony

You’re still mostly on Windows, but that margin is narrowing significantly; Microsoft’s operating system went from 72% of our readers to 62%. Of that, 53% was running Windows 7, while slightly more than a third of you have held on to Windows XP, down from 45% in 2010.

MacOS and Linux also saw declines, from a combined 27% to just 15% of our visitors.

The big winners were PlayStation 3(!) which went from 0 visits to 10.2% of traffic and Android, which went from less than one-tenth of a percent to more than 3.

Browsers: Microsoft Turns Around

Developers seem to expect their browsers to behave well with modern HTML, CSS and JavaScript features.

Last year, a mere 11% of you were visiting Ajax Bestiary on any version of Internet Explorer, but the widespread adoption of versions 8 and 9 turned that around. You are once again the predominate browser, with just under a quarter of the traffic. Sadly, Internet Explorer 6 also saw a spike, from 29 visits last year to 125 this year.

Firefox plummeted precipitously, from more than half of all traffic to 23% of it, with 75% of you on versions 6 and above, though curiously, there was a single visitor using version 3.0 Beta 1.

Chrome users can’t gloat though. While more than 75% of you are on versions 15 and 16, your share fell 6 percentage points to 24%.

Mobile devices are still a small fraction of our user base, at just 7% of visits. That said, they did grow an astonishing 1277% over the same period last year. Of those, Apple products represented only 18% of traffic. You are, apparently, an Android-based community, though RIM and Nokia saw increases as well. Opera Mini saw a sharp spike in traffic, which we will come back to momentarily.

Displays: Screen Sizes Grew – And Shrunk

The top 10 screen resolutions for December 2011 were 1280×1024, 1024×768, 1366×768, 1280×800, 1920×1080, 1440×900, 1680×1050, 1920×1200, 1600×900 and 320×480.

Half became more common (increasing by 13.66 percentage points) and half became less (declining by 24.1 points). All told, however, these resolutions were in use by more than three-quarters of all visitors.

Curiously, the largest increase was in 1024×768, which nearly doubled. Meanwhile, 320×480, a resolution used by some mobile phones, grew by a full 2 percentage points.

Where You Are: Probably Not The USA

Consistent with last year, two-thirds of our traffic use English as their default language.

Thirty percent of our visitors are from the US, up from 22% in 2010. California, Illinois, New York and Texas make up a third of that traffic. Almost every state visited us in December 2011. Montana, why don’t you love us?

The largest increases were in three regions: Unknown, the Middle East and the Caribbean.

Traffic from the mythical country of (not set) rose 7325%, though 90% of that was actually Opera Mini, which renders pages on their internal servers and presents them to the mobile device, obfuscating device and geolocation data in the process.

Almost all of the Middle East saw an increase in traffic, but a few countries stood out. Qatar rose 3700%, Saudi Arabia rose 3175%, UAE rose 1460%, Iraq grew 400%, while Oman and Yemen each rose 100%.

In the Caribbean, Puerto Rico increased 1800% and Trinidad and Tobago rose 233%.

Finally, Slovakia went up by 233%, Latvia went up by 200% and Algeria went up by 150%.

Sanity Check

Since some of this seems so anomalous, I pulled up the report for all of 2011.

Internet Explorer slipped to just 16% for the year. The transition to IE dominance actually happened in September. Meanwhile, Playstation users saw a steady climb throughout the year.

Windows rose slightly 65% and MacOS and Linux returned to a combined 22%. All three platforms still slipped from 2010.

For displays, 320×480, #10 in December, slipped to #14, with 1024×768 falling to fourth place.

In the US, Florida actually edged out Illinois and it appears that Montana was simply on vacation in December. We knew you loved us!

Outside the US, we saw traffic from 34 African countries, 42 countries in the Americas, 48 from Asia, 44 from Europe and 7 from Oceania.

Unless you develop in-house sites for companies that have standardized on a very modern browser, you’re stuck developing sites for IE8 and even IE7. (Heaven help you if you’re still developing for IE6).

How then do you access some of the powerful features of ECMAScript or the contextual tags of HTML5 without having to write layer upon layer of fallbacks?

Your fellow JavaScript developers have produced shims to emulate most of this functionality, even in environments where Chrome Frame isn’t viable.

The html5shiv project is still under active development. It not only adds support for HTML5 contextual tags, it sets their default styling and makes them print in IE8 and below. Naturally, their canvas element is not going to have the behaviors you see in other browsers, but at least you can get away with a single set of markup and fewer CSS rules.

The es5-shim project is also under active development. It alters prototypes of native objects, so I wouldn’t combine it with prototype.js. This library also has the benefit of being well documented – including what does and does not differ from the specification. Heed the cautions from Kris Kowal, its author, though:

“As closely as possible to ES5″ is not very close. Many of these shims are intended only to allow code to be written to ES5 without causing run-time errors in older engines. In many cases, this means that these shims cause many ES5 methods to silently fail. Decide carefully whether this is what you want.

There are lots of grid libraries out there. Naturally, this means we need to reinvent the wheel. With HTML5, we have a fantastic new tool called the CANVAS tag which frees us from the limitations of CSS.

The canvas tag is beautifully documented and supported on most modern browsers. Further, aside from IE 6-8, the vast majority of the browsers support the element as it is defined, which means we don’t need wrapping libraries to deal with browser quirks.

Using all of these libraries as baselines, there are a few essential pieces of functionality:
We must be dimension agnostic. Whether our space is 480px or 2560px wide, it should work well
We must support any number of columns
We must support n-wide columns
We must support arbitrary gutter sizes
We must get results consistent with those other libraries

To that end, I present the grid() function. It returns an object with an array of offsets, as well as plenty of other information that facilitates writing still more content to the canvas (or advanced features like simulating sub-columns). It has no library dependencies, although it currently does not gracefully handle older browsers.

Basic usage is simple:

result = AB.grid(canvas, 16, 20);

This one line takes the canvas element specified in the first parameter and divides it into 16 columns separated by 20 pixel gutters. The function returns an object that looks like this:

{
   cols: 16,
   element: canvas#tutorial,
   gutter: 20,
   offsets: [0, 60, 120, ..., 900],
   width: 40
}

You can also pass arrays:

result = AB.grid(canvas, [1, 3, 1], 10);

This divides the element into a 20% panel on the left and right and 60% in the middle. Now our response looks like this:

{
   cols: 3,
   element: canvas#tutorial,
   gutter: 20,
   offsets: [0, 194, 756],
   width: 186
}

Let’s go changing the colors:

result = AB.grid(canvas, [1, 4], 10, { foreground: '#CCC', background: '#EEE' });

Need to pad the outer margins as well?

result = AB.grid(canvas, [1, 4], 10, { includeOuter: true });

Maybe we just want to simulate the arithmetic, without drawing anything?

result = AB.grid(canvas.width, [1, 4], 10);

This would also work:

result = AB.grid(canvas, [1, 4], 10, { overrideWidth: canvas.width });

Now you’re probably saying that this is all well and good, but why bother when we are already using a CSS layout library for the site in general? Aside from making it easier to construct graphs, we can also use this to generate perfectly sized images within the client.

The following three lines will create an image tag from a canvas and do it faster than the human eye can detect:

var img = document.createElement('img');
img.setAttribute('src', canvas.toDataURL());
document.getElementsByTagName('body')[0].appendChild(img);

You could almost as easily assign it as a background image (here I am finally employing jQuery):

$('body').css('background', 'url(' + canvas.toDataURL() + ')');

Note that you must wrap the image data in the strings url( and ) for this to work.

Check out the code on GitHub.