[8/12/2010: I've posted a performance benchmark test to try and gather some numbers from the crowd-at-large on how single large concat'd files perform compared to chunking that big file into a couple of smaller files to download in parallel. I encourage you to go right now and click each of the buttons a few times, and then spread the link on to others. Concat Performance Benchmark Test #1]

By far the most prevalent question circulating right now in the wake of href="http://labjs.com">LABjs‘ initial public launch (IPL, I guess!?) is the understandable: “Why do I need a loader like LABjs for multiple files when I can just concat everything into one file?”

Fair enough, this is an important question to address. You shouldn’t just take my implied assumption at face value; we need to dig into this so you understand why I feel so strongly that LABjs offers something better than what you are probably currently doing on your sites.

The short answer is, BOTH! You should concat files together when possible, and you should load your file(s) with a loader like LABjs. If you only do one or the other, you have missed out on the bigger picture of page-load optimization.

YSlow/PageSpeed only value reducing HTTP requests

Before I get into specifics about why I think LABjs brings something new and beneficial to the table, let’s tackle what I consider to be an interesting but perhaps misleading “truth”. I am firmly in a minority camp in my skepticism that reduction of HTTP requests is the only answer to the question of how to load a site more efficiently — actually, to be more accurate, how to load a site’s JavaScript more efficiently.

I bet that you are recalling the YSlow rules and the tried-and-true practice that concat’ing all your JS files into one file for production is the surest way to speed up your page loads. And you’re probably doubting my credibility because I would question such a fundamental “truth” as HTTP request reduction.

It is true that, especially for the largest sites on the internet (millions of page-views per month and more), the sheer volume of HTTP requests from a typical page profile (dozens of images, multiple style sheets, half a dozen JS files, etc) quickly overloads internet connections, server load-balancers, the user’s device, and even the browser itself. And anything you can do to reduce that overload will go a long way toward helping the site load with acceptable performance.

Sites like Yahoo and Google have taught us the basic principles of HTTP request reduction as an effective way to cut down on the overall page loading delays. For instance, consider Spriting to combine images, CSS concatenation, and, yes, JavaScript concatenation. Some sites, like Digg, have even experimented with the extreme — inlining all page resources, even binary image data, into a single multi-part request! And just recently, we saw a proposal for Resource Packaging as another way to combine multiple types of data into single responses for HTTP request reduction.

If you run a site that has well over 50 million page views per month (I sure don’t!), you probably would assert that this is the easiest and most effective page-load optimization. And I’d be hard-pressed to go up against the brightest and the best at the world’s largest web properties to suggest that it’s not true, at least at their page-view volume.

What’s true is true. Period.

Here’s what I will suggest: maybe, just maybe, what’s true for Google at a hundred-million page views may not identically and exactly be true for the vast majority of us running sites in the tens-of-thousands page view realm. The “big boys” are up in rarefied air, and from there they certainly have the request volume to test, re-test, and “prove” these various rules of optimization. But in the same way that water behaves in strange, almost contradictory ways at extremes of hot or cold temperature, maybe the art of optimizing page loads isn’t as simple as “1 http request is always better than 2.”

Even for small sites (like the one you’re currently reading!), it’s still vitally important to reduce page load time and optimize for faster page views. So, I’ll readily and eagerly admit that a key strategy in that effort is for all of us to work to reduce our HTTP request loads.

OK, so you agree concat’ing is best?

Not so fast. Like I said, this issue is more complex than I think most of us are ready to admit. Too easily, we run the YSlow plugin tests on our site, and when we reduce HTTP requests by concat’ing our JS files, because it tells us to, not only does our perceived page-load times decrease, but our YSlow grade goes up! Success! We’ve proved the hypothesis, there’s nothing more to say, right?

I don’t think this in and of itself means you have achieved the pinnacle of page-load performance, you’ve simply taken the first of many steps toward nirvana.

In my opinion, loading your JavaScript in an efficient and optimized way cannot be boiled down into a simple set of “rules” — you can take such “truths” as good places to start, but you still have to refine the strategies to fit your particular site and audience. Indeed, this will start to look more like an art than a science, if done correctly.

I’m not so sure…

Maybe by this point, you’re still not convinced there’s more to the story, and you’re probably starting to feel like LABjs doesn’t really have that much to offer you.

Before you go, let me just say this: even if you only ever have one JavaScript file on your site, because concat’ing is the way to go for you, and you think that by putting a single <script> tag at the bottom of your page, you’ve achieved the ultimate JavaScript loading solution — there is still some benefit to using LABjs to load that one file.

Why? A regular <script> tag in the main HTML, even at the foot of your document, will always block the page’s loading completion, which means that users will still be “stuck” waiting for that file to complete before they can do anything with your page. Now, for some pages, you may really not want the users to do anything until the JavaScript fully processes everything, but most people agree that designing sites for progressive-enhancement is a positive thing.

So, if you can design your sites so that there’s something meaningful for the users to see or even do while they wait for the JS to kick in, they will probably be happier with your site than if you just make them wait. And that’s where LABjs comes in. Loading JavaScript dynamically unblocks the rest of the page’s resources (and DOM-ready!), freeing the user from the iron grip of the hanging “almost loaded but not yet” feeling that most sites force upon their users.

Include the small 4k of LABjs (via an inline <script> tag) into your page, and use it to load your single JavaScript file, and I believe your users will see an improved, even slightly, user-experience on page-load, as long as you’re careful about it.

OK, so maybe there is more to this loading thing

Keep reading if you’re ready to take the Red Pill and dive deeper down the rabbit hole.

(… more …)

This “event” is separate from (and ostensibly noticeably earlier than) the more familiar “onload” event, whose defined behavior means that all of the page’s initial resources have fully loaded — before “onload”, there is an earlier state at which things (such as images and styles) are still loading, but the basic plan or skeleton of the page is now known and understood and the browser knows what to expect.

This “event” is, in some browsers, an actual event, in other browsers it’s a state which can be passively observed, and in some browsers, it has to be divined by examining the alignment of the stars and taking into account the wind speed and gravitational pull of various heavenly bodies, divided by the…

You get the point, there’s an “event” that the browsers try (and some do better than others at this attempt) to expose for when the DOM is ready, which I will now conveniently and uncreatively call “DOM-ready”.

So what if the DOM is ready or not?

Let me make sure that you understand the importance of DOM-ready. In some cases, if you have JavaScript logic that tries to modify the DOM of your page before this DOM-ready event has occured/passed, you can wreak havoc with that browser, and in some cases, cause a fatal error and browser crash.

Think of it this way: if i’m standing on one foot, precariously balanced, and trying to pull on a sock over my bare foot, and my wife comes along and tugs on the rug I’m standing on, am I likely to get that sock put on successfully? Am I even likely to stay standing? Probably not.

So, it’s wise to make sure that you don’t take any actions which could adversely affect the browser’s ability to get his sock on, at least if you don’t want him to fall over. And that’s why we have the concept of DOM-ready — to make sure that you don’t manipulate the DOM too early.

Moreover, the major JS frameworks (for that matter, most of even the minor JS tools out there) all provide ways for you queue up code to be executed at some later time, that time being when the framework is able to detect that the event has occured and that it’s now safe for your code to go about the business of changing things.

The frameworks all take care of that mess for me, right?

Ahh, don’t we wish that were completely true. If it were, this post would be somewhat pointless, and you’d instead be enjoying eating a pizza or watching re-runs of Grey’s Anatomy.

Alas, the world is not that simple. There’s a little known, but I believe important, problem with the way DOM-ready detection is accomplished by the frameworks in a cross-browser way. I’m not going to bore you with all the details of just how this detection occurs, but let’s just say it gets a little hackish in some cases.

For the lion-share of use cases, the DOM-ready detection that the current framework versions employ is acceptable, at least for all the browsers that most of us really care about working correctly. If you’re still worried about Netscape 3.0 though, you’re on your own… good luck!

But, a subtle problem creeps in when you step outside the normal set of uses cases — those being where you simply include a <script> tag at the top (or bottom, if you’re really clever) of your HTML document that loads your framework of choice (let’s say “jquery.js” for the purposes of this post).

Let’s say you have a mashup where you are pulling together multiple websites, and maybe jQuery is not guaranteed to already be present on the page. “No problem!” you say, because you know how to load a script file on-demand, so you’ll just fire off a request to do so, and wait for jQuery to be present before you proceed to make calls against it (and, of course, then use it to change the page!)

Or, let’s say you have a bookmarklet that people can use to popup some nifty little widget in their page of choice, and again, you want to make sure that jQuery is present for your code to utilize. Again, we’ll just load jQuery on-the-fly, and we should be good to go, right?

Or even better, let’s just say you’ve got a really complex application, which loads different scripts into the page at different times depending on the state of what the user does. Yet again, you may need to fetch jQuery at some point well after the initial page loading occured.

There’s yet another use-case, which is actually the one that I ran into that caused me to discover this issue several months ago. If your page uses a script loader, such as LABjs, the one I wrote in cooperation with Steve Souders (of Yahoo! and Google fame), you have to be aware of how certain scripts, notably the frameworks, do their DOM-ready detection and how this applies when the script is loaded dynamically.

Because script-loaders remove the “blocking” nature of script loading that normal <script> tags provide, it is actually quite possible to get into a situation where you load a script, like “jquery.js”, but the rest of the page is so well optimized that the DOM-ready event passes before jQuery arrives. You may very well not intend this to occur, but it can, and worse yet, it can happen as a race-condition, where it happens sometimes, and not others.

(… more …)

The talk covers a number of different strategies, each of which I will be covering in subsequent posts here. But I wanted to give a quick overview of what you can expect:

1. Build-time optimization: it’s important to utilize build-process optimizations to manage local scripts and dependency chains. Depending on your server architecture and frameworks, there are lots of different choices that you have here, but the key is have a system that can manage this intelligence for you, freeing you and the other developers on your team from having to maintain the techniques yourselves.
2. Load/run-time profiling: We’ve all heard the advice that the best way to get JavaScript resources onto the page is to only have one JavaScript file, meaning we should always concat all our individual files into one massive output file as it goes to the browser in production. But does this always make sense? I believe it only rarely does.

   There are a number of tools for profiling your site/application at load-time and at run-time, to decide what parts of your JavaScript are critical to load at the beginning, and what parts can be safely deferred until later and be lazy-loaded. Doing so not only speeds up the load time user-experience of your site/application, but it also often helps create better long(er)-term user-experience by more effectively utilizing the user’s browser cache. You can profile manually, or automatically, but you shouldn’t ever put a site or application into production without at least examining these strategies and trying a few different options.

3. Post-optimize your minified JavaScript: For a long time, it felt like the book was closed on minification/obfuscation. There’s only so much white-space you can remove from a JavaScript file, and only so many variable names you can shorten, before a script seems as small as it’s going to get. But, it would seem there are more chapters to be written, yet.

   Recent releases of projects like Google Closure prove that there’s a lot more ground to cover on what we can do to optimize and shorten our scripts. I also have a project called Mincir which is aimed at specifically running post-minification optimization steps on files to squeeze out more bytes. Smaller files means quicker loading and better user-experience. And that means happier users and happier developers.

4. Dynamic, on-demand, parallel loading of JavaScript: While the landscape of how the browsers handle the <script> tag is quite diverse and often ugly, there are ways to normalize this behavior cross-browser, and in the process allow all your JavaScript resources to load at the same time, in parallel with each other and the rest of the page assets. This is actually not that hard. But ensuring proper execution order, not only of loaded scripts, but also of coupled inline script executions, is where the magic happens.

   Luckily, a project of mine, called LABjs, does just such a thing. It’s a general purpose, flexible, and capable JavaScript loader, that allows you to load just about any script or group of scripts onto any page, at any time.

5. Load and cache “not-code”, execute “code” later: Sometimes, the best way to get JavaScript onto the page is not to load JavaScript at all. This may seem like a crazy statement, but you actually have some options as to how your “code” gets delivered to the page. For instance, you can transfer the code inside a multi-line comment block, or even as an escaped string literal assigned to a JavaScript variable. You may ask, why would I do such a crazy thing? Well, it turns out that there’s a significant amount of time spent processing JavaScript once it’s loaded onto a page.

   So, if on one page you want to “preload” a script into the cache for use on a subsequent page, like you have done with images for years, you pay a penalty for that script simply being added to your page. Unless of course it’s not really a script at that point. If it’s a comment or a string literal, then it is loaded and saved into the browser cache, but there’s almost no processing time on the page at that point. Later, on another page, you can recall this “not-code” from cache, and transform it into executable code.

   This type of technique is quite valid for desktop browsers, but is particularly helpful for mobile browsers where the biggest bottleneck is not actually usually the connection speed but the processing power of the mobile device in handling large amounts of not-used-yet code while rendering a page.

None of these techniques are in and of themselves a complete solution to this issue, nor are these ideas the final word on the topic. But it’s time we evolve how we load our JavaScript onto pages and move beyond the 1990’s version of the plain ol’ <script> tag. I hope this talk and these posts will get that ball rolling.