getiblog » labjs

Simulated Chaining in JavaScript

getify — Wed, 10 Feb 2010 16:18:04 +0000

What I have to talk about here is not going to be earth shattering, but it is a technique that has proved useful in a few different sites and scenarios for me. In fact, I regularly get questions regarding advanced use cases for LABjs (a parallel JavaScript loader), and invariably it comes back to something along these lines.

If you already understand object/function call chaining, just skip down to the good stuff.

The chains that bind you

Just so we’re all on the same page, “chaining” is this wonderful functional property of languages like JavaScript (which treat a function as a first-class citizen). It essentially amounts to making a function call, and the return value from the function is itself either another callable function, or more often, an object that has functions which are directly callable.

jQuery is probably the most prevalent use of this concept. Some example jQuery code:

$("p.neat").addClass("ohmy").show("slow");

In this snippet, $(…) is actually a function call, just like foo(…) or doSomethingReallyEvilButDontTellAnyone(…) (just kidding; only Google would name a function something crazy like that).

The return value from the $(…) call is an object, that has a bunch of properties on it, many of whom are functions. Consider that conceptually an object is returned like this (not actual code):

function $(...) {
   // do something cool
   return {
      addClass:function(...){...},
      show:function(...){...},
      ...
   };
}

So, what happens is that when that object is returned from the function, JavaScript will allow you to immediately de-reference the return value (the object), and using “.” (dot notation), access its properties. So $(…).addClass(…) makes the first chained call (“addClass”) off the return value from $(…). Then, addClass(…) also returns the same (same kind, not identical) object, and so this second return value can identically be de-referenced and a chained function be called. I like to refer to each chained function call as a “chain link”.

This chaining can theoretically go on forever (although for code readability it can be argued that 3 or 4 chain links is probably the most you want to practically do in production code, exceptions aside).

Why chains?

Probably the single biggest argument for developing code API’s that are of this chaining style is to pass context along from one function call to another in an automatic and graceful feeling way. For instance, in jQuery’s case, the reason for chaining is that internally, there’s a collection (a fancy array) of objects, and as each chained function is called, internally it loops over the collection of objects and performs tasks (such as expanding/contracting the collection, modifying objects in the collection, etc).

jQuery could work functionally this way and achieve the same effect:

var collection = $("p.neat");
$.addClass(collection,"ohmy");
$.show(collection,"slow");

See how we had to pass in the “collection” object/array for every function call? It works, but that code is arguably a little less graceful. And certainly it’s a little more repetitive.

So, chaining was a way to accomplish the same task (context passing across function calls) but in a prettier, more graceful fashion. Generally, the idea of adjusting the way a set of code works to make it more expressive or pleasant is called adding “syntactic sugar”.

To be fair, syntactic sugar is not the only reason why chaining in functional languages is useful. In fact, with LABjs, which also has a similar type of chaining API (although internally operates quite differently from jQuery), the reasons were only partially “sugar”; chaining in LABjs’ case actually provides an effective solution to guard against certain types of race conditions.

Because of how JavaScript (single-threaded) works, a chain of function calls is more or less guaranteed to complete fully before any other code comes in and interrupts to execute. This is also probably true of a series of distinct function calls in a single code block, but even more definitely so for a chain of function calls or (as we’ll see in a moment), a loop of function calls.

So why not a chain?

Since chains are powerful and also tend to lead to more expressive code (unless abused) and also generally less code, you may wonder, why wouldn’t we always use chains for everything?

The obvious: some function calls are, by their nature, independent and thus do not need to (or should not!) share a context. Also, under certain circumstances, chaining can be less efficient internally, depending on how the context is stored/shared/used.

Let’s face it: if you really want to write your entire “program” on one line with a bunch of obscure/abusive syntactical tricks, just go write some Perl. I kid, I kid.

But beyond those things, one fundamental fact of chaining is that it’s not “dynamic”. Basically, what I mean by this is that you pretty much hard-code at the time of code writing (aka, “build time”) how a chain is constructed. You can’t decide at run time based on environmental conditions that some parts of the chain are conditionally not executed (like by wrapping them in an if-statement, etc).

The ability to do that kind of run-time logic is restricted syntactically to individual statements/function calls, so a chain will be considered an atomic single indivisible unit for those purposes. You could conditionally decide whether to run one of several different chains depending on conditions, but each chain to be chosen would be pre-defined and for the most part immutable.

So, the truth is, chaining provides a syntactically prettier (more expressive) code approach, but it limits us a little bit in terms of coding flexibility.

I need more concrete

To illustrate, let’s revisit jQuery’s example code from above. Let’s say instead that what you really want to do is only run the addClass(…) call if some condition is true (like for instance, the user has a valid login session, or whatever). So, you want to basically do conceptually something like this:

$("p.neat")
if (userLoggedIn()) {  .addClass("ohmy")  }  // invalid syntax; will not work!
.show("slow");

As I mentioned, you could just have two different chains, one with addClass(…) in the chain, one without it, and choose which chain to execute with your if-statement. This leads to more and repetitive code; a bad thing. DRY (don’t repeat yourself).

And to be honest, you could also model this in a pure jQuery way (only in the chain using various filters), a variety of different ways. For instance, something like this (while ugly) might work:

$("p.neat")
.filter(function(){
   return isLoggedIn();
})
.addClass("ohmy")
.end() // ends the most recent filtering operation
.show("slow");

What I did here was use filter(…) to basically conditionally empty out the internal collection if not logged in (by returning false for all iterations), meaning the addClass(…) call is essentially ignored, and then revert back to my original collection with a .end() call before lastly calling .show(…).

Let me be clear: while you can do it this way, this is by far not the most efficient or code-friendly way to do so. This would be an example of an anti-pattern, where you are abusing jQuery’s syntax to solve a problem in a way that it should not be solved. It goes contrary to the spirit of jQuery (“write less, do more”).

Other examples?

Conditional inclusion of “chain links” is only one scenario where traditional chaining is kind of limiting. What if you want to be able to piece together different parts of a chain from various different snippets of code across mutliple functions? This may sound strange, but with complex Web 2.0 web applications, this kind of thing is not uncommon. And it flys in the face of the simple syntactic sugar that chaining is supposed to give us.

Nor are these problems limited to jQuery. With LABjs for instance, a chain might look like this:

$LAB
.script("framework.js")
.script("loggedin.js")
.wait(function(){
  // do something
});

Notice there I may want to only load “loggedin.js” if the user is, in fact, logged in. Same syntactic problem as above. But LABjs has no conditional “filters” that can modify the chain’s context, so we have less options (which in this case is a good thing!)

Or, what if I have a CMS system (like Wordpress for instance) that, across various functions, files, templates, etc, needs to load up a variety of different files, all based on various different environmental conditions and which templates are active, etc. Could I do individual $LAB chain calls in each location? Yes. But then I lose the ability to sequence the loads (ensure execution order). Also, it’s less memory efficient to do 8 different chains with 1 script in each, as opposed to one chain with 8 scripts in it.

Or, what if I want to use LABjs in a larger solution for creating robust fallback for resource loading from CDN locations? I need to conditionally decide during loading/run-time if I need to add more scripts to the “queue” because one load failed. Not possible with traditional chaining.

Unravel the ball of yarn

All this leads up to say that chaining is great and expressive and powerful, but it’s not complete for all our needs. So, we need to get a little more creative in those cases. And this technique is what I’m going to call “Simulated Chaining”.

Recall that at the top of this post, we talked about how chaining works because the return value from a function is either a function itself (and thus can be called directly), or it can be an object that can be de-refenenced and has properties which are functions to be executed.

So, our solution lies in that sentence (emphasis added). The return value from a function can either be used immediately, or it can be stored and used later. This isn’t anything new; if you’ve ever called any function in any language, you know it can return a value that you can store in a variable. What might have escaped you until now is that this return value that you store can be a contextually sensitive object that would otherwise have been used immediately in a chaining situation. And the cool part is it can be used later (like the next loop iteration) to pick up where the chain left off.

So, instead of pre-defining a chain in our code/build-time, we can instead programatically (at run-time) build up a “list” (an array/queue for instance) of functions to be called, and then “simulate” a chain by looping through the list, calling each function, and storing the return value for use in the next iteration.

For instance, the LABjs script code above can be done like this instead:

var queue = []; // empty array for now
...
queue[queue.length] = "framework.js";
...
if (isLoggedIn()) { queue[queue.length] = "loggedin.js";
...
var $L = $LAB;
for (var i=0, len=queue.length; i
I simulated a chain of function calls with an iterative for-loop. (NOTE: this is conceptually very similar to how you might take what would otherwise be a recursive set of function calls and turn it into a non-recrusive, iterative loop instead, like for performance reasons). The key is: store the return value from a function call, and then use that stored value as the object that makes the next call. Plain and simple. And now more powerful as you can build up your chains at run-time in as complex a way as is necessary.
Theoretically, a very similar approach could be taken for the jQuery examples above. However, I’ll leave that as an exercise for the reader.
Let’s look at one last slightly more advanced example of how to use simulated chaining with LABjs (which is nearly identical to how I do it on several of my sites):
var queue = [];
...
if (conditionOne()) {
   queue.push("framework.js",false); // false will create an empty .wait()
}
...
if (conditionTwo()) {
   queue.push("plugin1.js","plugin2.js",function(){
      // initialize plugins
   });
}
...
queue.push("mypage.js");
...
var $L = $LAB;
for (var i=0, len=queue.length; i
I specify a queue array that can have one of three types of entries: a [string] is interpreted as a script to load, a [false] is interpreted as an empty .wait() call, and a [function] is passed to a .wait(…) call. I can use any manner of complex logic to build up the queue, and then in one for-loop at the bottom of the page, simulate the chain by looping over the queue.
Hopefully this helps clear up some of the mysticism that is chaining and give you some other ways to approach the challenging use cases you may face in your code. Go forth and chain (or loop)!

A visual of page-load improvements (via LABjs)

getify — Thu, 03 Dec 2009 07:21:23 +0000

If you run a website but you’re not aware of Google’s free Webmaster Tools, you need to get in the loop! These tools allow you to analyze a number of important aspects of how your site is viewed/analyzed by Google’s search engine index.

Yet, with all the power and insight these tools give, it still feels like it’s best-kept-secret kind of thing that such tools even exist. I never hear anyone talking or writing about them. But I fire them up probably at least once a month to keep an eye on my various sites.

But the hush-hush of Google’s toolset makes it more exciting to highlight a really cool addition announced today, the new Site Performance tool. True, it’s still in development and very early on, but it already is showing the promise to give great new insights into how your site is performing out in the wild.

We all got the hint a few weeks ago that Google is planning to start considering page-load speed in its indexing algorithm. It would seem to be somewhat “evil” for the world’s largest search engine to start holding page-load speed against everyone’s sites without being part of the solution. So it shouldn’t be a surprise that the G would begin releasing various “tools” to help the rest of us actually make our sites load faster.

Bye, bye, birdie!

Yesterday, the Google Analytics team released a long-overdue update to their suggested snippet for including “ga.js” in pages. They wisely ditched the horrid “document.write()” approach that has been the standard for the longest time, in favor of this new asynchronous loading technique. The reason? Quite obviously, this now will unblock the loading of GA from page loads, thus preventing Google from being at fault for slower load times!

Side Note: the suggested code that Google has released has been shown not to be safe with all page structures in some browsers, including (ironically!) Chrome, and Opera. The problem is a faulty assumption that “document.documentElement.firstChild” will always return a reference to the HEAD element to append the

In this example, “myscript.init” is a function that is defined in “myscript.js”, and “framework.init” and “framework.doSomething” are both defined inside of “framework.js”.

The first pass at implementing LABjs to load these scripts may seem like this code would work correctly:

Why is my function reference undefined?

Firstly, the code “.wait(myscript.init)” seems like this is the correct way to tell LABjs to run “myscript.init” function after the scripts finish loading. But there’s a subtle problem: with regular

So, what we pass to “wait” is not a reference to the as-of-yet-undefined “myscript.init”, but an anoynmous function that will call/execute “myscript.init()” at the proper time. Because of how functions and closures and execution bindings work in JavaScript, this syntax will wait to try to resolve/execute “myscript.init()” until that function is guaranteed to have been loaded (because “myscript.js” finished loading!).

It’s a simple but devious concept in JavaScript… the difference between a function reference “myscript.init” and an anonymous function that will call the intended function later “function(){myscript.init();}”.

OK, but I still have undefined code

For the exact same reason that “myscript.init” wasn’t yet defined when we tried to pass it to .wait(…), the code in the second inline

We could do two separate .wait(…) calls, but that’s unnecessary in this example, so I just combine the two sets of inline code into one wait() function.

Think asynchronously

The key issue is the difference between immediately executing code (like in the original inline tag you are replacing, you should have a “.script(…)” call

For every inline script block with code in it, we need a “.wait(function(){ … })” call to wrap around the code

Hopefully that helps clear up some confusion you may run into!

LABjs: why not just concat?

getify — Sat, 28 Nov 2009 05:41:08 +0000

Update

[8/12/2010: I've posted a performance benchmark test to try and gather some numbers from the crowd-at-large on how single large concat'd files perform compared to chunking that big file into a couple of smaller files to download in parallel. I encourage you to go right now and click each of the buttons a few times, and then spread the link on to others. Concat Performance Benchmark Test #1]

By far the most prevalent question circulating right now in the wake of href="http://labjs.com">LABjs‘ initial public launch (IPL, I guess!?) is the understandable: “Why do I need a loader like LABjs for multiple files when I can just concat everything into one file?”

Fair enough, this is an important question to address. You shouldn’t just take my implied assumption at face value; we need to dig into this so you understand why I feel so strongly that LABjs offers something better than what you are probably currently doing on your sites.

The short answer is, BOTH! You should concat files together when possible, and you should load your file(s) with a loader like LABjs. If you only do one or the other, you have missed out on the bigger picture of page-load optimization.

YSlow/PageSpeed only value reducing HTTP requests

Before I get into specifics about why I think LABjs brings something new and beneficial to the table, let’s tackle what I consider to be an interesting but perhaps misleading “truth”. I am firmly in a minority camp in my skepticism that reduction of HTTP requests is the only answer to the question of how to load a site more efficiently — actually, to be more accurate, how to load a site’s JavaScript more efficiently.

I bet that you are recalling the YSlow rules and the tried-and-true practice that concat’ing all your JS files into one file for production is the surest way to speed up your page loads. And you’re probably doubting my credibility because I would question such a fundamental “truth” as HTTP request reduction.

It is true that, especially for the largest sites on the internet (millions of page-views per month and more), the sheer volume of HTTP requests from a typical page profile (dozens of images, multiple style sheets, half a dozen JS files, etc) quickly overloads internet connections, server load-balancers, the user’s device, and even the browser itself. And anything you can do to reduce that overload will go a long way toward helping the site load with acceptable performance.

Sites like Yahoo and Google have taught us the basic principles of HTTP request reduction as an effective way to cut down on the overall page loading delays. For instance, consider Spriting to combine images, CSS concatenation, and, yes, JavaScript concatenation. Some sites, like Digg, have even experimented with the extreme — inlining all page resources, even binary image data, into a single multi-part request! And just recently, we saw a proposal for Resource Packaging as another way to combine multiple types of data into single responses for HTTP request reduction.

If you run a site that has well over 50 million page views per month (I sure don’t!), you probably would assert that this is the easiest and most effective page-load optimization. And I’d be hard-pressed to go up against the brightest and the best at the world’s largest web properties to suggest that it’s not true, at least at their page-view volume.

What’s true is true. Period.

Here’s what I will suggest: maybe, just maybe, what’s true for Google at a hundred-million page views may not identically and exactly be true for the vast majority of us running sites in the tens-of-thousands page view realm. The “big boys” are up in rarefied air, and from there they certainly have the request volume to test, re-test, and “prove” these various rules of optimization. But in the same way that water behaves in strange, almost contradictory ways at extremes of hot or cold temperature, maybe the art of optimizing page loads isn’t as simple as “1 http request is always better than 2.”

Even for small sites (like the one you’re currently reading!), it’s still vitally important to reduce page load time and optimize for faster page views. So, I’ll readily and eagerly admit that a key strategy in that effort is for all of us to work to reduce our HTTP request loads.

OK, so you agree concat’ing is best?

Not so fast. Like I said, this issue is more complex than I think most of us are ready to admit. Too easily, we run the YSlow plugin tests on our site, and when we reduce HTTP requests by concat’ing our JS files, because it tells us to, not only does our perceived page-load times decrease, but our YSlow grade goes up! Success! We’ve proved the hypothesis, there’s nothing more to say, right?

I don’t think this in and of itself means you have achieved the pinnacle of page-load performance, you’ve simply taken the first of many steps toward nirvana.

In my opinion, loading your JavaScript in an efficient and optimized way cannot be boiled down into a simple set of “rules” — you can take such “truths” as good places to start, but you still have to refine the strategies to fit your particular site and audience. Indeed, this will start to look more like an art than a science, if done correctly.

I’m not so sure…

Maybe by this point, you’re still not convinced there’s more to the story, and you’re probably starting to feel like LABjs doesn’t really have that much to offer you.

Before you go, let me just say this: even if you only ever have one JavaScript file on your site, because concat’ing is the way to go for you, and you think that by putting a single