Adaptive Hybrid JavaScript

The ideas I’m about to dump here are not new. I came up with them 4+ years ago (and it probably was not unique to me), but I coined a term for my formulation of it: middle-end. That link redirects you to a series of posts I wrote back then exploring my ideas. I also gave about a dozen conference talks in the same time period.

It’s 4 years later, and I still think we haven’t even begun to scratch the surface of what I hope(d) we’ll eventually get to. And worse, we now have a bunch of hollow buzzwordy hype around a new term (which I really hate): “isomorphic JS”. Why do I say “worse” and “hollow”? Because none of the discussion around isomorphic JS is even remotely addressing the true potential, and also because the term itself doesn’t even make much sense.

“Isomorphic” roughly means “same structure, different value”. The closest we could think of for that in this context is if you had a server-side MVC architecture in something not JavaScript, and then duplicate the entire MVC stack in JS in the browser. But what the hypesters are saying “isomorphic JS” means is just JS that runs in both browser and server.

It’s like buying a famous novel and having a discussion about the contents of the copyright page inside the cover.

Middle-End

Let me give a glance at what I mean by “middle-end” (want more detail? see “what exactly is the middle end?”).

All web applications have certain tasks buried in various depths of their server-side back-end, tasks such as templating, URL routing, (stateless) data-validation, data formatting, header management, resource packaging, etc. Depending on your framework or platform of choice, these tasks are usually inextricably woven into the bowels of back-end code.

Why does this matter? Because many of those same tasks need to also run in the browser, which means you almost always have to duplicate them (in JavaScript). In fact, I often describe the “middle-end” as the top 10% of what happens on the server, and the bottom 10% of what happens in the browser, and everything in between.

While this duplication between server and browser sucks, it’s far from the worst part of common web architecture.

These tasks require intimate knowledge of the unique challenges of client-side functionality, which only gets more complicated the more diversity we see in client devices, from phones to tablets to laptops to glasses to watches.

JavaScript developers are best suited to managing middle-end tasks, which means that JS is uniquely the best language for those tasks to be implemented in. But when middle-end tasks are handled way down in the guts of your .Net or Java platform, and front-end’rs have no visibility or control over them, the front-end capability always suffers.

Pulling these functionalities out of the back-end (whatever it’s implemented in) and building them in a JavaScript middle-end puts them squarely in the control of those who know and can handle them best. Another motivation is that 90% or more of front-end web optimizations require control over middle-end functionality that front-end developers often don’t have!

The middle-end architecture I’m suggesting insists that server-side JS (probably in node.js) is the best (not the only, of course!) way to implement these tasks.

For an existing application stack already implemented in something non-JavaScript, the easiest and most natural way to consider integrating server-side JavaScript is to pull these sorts of middle-end tasks out of your back-end (and front-end!) and implement them in JS. The whole stack doesn’t have to be JS, and many times, it wouldn’t make sense to rewrite as such. But the middle-end is a perfect candidate for the JS way of life!

Hybrid

Server-driven apps have the drawbacks that they can’t fully take advantage of the capabilities of the client. Client-driven apps (SPA’s) have the drawbacks that they discard any substantive role/capability of the server beyond REST APIs, which means they are entirely bound by the capabilities of the client, limited or advanced as they may be.

And there’s plenty more drawbacks on either side which I won’t waste effort re-counting.

So what is “hybrid”?

The middle-end lets us leverage the best of both sides, and minimize the drawbacks. We can render on the server or in the client with exactly the same code. We can validate incoming data on either side with exactly the same code. We can route app logic on either side with exactly the same code.

In short, code is hybrid when it lives neither only on the server nor only in the browser, but in both equally.

Hybrid apps are apps that live on the middle-end, which straddles the gap between server and client, and in most respects attempts to abstract and erase the boundary. They let the client do what it does best (interact with the user) and the server do what it does best (track & persist state). And everything in between, they let both sides divide up the work as necessary.

The initial page request to a hybrid app is fielded by a server-side JS middle-end that routes the request as appropriate, consults the back-end for any state or data (whether that back-end is JS or not), and then uses templating to formulate an appropriate full-page response.

Once that full-page response populates the client, the front-end begins to load up the appropriate bits of middle-end machinery so that the client can take over rendering (templating), validations, etc.

Since the middle-end drives the front-end (and vice versa), and the middle-end is the same JS code used on either side of the wire, the application interacts with middle-end logic the same no matter where the code is executing.

Hybrid sites/apps give us the best of both worlds, but with simplicity that beats either alone.

Adaptive

But we still haven’t begun to explore what hybrid apps can really do. Hybrid apps need to become what I call “Adaptive”.

Most apps that bother to do any kind of feature-testing to control how the app will behave do so only once, at the beginning of the interaction. This is far too limited.

An Adaptive app instead constantly monitors how it behaves, and adapts itself to ever-changing conditions.

As one example (there are many more!), an adaptive app could render the initial page response on the server, then begin rendering the rest of the pages in the client. But what makes Adaptive really powerful is monitoring how well it’s behaving/performing, and adapting as necessary.

When the page loaded, the device might have had full battery and strong radio signal. But now, the device is low on battery, so the CPU is throttled back, and so is the radio signal. Now, the rendering of UI elements on the client is not performing well. It’d be better for the app to switch back to server-side rendering, with less frequent requests.

Luckily, an adaptive hybrid app can do that without breaking a sweat!

The power of adaptive hybrid middle-end design is that it erases the lines between client and server, so there’s just one middle-end code base, and it runs however it needs to, wherever it needs to.

Barriers

The picture is not all roses for adaptive hybrid apps. I see two main hitches that keep us from fully realizing this capability.

  1. Session Cookies vs. sessionStorage

    I wrote extensively about this issue in “Tale of Two Ends”, including a proposal to address the shortcoming. So far, no movement, but I still have “hope”.

    Briefly restated, the problem is that session cookies, which are primarily used to store session IDs, have many, many draw backs in modern web architecture. Cookies in general are ever-more ostracized as detrimental to user privacy, and in some jurisdictions they are downright illegal. They also have substandard behavior and legacy interaction APIs straight out of 1996.

    sessionStorage solves all of these problems in very desirable ways, but there’s one main showstopper. If you store a session ID in sessionStorage, only client-side JS can access it. The session ID will unfortunately not be transmitted to the server in the HTTP request layer of an initial page request, which means our hybrid middle-end server code cannot construct a stateful page response because it has no session ID.

    So the choice is use sessionStorage, and any session-aware pages must defer their rendering to the client (a lesser UX from the delay), or go back to session cookies and swallow all those negatives. This sucks.

    We have to fix this.

  2. “Client Hints”

    One key necessity of erasing the boundary between browser and server so adaptive hybrid apps can shine to their full potential is that the server-side of the middle-end has to be able to make all the same sorts of decisions about client capability as the browser JS can do.

    Right now, that’s only possible in hacky ways like user-agent string parsing + huge guesses, or extra round-trips where JS beacons detection results back to the server, or other exotic craziness.

    Client Hints is a fledgling idea to send information to the server, such as client screen metrics (DPI, etc). The primary motivation seems to be to enable “responsive design”.

    But I think this falls way too short. We must have the ability to let a server signal to the client all the same things that the client would detect for itself to make proper rendering decisions. That is the only way a server-side middle-end can adequately handle the client-side load when necessary.

Wrap

Drop the hype of hollow phrases like “isomorphic JavaScript”. There’s so much more potential than just cross-compiling a module with browserify so it runs “everywhere”.

Let’s embrace the untapped potential in Adaptive Hybrid JavaScript, where our apps embrace the unique power of a dedicated middle-end tier that spans both sides of the wire — apps that can finally erase the frustrating UI architecture limitations of differentiated browser and server.

Let’s advocate standards and browsers to fix the barriers that are holding back the middle-end. Let’s make apps that realize the power of server and client working together as one unified delivery platform. Let’s make web applications, not just browser or server applications.