Why JavaScript web applications should embrace traditional URLs
In this article, I will first take a high-level look at modern frontend architectures: In a time where web apps easily surpass 1 MB of JavaScript, what should we try to achieve?
Second, based on these considerations, I’m going to argue that Backbone.js should fully support the traditional HTTP URL scheme.
The ideal web site architecture
Today’s typical web site architectures can be placed between two extremes, one being traditional server-side logic, the other being JavaScript-only single-page apps. In between, there are hybrid approaches. Pamela Fox does a great job of describing these architectures and their pros and cons. She also introduces some key requirements from the user’s perspective: Usability, Linkability and Searchability/Shareability. In her presentation, she gives a quick overview of how the architectures perform. This outlines the current situation quite well.
How should a modern site work? There are several reasons why one should combine the best of all approaches: Server-side robustness with a client-side turbo-boost. In practice, we run into problems when trying to share logic between server and client. I think this is an engineering problem that can and will be solved in the future.
So what is the key to future architecture? I think it is Progressive Enhancement from soup to nuts. Progressive Enhancement is still useful and necessary. A typical site should be able to fulfill its basic purpose somehow even without JavaScript. A machine that speaks HTTP and HTML should be able to read a site. Of course, modern web sites aren’t about static content, but about user interactivity. But in most of the cases, there are resources with a static representation, either text, video or audio.
In order to achieve Searchability and also performance, content needs to be rendered on the server to some extent. Twitter learned this lesson the hard way in the “#NewTwitter” days, when they experimented with completely client-side architecture, but ultimately went back to serving traditional HTML pages for each initial request. Still, twitter.com is a huge JavaScript app. JavaScript operates on top of the initial DOM and then takes over to speed up subsequent actions. Hopefully, we’ll see this hybrid approach more and more in the future.
Rendering HTML on the server-side is considered valuable again. That’s because the traditional stack of HTTP, URL and HTML is simple, robust and proven. It can be incredibly fast. It works in every user agent; browsers, robots and proxies are treated uniformly. Users can bookmark, share and save the content easily.
Cool URLs are cool!
Used correctly, URLs are a great thing. Web development is centered around them: Cool URLs don’t change, URLs as UI, RESTful HTTP interfaces, hackability and so on. The concept of HTTP URLs dates back to 1994.
When “Ajax” appeared in 2005, people quickly realized that it’s necessary to reflect the application state in the URL. For a long time, JavaScript apps weren’t able to manipulate the full URL silently without triggering a server request. To achieve Linkability, therefore, many JavaScript apps are using “Hash URLs”. It’s safe to set the fragment part of the URL, so this became common practice. Most JavaScript libraries for single-page apps still rely on Hash URLs. Among others, Backbone.js uses Hash URLs per default in its routing implementation.
Today we know that Hash URLs aren’t the best solution. In 2011 there was a big discussion after Twitter and Google introduced Hash URLs and “Hash Bang URLs” in particular. Most people agreed that this was a bad hack. Fortunately, HTML5 History (history.pushState and the popstate event) makes it possible to manipulate the URL without leaving the single-page app. In general, Hash URLs should only be used as a fallback for older browsers.
If you use pushState, all URLs used on the client need to be recognized by the server as well. If a client sends a request such as GET /some/path HTTP/1.1, the server needs to respond with a page that at least starts the JavaScript app. In the end, making the server aware of the request path is a good thing. Instead of just responding with the code for the JavaScript app as a catch all, the server should better respond with useful content. In this case, traditional URLs enable Searchability and Shareability. Take for example an URL like this:
http://www.google.com/search?hl=en&ie=utf-8&q=pushStateThese kinds of URLs are a well-established standard, widely supported, and can be handled on both the server and the client. So my conclusion is: Future JavaScript-heavy web sites may be “single page apps” because there’s only one initial HTML document per visit, but they still use traditional URLs.
Backbone.js and query strings
Backbone.js has a great History module that observes URL changes and allows to set the URL programatically. However, it doesn’t support traditional URLs completely. The query part (?hl=en&ie=utf-8&q=pushState), also known as query string, is ignored when routing. In this second part of the article, I’d like to discuss the ramifications of this missing feature.
Backbone treats /search?q=heaven and /search?q=hell as the same URL. This renders the query string useless. You can “push” URLs with different query strings, but if the user hits the back button, Backbone won’t consider this as a URL change, since it ignores the change in the query string.
Chaplin.js, an opinionated framework on top of Backbone.js, tries to work around this by parsing the query string into a Rails-like params hash. But it ultimately fails to support query strings because of Backbone.History’s limitation. Full disclosure: I’m a co-author of Chaplin.
The lack of query string support in Backbone is deliberate. The maintainer Jeremy Ashkenas decided against it. In several GitHub issues, he provides rationale:
In the end, I think most Backbone apps should definitely not have query params in their app URLs — they’re a server-side URL convention that doesn’t have much useful place in client-side routing. So we shouldn’t be supporting them by default — but if you want this behavior, it should be easy enough for you to implement
Backbone shouldn’t be messing with the search params, as they don’t have a valid semantic meaning from the point of view of a Backbone app. If you want to use them (on a page that has a running backbone app), that’s totally fine …
In the most recent issue, Jeremy points out that this is not a browser compability issue:
wookiehangover: The thing that’s problematic about this (and why querystrings are ignored as of 0.9.9) is due to a handful of very weird but very real bugs with querystring processing and character encoding between browsers.
Nope. Not in the slightest ;)
The reason why querystrings are ignored by Backbone is because:
Querystrings only have a defined meaning on the server-side. The browser does not normally parse or otherwise handle them.
While querystrings are fine in the context of real URLs, querystrings are entirely invalid in the context of #fragment URLs. Most Backbone apps deal with fragment urls sooner or later — even if you’re using pushState for most of your users, IE folks will still have fragments. So querystrings can’t be used in a compatible way.
Better to leave them out of your Backbone app, and use nice URLs instead. If you must have them for the server side of the equation, that’s fine — Backbone will just ignore them and continue about its business.
Party like it’s 1994!
I’d like to answer to these statements here. First of all, it’s great to hear that there are no major browser issues blocking full URL support. Jeremy argues against query strings on another level:
Querystrings only have a defined meaning on the server-side. The browser does not normally parse or otherwise handle them.
Honestly, I don’t understand this point. You can process a query string on the server, but you can do that on the client as well. There are cases where query strings are processed almost exclusively on the client, for example the infamous utm_ parameters for Google Analytics.
A URL is a URL. Wherever a URL appears, its parts have a defined meaning – there are Internet Standards which define the meaning. It doesn’t matter which software generates the URL and which processes it, a query string should have the same meaning.
While querystrings are fine in the context of real URLs, querystrings are entirely invalid in the context of #fragment URLs.
This assumes that Backbone apps use Hash URLs instead of pushState. Well, most of them do and that’s indeed a source of pain. But technically the query string ?foo=bar is entirely valid inside the fragment part of a URL.
A URL like http://dahl.example.org/#search?q=matilda may look weird, but it is completely in line with RFC 3986. With pushState, you don’t have to think about URLs in URLs. You can use URLs like http://dahl.example.org/search?q=matilda. This is the form of URLs that has been around since 1994, for good reasons.
… even if you’re using pushState for most of your users, IE folks will still have fragments. So querystrings can’t be used in a compatible way.
Well, they can be used in a compatible way. It’s technically possible to put path and query string into a fragment. It might violate the semantics of traditional URLs, but syntactically, it’s still a valid URL.
Better to leave them out of your Backbone app, and use nice URLs instead.
Jeremy argues that client-side apps should encode query params inside the path, like
http://dahl.example.org/#books/order=asc/sort=published/That’s what he calls a “nice URL”. I beg to differ. In the spirit of 1994, why not stick to traditional URLs like:
http://dahl.example.org/books?order=asc&sort=publishedI see no reason why JavaScript apps should invent new URL syntaxes. Today’s JavaScript apps are using pushState and properly accessible URLs. They should not and don’t have to differ from the URL conventions that have been used since the beginning of the web.
It’s an RFC-compliant URL, there are plenty of server and client implementations to parse the query params into a useful hash structure. In contrast, if you use URLs like
http://dahl.example.org/#books/order=asc/sort=published/… you cannot use these implementations, but have to write your own “nice URL” parser instead.
If you must have them for the server side of the equation, that’s fine — Backbone will just ignore them and continue about its business.
If you’re building an app that has accessible documents, traditional URLs and query strings, most likely you need to process the query string on the server and on the client side. For such apps it’s not an option that the server understands them and Backbone ignores them.
My fellow Chaplin author Johannes Emerich pointed out another reason why Backbone should not limit the use of URLs:
In the end the point is that Backbone is said to be an unopinionated framework. But pushing for query params to be encoded as paths is anything but unopinionated or flexible.
There are many reasons why you would want to see those params on the server: Include some JSON data to be processed immediately on client-side app startup; render a full initial static document that contains all the data and only let the client-side app take over from there (for speed/SEO), etc.
In effect, this way of handling params in URLs is saying that Backbone is really only meant for completely client-side apps, and that you have to jump through extra hoops if you are going for a hybrid approach.
Of course, Chaplin and other code could monkey-patch Backbone in order to introduce query string suppport. But since Backbone claims to be “unopinionated”, it should just support traditional URLs instead of making query strings impossible to use. The ultimate decision for or against query strings should be the user’s, not the library’s.
In short, Backbone should support query strings because future-proof JavaScript apps are based on traditional URLs.
Thanks to Johannes Emerich (knuton) for feedback and input.
