Choices.js

In the last six months I’ve had my first real taste of what it’s like working in the modern JavaScript ecosystem. The site my team is creating isn’t a single-page app — rather, we’re applying progressive enhancement to standard HTML pages (rendered on the server with Handlebars.java) — but the UI designers and developers have created a bespoke set of tools for defining composable blocks of markup with which to design the site, and a fairly basic component browser to allow them to work on the markup and styling, with dummy data. It won’t surprise anyone to find that for these design tools we are using Node.js and npm.

I’ve found while working in this new JS world that several things stand out for me:

There are several, or even many, choices of library/tool to accomplish any given task, and there is rarely a stand-out obvious/best option. The tools frequently wrap or layer on top of other tools, and the documentation tends to assume you fully understand the next layer down. I’ll try to illustrate what I mean. This may be overwhelming, or just plain dull, if you’re unfamiliar with the tools and technologies described. If you lose track, just skip to the end, because that’s pretty much my point…

At some point along the line my team decided to get ready for the future by using ECMAScript 2015, better known as ECMAScript 6 or just ES6. It has lots of new syntactic sugar as well as genuinely new language features; a nice reference is es6-features.org.

Mainstream browsers don’t fully support ES6 yet, so we need a “transpiler” to convert ES6 code into the equivalent ES5 code which will actually run in most browsers. The two main options are Traceur (from Google) and Babel. We’re using Babel.

But it’s not just new language syntax in ES6; there are new types like Promise and new methods on built-in types (such as String) which are expected to be provided by the JS runtime; in other languages you might refer to these things as the “standard library” or similar. We need to make sure those are there, too. So we need one or more “polyfills” to enhance the standard library. Babel includes core-js, but a quick Google for “es6 polyfill” reveals babel-es6-polyfill as well as several alternative Promise polyfills.

Now, well-behaved JS developers write modules rather than having one massive file and to avoid naming clashes in the global scope. How should we define our modules? CommonJS and AMD are popular choices, but ES6 introduces its own syntax for modules via import and export keywords, and a separate module Loader standard. We’ll use the ES6 module format as that’s nicely in keeping with our other ES6 stuff.

There’s an implementation of the Loader in the form of the ES6 Module Loader Polyfill, which can make use of transpilers in the browser, but what if we want to make use of existing (non-ES6) modules? Fear not: SystemJS is a “universal dynamic module loader… built on top of the ES6 Module Loader Polyfill”. Translation: you can mix and match scripts using any of the popular module formats. (It has some other features too, but I won’t discuss those today.)

Great, but we will have loads of JS modules and we don’t want to have dozens of requests for the (transpiled) scripts on each page. Can we bundle the scripts together? Oh, and while we’re about it, how should we be managing dependencies on 3rd-party modules (which themselves may depend on other modules, and so forth)?

We can use jspm, “a package manager for the SystemJS universal module loader, built on top of the dynamic ES6 module loader”. It promises “frictionless browser package management”. This (installed using npm) offers a command-line tool which helps fetch packages, update the SystemJS config file, and bundle JS modules together – with an option to create a “self-executing bundle” which obviates the need for your pages to load SystemJS and config files before loading your own JS. Other possibilities (as far as I can tell) include Browserify and Webpack.

Gulp is our build tool — though it could have been Grunt, or perhaps Broccoli or Brunch or Gobble — so we just have to install a Gulp plugin for jspm. Let’s Google ‘gulp jspm’… OK, top two hits from the npm site are gulp-jspm and gulp-jspm-build. How do we choose? Download figures aren’t dramatically different, but gulp-jspm is higher, so we’ll give that a go.

We’re nearly there… but if you need to give your brain a short break at this point, go and look at some kittens or puppies or whatever makes you happy.

Still here? OK, to summarise, we’re writing ES6 code and using ES6 module syntax, using 3rd-party modules installed/managed using jspm. Using gulp-jspm we’ll bundle our code, along with its dependencies, into one or more combined files; jspm will invoke Babel along the way and will bake in a minimal SystemJS module loader. Babel, in turn, will also include the core.js polyfill for us.

All of this may be accomplished with the following Gulpfile:

var gulp = require('gulp');
var jspm = require('gulp-jspm');
require('babel-core/register');

gulp.task('bundle', function() {
  return gulp.src('src/js/app.js')
    .pipe(jspm({selfExecutingBundle: true}))
    .pipe(gulp.dest('build/assets/js/'));
});

and this SystemJS config file, largely auto-generated by the jspm command line tool:

System.config({
  baseURL: "/",
  defaultJSExtensions: true,
  transpiler: "babel",
  babelOptions: {
    "optional": [
      "runtime",
      "optimisation.modules.system"
    ]
  },
  paths: {
    "github:*": "jspm_packages/github/*",
    "npm:*": "jspm_packages/npm/*"
  },
  map: {
    // client-side module dependencies here
  }
});

Those few lines in the Gulpfile mask a huge amount of complexity. To someone joining the team, unfamiliar with these tools, it’s pretty hard to figure out how all of this is done. I still don’t have complete confidence when I try to explain which tools are responsible for what — if I’ve got any of this wrong, please comment and let me know! — and for so many of these choices that someone made along the way, I have no idea why (or if) option A is better than option B. And you just know that, in a few months, Gulp will be yesterday’s news and everyone will be jumping on the cool new build tool developed at Etsy or KickStarter or something, and the main developer of that obscure but critical module you introduced to solve a problem will have stopped updating it.

Is it just my unfamiliarity with the JavaScript world, or is there really more choice here than in the Java ecosystem, and less information to help you decide between options? I get the impression that Not Invented Here syndrome is endemic in the Node/JS world. In a Java project, pretty much everyone uses at least one of the various Apache Commons libraries, perhaps supplemented (but probably not replaced) by Google’s Guava. As far as I can tell, there aren’t such de facto standards in the JS world.

Perhaps it’s just an age difference: Node.js is approaching its 7th birthday while Java is three times as old. But perhaps all those NPM modules are a product of the GitHub generation; it’s incredibly (worryingly?) easy to create a module and publish it to NPM. Contrast that with the famously painful process of getting a JAR published onto a well-known public Maven repository.

Will the Node.js world become less fragmented as it matures? Will we see a few major frameworks and libraries dominate? Will Node ever support ES6 directly? We can only wait and see…

Addendum: just as I finished writing this, I happened upon a Github repo, es6-tools, where someone is trying to collate and categorise tools for working with ES6. I really wish I’d found this earlier!

Image credit: “Too Many Choices” by Elvis Kennedy is licensed under CC BY-NC-ND 2.0