Inlining package version into ES6 npm packages using sed

Here’s how I’m using sed to inline the version field from an npm package.json into my code in an ES6 package. It’s not very exciting, which is probably clear from the exhilarating title.

Lately, I’ve been writing most of my JS in ES6. It’s just a nicer language than ES5. It’s not as small, which is a shame; but it’s clean. It has a built-in module system, only half a century after the concept was invented. And by using Babel and a prepublish script, it’s trivial to distribute a library without having to worry about compatibility. People have been doing this for years with CoffeeScript, so it’s nothing new.

Here is an excellent blog post by James Allardice about how to write npm packages using ES6. So if you’re looking for a place to get started, read that first. Just keep in mind that if you’re writing a library that will be used in many environments, you should use the Babel runtime transformer and make babel-runtime one of your package dependencies.

The problem

I want the client library I’m writing to facilitate using a REST service to include its version number in a user agent string so that it will be easy to see from server logs which version of the library is making a request.

N.b., the User-Agent header cannot be set in browser-based Ajax requests, but if your library is meant to work both client-side and server-side, you may find it useful to add that header in server-based requests.

A first pass at a solution

My immediate thought was to simply import the package.json file into my code and use it directly:

// File: src/client.js
import library from '../package.json';
const agent = library.name + '/' + library.version;
export default class Client() {
    // Client code goes here.
}

And this is sometimes fine if you know all the potential environments for your library. But I found that when I imported my client library into a project that uses webpack to bundle modules, that it chokes trying to import a .json file. This can be resolved in webpack.config.js, but it just doesn’t feel right to impose a burden on the consumer of a library. The issue arises in the compilation of ES6 import statements into some variety of require function based on the module bundling tool that might be employed.

A better way

So I stepped back and thought about what I am actually trying to accomplish: I want to compile ES6 code down to ES5 and I want some basic string substitution, the sort of a thing a preprocessor might do. If the situation were very complex, I might be tempted to use something like grunt or gulp or even make. No, really! Sometimes make is a fine tool for JS projects.

But my needs are vastly simpler. I’m already running my code through a compile phase before publishing it to the npm registry, so there is an obvious place where I can further transform the code: the prepublish script.

In this particular case, preprocessing my original source before compiling it means I can replace the string literal AGENT with the actual user agent information. And because the preprocessing happens before compilation, my source map will contain the correct user agent information as well.

So the change to the client source code is:

// File: src/client.js
const agent = 'AGENT'; // This gets replaced.
export default class Client() {
    // Client code goes here.
}

And the scripts section of my package.json looks like this:

{
    "babel": "babel --optional runtime --source-maps --out-dir lib/ src/",
    "compile": "npm run preprocess && npm run babel && npm run postprocess",
    "preprocess": "sed -i.b \"s|AGENT|$(node scripts/agent)|g\" src/client.js",
    "postprocess": "mv src/client.js.b src/client.js",
    "prepublish": "npm run compile"
}

Let’s consider the three compilation stages (preprocess => babel => postprocess) in closer detail.

Stage (1 of 3): preprocess

This stage needs to generate the correct user agent string and substitute the AGENT token in the original source file with the actual user agent. To do this, it’s running this sed command:

$ sed -i.b "s|AGENT|$(node ./scripts/agent)|g" src/client.js

There’s a lot happening here. The s command in sed is the substitution command, which replaces the first argument with the second. Customarily, a string that looks like regular expressions is used. As with other regular expressions, the g flag at the end means that the substitution should be applied globally:

$ sed "s/REPLACE-THIS/WITH-THIS/g" src/client.js

But because the user agent string I want contains a / character (e.g. MyClient/1.0.0), I’m using a different delimiter string, |. So the example command above could be written as:

$ sed "s|REPLACE-THIS|WITH-THIS|g" src/client.js

I also want to replace the original source file before feeding it to the Babel compiler, so I’m using the -i flag to indicate that I want an inline replacement and adding a .b modifier to that flag to indicate that the original file should be backed up with the original file name plus .b, i.e. client.js.b:

$ sed -i.b "s|REPLACE-THIS|WITH-THIS|g" src/client.js

And finally, I’m using shell command substitution to take the output of a helper script (which, crucially, can require the package.json file because it is running in node) and using that as my substitution value. So here is the final command again:

$ sed -i.b "s|AGENT|$(node scripts/agent)|g" src/client.js

And my helper script looks like this:

// File: scripts/agent.js
var library = require(__dirname + '/../package.json');
console.log(library.name + '/' + library.version);

Since it’s a trivial script, it’s plain old ES5, but I could have used the babel-node command if I wanted to write it in ES6.

Stage (2 of 3): babel

By contrast, the actual compilation stage is pretty simple:

$ babel --optional runtime --source-maps --out-dir lib/ src/

It adds babel-runtime to the library so that any ES6 primitives that don’t exist in ES5 get polyfilled and also generates a source map for my library. It compiles the source files in src and writes the output into lib.

Stage (3 of 3): postprocess

And finally, this step just replaces the modified source file with the original:

$ mv src/client.js.b src/client.js

Conclusion

This is all pretty straightforward command-line usage, but it’s good to remember that utilities like sed have existed for decades. And sometimes the simple, built-in toolchains that are available are better options than the complex build tools that are commonly used in the JS ecosystem.