How I packaged my JS library

When I have spare time, I work on an open source audio feature extraction library in Javascript called Meyda. Over the years, I've had several occasions of getting really confused about how JS modules work, and how I should package the library on npm. Today is one such occasion. But this time, I'm writing down what I'm learning about JS packages so that next time, I will be able to quickly get through it!

What I need to do?

I need to ship Meyda to npm so that people can use it in all their lovely projects. However they want to use the library, I want it to be possible for them. Additionally, Meyda is composed of lots of small js files, each for its own audio feature. It would be pretty unfortunate if someone wanted to use Meyda to run only one feature extractor, but in order to do so they had to bundle all the feature extractors. So I'd like to enable tree shaking.

There are several different situations people might want to use Meyda.

1. As a library in nodejs
2. As a library in a fancy web app
3. As a global variable in their less fancy web app, included in a script tag

The required reading

Each of those have different requirements. To solve those requirements, we'll create the matrix of doom.

There are two major axes of decisions to make here. The first axis on the matrix of doom is the module type. I'm only going to care about two types of modules, the ones we need to solve this predicament. CommonJS is the module type when your code does module.exports = myAwesomeLibrary, and users of your code do const myAwesomeLibrary = require('my-awesome-library'). ES2015 modules are the ones where your code does export default myAwesomeLibrary, and users of your code do import myAwesomeLibrary from 'my-awesome-library'. Users are pretty flexible. In ES2015, you can import [CommonJS modules with import]babel-import-commonjs+ (if your runtime doesn't support it, you can babel your code, and you will be able to import a cjs module) or with require. In commonjs, you can import other CommonJS imports, but you cannot import ES2015 imports.

Adjacent to types of modules is the whole "uuhhhh, back in the 90s js didn't have a module system so we just run a script in the same runtime as we'll run our code and it'll just assign a global variable and we'll pick it up in our code and it'll all be fine, right?" style of modules. That's not really a module system, but I'll stick it in the module axis on the matrix of doom because that seems like a reasonable place to put it.

The second major axis of decisions to make on the matrix of doom is whether or not to bundle your code. Bundling will take all your code, and all of your dependencies, and bundle them into one big js file. This can be advantageous if your runtime isn't able to import or require modules, like in old browsers (new browsers can handle modules now but old ones can't, so this is still useful, or if you don't want your web app to be making loads of requests to recursively download all your dependencies one by one.

The application of the required reading to the goals

web scale solutions hat on

Lets have a look at the matrix of doom

Now we need to find places in the matrix of doom that would solve for each of our 3 use cases.

First off, old websites that need a global variable. These should be bundled, because if a website is including a library with a script file, they're probably not supporting import infrastructure. Lets mark off a spot in the matrix of doom.

Next up, lets do Node. Node seems to be able to handle unbundled modules. They also seem to support both CommonJS files, and ES2015 files (with a .mjs extension). In order to start adopting new features, lets do both.

.mjs

Finally, modern websites. Lets take for example a create-react-app site. In create-react-app sites, you can import or require libraries that you install with npm, and CRA will do some compiling and bundling and make it all work for you. If you consume ES2015 libraries, CRA/webpack will tree-shake your imports, so that you only have the code that you need making it into your bundle. Nice! Lets add it to the matrix of doom.

.mjs

ok so the solution was....

I can use a tool like rollup to do the bundling and global export for Classic websites, so that's that one checked off. Classic browsers don't really care about your package.json, so you don't need to specify that export anywhere. You may wish to document an unpkg link to your bundled minified code, like https://unpkg.com/meyda@4.2.0/dist/web/meyda.min.js.

Meyda is written in ES2015 modules. So I'll need to use a compiler of some kind, probably babel, to convert it to unbundled CommonJS. That checks off CommonJS Node. I'll then specify the 'main' property of my package.json to point at the unbundled CommonJS code.

Meyda's ES2015 modules all use .js extensions, so I can't export those directly, I'll need to first copy them to the build directory with .mjs extensions. Frontends can use .mjs files as if they're Javascript files, so that's good. It seems like I can point to an unbundled copy of Meyda using ES2015 modules with .mjs extensions using the module field of package.json, so this course of action should work without breaking imports in node (node can't import ES2015 modules unless they're in files ending in .mjs.

According to this blog post I found about main, jsnext:main and module in package.json, I should export my CommonJS unbundled copy of Meyda in the main field of package.json, and I should export the ES2015 files (as .mjs files (I added that part)) in both the js:next and module fields of package.json.

So now that I know what to do, I'll go do it. Want to know how it worked out? Tweet me

Bonus quest

I was working on a hobby website to help me organize my GitHub repos, and needed to import the [@octokit/graphql] dependency so that I could make queries to GitHub's graphql API. I installed the dependency into my create-react-app based site, and went to import it. When I ran my site, inside a dependency of the graphql library, I got an error: isPlainObject is not a function. I set a breakpoint and went into the code, and realized that isPlainObject was in fact not a function, it was an object with a single property, default, which was my function. That's what you get when you use require to load an ES2015 export using webpack!

What had happened was that the subdependency, is-plain-object, had recently changed such that it's exporting both an ES2015 module, and a CommonJS module. Webpack knew that my code was using ES2015 modules, and this package was using ES2015 modules, so it imported the ES2015 module from that library. Unfortunately, the library that was importing is-plain-object was expecting a CommonJS module, so when it received an object with a property called default that was the function it was looking for, rather than just the function it was looking for, it threw an error. The solution in this case was to write a PR to the intermediate library that made it explicitly import the CommonJS module that is-plain-object exported.