This is a space that holds a collection of my personal work and ideas

Package-lock.json Explained

Posted on 01/18/2021

Since the introduction of the package-lock.json file from NPM, I never had a good grasp of it which lead to my confusion when resolving a merge conflict that is due to the package-lock.json file. To understand it clearly becomes the first thing on my new year's resolution list. This is the way how I managed to get myself understood with the package-lock.json file.

Fields explained

These are the fields I found most important when I resolve a merge conflicts in the package-lock.json file. They provide me with the minimal information I need to identify what has been changed that is directly related to the version of the dependencies.

Field Description
name the name of the installed module
version the fixed version of the installed module
dependencies a list of modules that are installed in the node_modules folder
requires a list of module requirements, this is a list of key-value pairs where key is the module name and value is the semantic version range

Benefits of package-lock.json

Here are the four quoted benefits from the official NPM document with my interpretation:

Describe a single representation of a dependency tree such that teammates, deployments, and continuous integration are guaranteed to install exactly the same dependencies.

Unless the application author specifies the fixed version, it is not enough to control what exact version will be installed across the team. What's defined in the package.json only describes the version requirement, often time each dependency is provided with a version range instead of a specific fixed version. When running the npm install command, the version that is tagged as the @latest that satisfies the semantic version rules will be installed and its version is then captured in the package-lock.json file. The same logic is recursively applied to the dependencies of the dependencies. Now with the package-lock.json, when running the npm install, the exact version that is previously captured in the lock file will be used instead of the version tagged as the @latest. This guarantees the exact same versions of the dependencies are installed across the team.

Provide a facility for users to "time-travel" to previous states of node_modules without having to commit the directory itself.

Now that package-lock.json captures the exact versions that are installed for all dependencies, if this file is then committed to a version control system, it is now possible to check out any previous version of package-lock.json from the version control history and then will be able to find out any previously installed versions of the dependencies. A single file avoids the need to version control the package.json files from all the dependencies in the node_modules.

To facilitate greater visibility of tree changes through readable source control diffs.

Even though it is still possible to compare the diffs of package.json from each dependency in the node_modules if we choose to commit them, it is hard to find the relations between these dependencies without rebuild the dependencies graph from all the installed package.json file. With a single package-lock.json file providing the dependencies tree that captures what versions are installed, it takes comparatively less effort for us to find all the changes in one place. Note that it is also possible to traverse the single tree to build the dependencies graph, which I have a demo about it in the final section.

And optimize the installation process by allowing npm to skip repeated metadata resolutions for previously-installed packages.

There are a few metadata attributes also captured during the npm install process. For example, the bundled metadata makes it possible to skip installing the dependencies that are already installed from another dependency by the parent module that is declared as bundled: true. There are some more metadata attributes that provide other useful information during the installation.

Package-lock.json visualizer

With a single dependencies tree provided by the package-lock.json file, it is possible to traverse it and build the dependencies graph based on the previously mentioned key attributes. We can start from the root module and do a depth first traversal, and for each node we visit, add them to the node definition, and after visiting each node and its descendents, add the node-descendent relation to the edge definition.

The graph layout algorithm is done by using Dagre, the renderer is done by using DagreD3, and the zoom is done by using d3-zoom.

The original algorithm is from the Package-lock.json Visualizer by Mike Bostock.

If the size of the package-lock.json file is greater than 300KB, it may take a while for this renderer to draw all the SVG out. This is just for demonstration purpose, not meant for production.

Read on GitHub