Learning to Use Semantic-Release the Hard Way

21 Jul 2021 Learning to Use Semantic-Release the Hard Way

In this post I want to share a story. The story is about how I added semantic-release to an existing Angular library project. It’s also about some of the errors I encountered along the way, and how I resolved them. There aren’t many such stories about semantic-release out there, so I think others may benefit from hearing my experiences.

Background

We have a library built in Angular and there are two package.json files in the repo:

./package.json                       // For the demo project
./projects/ng-library/package.json   // For the library project

At the time when our story started, the major branches in the repo were:

• master / 2.x.x (with Angular 8 as peer dependency)
• 1.x.x (deprecated, with Angular 8 as peer dependency)
• 3.x.x (with Angular 11 as peer dependency)

Our CI/CD pipeline ran in Jenkins. For each release there were some manual steps, including updating both the package version and the change-log file.

When new code was merged to the major branches, a build kicked in with the following steps:

1. Check out the code
2. Build the library project and the demo project
3. Run unit tests of both projects
4. Run e2e tests of the demo project
5. Run static analytics for code quality checking
6. Run a vulnerability scan
7. Publish a new version when the package version of the library is bumped, and tag the new version in Git

Our version tags in Git looked like:

• 1.2.0
• 2.3.10
• 3.2.1

The manual process to bump the version worked well enough in the beginning, but it did not scale well when people from other teams began to make contributions to the library. Every time, we had to explain the process, even though we have a contribution.md file.

One of my colleagues from another team had been advocating semantic-release, and it had been used in other projects successfully for a while. So when we had to migrate the project to an Azure repo and pipeline for other reasons, we decided to add semantic-release as well.

What is semantic-release?

semantic-release is an NPM tool to automate the workflow to release an NPM package. It follows the semantic versioning specification strictly, including: determining the next version number, generating the release notes, and publishing the package.

semantic-release can be installed in your project with a single command:

$ npm install --save-dev semantic-release

semantic-release supports plugins, although by default it comes with the following four built-in:

@semantic-release/commit-analyzer
@semantic-release/release-notes-generator
@semantic-release/npm
@semantic-release/github

To create or update the changelog file, and to push a release commit and tag to the git repo, we also needed to install two additional plugins:

$ npm install @semantic-release/git @semantic-release/changelog -D

The next step was to create a ./project.json file:

{
  "name": "my-ng-library",
  "version": "0.0.0-semantically-released",
  "repository": "[git repo URL]",
  "private": true,
  ...
}

Note that you can also use the semantic-release-cli tool to do this.

Our project file also included the following configuration settings:

{
  ...
  "release": {
    "plugins": [
      "@semantic-release/commit-analyzer",
      [
        "@semantic-release/npm",
        {
          "pkgRoot": "dist/ng-library/"
        }
      ],
      "@semantic-release/release-notes-generator",
      [
        "@semantic-release/changelog",
        {
          "changelogFile": "CHANGELOG.md"
        }
      ],
      [
        "@semantic-release/git",
        {
          "assets": [
            "CHANGELOG.md"
          ]
        }
      ]
    ]
  }
}

We can use npx semantic-release in the pipeline to run the release job (or yarn semantic-release if you’re using yarn). It will analyse the commit message to find keywords like chore, fix, feat, perf to determine if a release is required. Assuming our current version is 2.1.0, here are some examples:

By default, semantic-release uses Angular commit message conventions. We installed husky with @commitlint/cli to create a git pre-commit hook to check that commit messages would be valid. That said, the commit message convention keywords can be changed in the configuration settings.

Once it determines the release type and decides that a release is required, semantic-release will then:

1. Update changelog based on all new commit messages
2. Commit the change
3. Create a git tag for the new version and push it to the git repo.
4. Bump the version accordingly
5. Publish the new version to our private NPM registry.

In the new Azure pipeline, we use the npmAuthenticate task to provide npm credentials to the .npmrc file in our repo for the scope of the build in the beginning.

Lots of errors!

After making all the settings described above, I began to test. Would semantic-release work properly in the build? I got all kinds of errors when I tried to release a new version in the 3.x.x branch.

Permission error

First of all was a permission error. Specifically, semantic-release tried to run the command “git push --dry-run --no-verify https://Example@dev.azure.com/Example/my-ng-library/_git/my-ng-library HEAD:3.x.x” but failed with the error message:

remote: 001f# service=git-receive-pack remote: 0000000000aaTF401027: You need the Git 'GenericContribute' permission to perform this action. Details: identity 'Build\xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', scope 'repository'. remote: TF401027: You need the Git 'GenericContribute' permission to perform this action. Details: identity 'Build\xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', scope 'repository‘.

And:

SemanticReleaseError: Cannot push to the Git repository.
code: 'EGITNOPERMISSION',
details: '**semantic-release** cannot push the version tag to the branch `3.x.x` on the remote Git repository with URL `https://Example@dev.azure.com/Example/my-ng-library/_git/my-ng-library`.\n' +

This happened because the Azure build service user requires “contribute” permission to push the code. So I set ‘Contribute’ to ‘Allow’ for the ‘Builder Service’ user in the security settings of all repositories:

That error was then resolved.

Existing Tags Are Not Recognised

Next, semantic-release reported the following:

SemanticReleaseError: The release `1.0.0` on branch `3.x.x` cannot be published as it is out of range. code: 'EINVALIDNEXTVERSION', details: 'Based on the releases published on other branches, only versions within the range `>=3.0.0 <1.0.0` can be published from branch `3.x.x`.\n' 

This one was unclear when I first saw it. The range >=3.0.0 <1.0.0 looked odd. I wanted to publish the new version on the 3.x.x branch and there were existing git tags for versions like 3.1.2. 

After digging into the source code of semantic-release, I found that it did not recognise all of our existing tags. Instead, it thought this was the first time the package was being published in this repo, so it came up with <1.0.0 . However, because it’s also on a 3.x.x branch, it thinks the next version to release should be >=3.0.0.

By default, the tag format that semantic-release looks for is v${version}. So it will recognise "v3.1.2" but not “3.1.2". Without the prefix of “v“,  semantic-release could not find the existing versions. The solution was to configure the tagFormat in the ./package.json as below

{
  ...
  "release": {
    "tagFormat": "${version}",
    ...
  }
}

With this fix, it could recognise the existing git tags. If you wanted to keep the default tag format with the “v” prefix, you could manually create git tags for the latest version on each major branch, using the default tag format.

3.x.x and 3.x Unique Branch Error

While I was troubleshooting the previous error, I was advised by a workmate to use the format “3.x” instead of “3.x.x” for version numbers. He had encountered problems when using the 3.x.x pattern in another project. So I created a 3.x major branch based on 3.x.x. However, that brought problems of it’s own:

EMAINTENANCEBRANCHES The maintenance branches are invalid in the `branches` configuration. Each maintenance branch in the branches configuration (https://github.com/semantic-release/semantic-release/blob/master/docs/usage/configuration.md#branches) must have a unique range property. Your configuration for the problematic branches is [ { channel: '1.x.x', type: 'maintenance', name: '1.x.x', tags: [], range: '>=1.0.0 <1.0.0', accept: [], mergeRange: '>=1.0.0 <2.0.0' }, { channel: '2.x.x', type: 'maintenance', name: '2.x.x', tags: [], range: '>=2.0.0 <1.0.0', accept: [], mergeRange: '>=2.0.0 <3.0.0' }, { channel: '3.x', type: 'maintenance', name: '3.x', tags: [], range: '>=3.0.0 <1.0.0', accept: [], mergeRange: '>=3.0.0 <4.0.0' }, { channel: '3.x.x', type: 'maintenance', name: '3.x.x', tags: [], range: '>=3.0.0 <1.0.0', accept: [], mergeRange: '>=3.0.0 <4.0.0' } ]. SemanticReleaseError: The maintenance branches are invalid in the `branches` configuration. 

AggregateError:   SemanticReleaseError: The maintenance branches are invalid in the `branches` configuration.

The key phrase in this message is “must have a unique range property”. In my project 3.x.x and 3.x are identical, which generates the version range as >=3.0.0 <1.0.0 , That resulted in the EMAINTENANCEBRANCHES error. It is fine to keep either the 3.x.x or the 3.x pattern, but you cannot have both at the same time.

NPM Error

Next, we got the following:

[5:15:13 AM] [semantic-release] > ✖ Failed step "addChannel" of plugin "@semantic-release/npm" [5:15:13 AM] [semantic-release] > ✖ An error occurred while running semantic-release: Error: Command failed with exit code 1: npm dist-tag add @example/ng-library@3.3.0 release-3.x.x --userconfig /tmp/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/.npmrc --registryhttps://private-npm-registry-url/repository/npm-group/ npm ERR! code E400 npm ERR! 400 Bad Request - PUThttps://private-npm-registry-url/repository/npm-group/-/package/@example%2fng-library/dist-tags/release-3.x.x

We use Nexus Repository Manager for our private npm registry. We use the npm-all endpoint for a normal npm install, but when we need to publish the package, the registry URL needs to be private, like npm-internal. For this to work, we needed to configure the registry setting, which takes precedence over the configuration in .npmrc. This setting was added to ./package.json:

{
  ...
  "publishConfig": {
    "registry" : "https://private-npm-registry-url/repository/npm-internal/"
  },
}

Next Major Version

Our last hurdle was this:

SemanticReleaseError: The release `3.4.0` on branch `3.x.x` cannot be published as it is out of range. code: 'EINVALIDNEXTVERSION', details: 'Based on the releases published on other branches, only versions within the range `>=3.3.0 <2.1.3` can be published from branch `3.x.x`.\n' +

The master is on version 2, and when I try to publish a new package from 3.x.x, it does not allow me to publish it as package 3.4.0, because the major version is greater than 2. To fix this, there were two options:

1. Merge 3.x.x to master to allow us to publish 3.4.0
2. Use a branch called next-major to release a package with a major version greater than that of master 

For this project, I chose the second option: create a new branch named as next-major from 3.x.x. After a re-triggered the build, semantic-release published 3.4.0 in our private registry without drama.

One last thing

It’s awesome that @semantic-release/release-notes-generator can generate text to update the changelog file. But the default URL it uses for comparing tags won’t work with Azure repos. For example, in our case it would use the URL /compare/3.3.0...3.4.0. Instead, we configured it to use the form /branchCompare?baseVersion=GT3.3.0&targetVersion=GT3.4.0&_a=commits.

Conclusion

In this post I’ve shared the story of our journey from handling releases manually through to automating them using semantic-release. I’ve also described several of the potholes we encountered along the way. I hope that if choose to embark on a similar journey, this post will help you have a smoother ride!