
21 Jul 2021 Learning to Use Semantic-Release the Hard Way
In this post I want to share a story. The story is about how I added semantic-release to an existing Angular library project. It’s also about some of the errors I encountered along the way, and how I resolved them. There aren’t many such stories about semantic-release out there, so I think others may benefit from hearing my experiences.
Background
We have a library built in Angular and there are two package.json
files in the repo:
./package.json // For the demo project
./projects/ng-library/package.json // For the library project
At the time when our story started, the major branches in the repo were:
master
/2.x.x
(with Angular 8 as peer dependency)1.x.x
(deprecated, with Angular 8 as peer dependency)3.x.x
(with Angular 11 as peer dependency)
Our CI/CD pipeline ran in Jenkins. For each release there were some manual steps, including updating both the package version and the change-log file.
When new code was merged to the major branches, a build kicked in with the following steps:
- Check out the code
- Build the library project and the demo project
- Run unit tests of both projects
- Run e2e tests of the demo project
- Run static analytics for code quality checking
- Run a vulnerability scan
- Publish a new version when the package version of the library is bumped, and tag the new version in Git
Our version tags in Git looked like:
1.2.0
2.3.10
3.2.1
The manual process to bump the version worked well enough in the beginning, but it did not scale well when people from other teams began to make contributions to the library. Every time, we had to explain the process, even though we have a contribution.md
file.
One of my colleagues from another team had been advocating semantic-release, and it had been used in other projects successfully for a while. So when we had to migrate the project to an Azure repo and pipeline for other reasons, we decided to add semantic-release as well.
What is semantic-release?
semantic-release is an NPM tool to automate the workflow to release an NPM package. It follows the semantic versioning specification strictly, including: determining the next version number, generating the release notes, and publishing the package.
semantic-release can be installed in your project with a single command:
$ npm install --save-dev semantic-release
semantic-release supports plugins, although by default it comes with the following four built-in:
@semantic-release/commit-analyzer
@semantic-release/release-notes-generator
@semantic-release/npm
@semantic-release/github
To create or update the changelog file, and to push a release commit and tag to the git repo, we also needed to install two additional plugins:
$ npm install @semantic-release/git @semantic-release/changelog -D
The next step was to create a ./project.json
file:
{
"name": "my-ng-library",
"version": "0.0.0-semantically-released",
"repository": "[git repo URL]",
"private": true,
...
}
Note that you can also use the semantic-release-cli
tool to do this.
Our project file also included the following configuration settings:
{
...
"release": {
"plugins": [
"@semantic-release/commit-analyzer",
[
"@semantic-release/npm",
{
"pkgRoot": "dist/ng-library/"
}
],
"@semantic-release/release-notes-generator",
[
"@semantic-release/changelog",
{
"changelogFile": "CHANGELOG.md"
}
],
[
"@semantic-release/git",
{
"assets": [
"CHANGELOG.md"
]
}
]
]
}
}
We can use npx semantic-release
in the pipeline to run the release job (or yarn semantic-release
if you’re using yarn
). It will analyse the commit message to find keywords like chore, fix, feat, perf to determine if a release is required. Assuming our current version is 2.1.0
, here are some examples:
Commit message | Release type |
---|---|
fix: Fix a typofix(TICKET-123): Fix a typo | Patch release from 2.1.0 to 2.1.1 |
feat(TICKET-123): Add ‘test’ option | Minor release from 2.1.0 to 2.2.0 |
perf(TICKET-123): Upgrade Angular to 12BREAKING CHANGE: Upgrade Angular to 12 | Major release from 2.1.0 to 3.0.0 |
ci(TICKET-123): Add semantic release support docs: Add instruction of semantic release test: Add new testchore: Fix typo in the comment | No release |
By default, semantic-release uses Angular commit message conventions. We installed husky
with @commitlint/cli
to create a git pre-commit hook to check that commit messages would be valid. That said, the commit message convention keywords can be changed in the configuration settings.
Once it determines the release type and decides that a release is required, semantic-release will then:
- Update changelog based on all new commit messages
- Commit the change
- Create a git tag for the new version and push it to the git repo.
- Bump the version accordingly
- Publish the new version to our private NPM registry.
In the new Azure pipeline, we use the npmAuthenticate
task to provide npm credentials to the .npmrc
file in our repo for the scope of the build in the beginning.
Lots of errors!
After making all the settings described above, I began to test. Would semantic-release work properly in the build? I got all kinds of errors when I tried to release a new version in the 3.x.x
branch.
Permission error
First of all was a permission error. Specifically, semantic-release tried to run the command “git push --dry-run --no-verify https://Example@dev.azure.com/Example/my-ng-library/_git/my-ng-library HEAD:3.x.x
” but failed with the error message:
remote: 001f# service=git-receive-pack remote: 0000000000aaTF401027: You need the Git 'GenericContribute' permission to perform this action. Details: identity 'Build\xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', scope 'repository'. remote: TF401027: You need the Git 'GenericContribute' permission to perform this action. Details: identity 'Build\xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', scope 'repository
‘.
And:
SemanticReleaseError: Cannot push to the Git repository.
code: 'EGITNOPERMISSION',
details: '**semantic-release** cannot push the version tag to the branch `3.x.x` on the remote Git repository with URL `https://EADigital@dev.azure.com/EADigital/ea-ng-analytics/_git/ea-ng-analytics`.\n' +
This happened because the Azure build service user requires “contribute” permission to push the code. So I set ‘Contribute’ to ‘Allow’ for the ‘Builder Service’ user in the security settings of all repositories:
That error was then resolved.
Existing Tags Are Not Recognised
Next, semantic-release reported the following:
SemanticReleaseError: The release `1.0.0` on branch `3.x.x` cannot be published as it is out of range. code: 'EINVALIDNEXTVERSION', details: 'Based on the releases published on other branches, only versions within the range `>=3.0.0 <1.0.0` can be published from branch `3.x.x`.\n'
This one was unclear when I first saw it. The range >=3.0.0 <1.0.0
looked odd. I wanted to publish the new version on the 3.x.x
branch and there were existing git tags for versions like 3.1.2
.
After digging into the source code of semantic-release, I found that it did not recognise all of our existing tags. Instead, it thought this was the first time the package was being published in this repo, so it came up with <1.0.0
. However, because it’s also on a 3.x.x
branch, it thinks the next version to release should be >=3.0.0
.
By default, the tag format that semantic-release looks for is v${version}.
So it will recognise "v3.1.2"
but not “3.1.2".
Without the prefix of “v
“, semantic-release could not find the existing versions. The solution was to configure the tagFormat
in the ./package.json
as below
{
...
"release": {
"tagFormat": "${version}",
...
}
}
With this fix, it could recognise the existing git tags. If you wanted to keep the default tag format with the “v
” prefix, you could manually create git tags for the latest version on each major branch, using the default tag format.
3.x.x and 3.x Unique Branch Error
While I was troubleshooting the previous error, I was advised by a workmate to use the format “3.x
” instead of “3.x.x
” for version numbers. He had encountered problems when using the 3.x.x
pattern in another project. So I created a 3.x
major branch based on 3.x.x
. However, that brought problems of it’s own:
EMAINTENANCEBRANCHES The maintenance branches are invalid in the `branches` configuration. Each maintenance branch in the branches configuration (https://github.com/semantic-release/semantic-release/blob/master/docs/usage/configuration.md#branches) must have a unique range property. Your configuration for the problematic branches is [ { channel: '1.x.x', type: 'maintenance', name: '1.x.x', tags: [], range: '>=1.0.0 <1.0.0', accept: [], mergeRange: '>=1.0.0 <2.0.0' }, { channel: '2.x.x', type: 'maintenance', name: '2.x.x', tags: [], range: '>=2.0.0 <1.0.0', accept: [], mergeRange: '>=2.0.0 <3.0.0' }, { channel: '3.x', type: 'maintenance', name: '3.x', tags: [], range: '>=3.0.0 <1.0.0', accept: [], mergeRange: '>=3.0.0 <4.0.0' }, { channel: '3.x.x', type: 'maintenance', name: '3.x.x', tags: [], range: '>=3.0.0 <1.0.0', accept: [], mergeRange: '>=3.0.0 <4.0.0' } ]. SemanticReleaseError: The maintenance branches are invalid in the `branches` configuration.
AggregateError: SemanticReleaseError: The maintenance branches are invalid in the `branches` configuration.
The key phrase in this message is “must have a unique range property”. In my project 3.x.x
and 3.x
are identical, which generates the version range as >=3.0.0 <1.0.0
, That resulted in the EMAINTENANCEBRANCHES
error. It is fine to keep either the 3.x.x
or the 3.x
pattern, but you cannot have both at the same time.
NPM Error
Next, we got the following:
[5:15:13 AM] [semantic-release] > ✖ Failed step "addChannel" of plugin "@semantic-release/npm" [5:15:13 AM] [semantic-release] > ✖ An error occurred while running semantic-release: Error: Command failed with exit code 1: npm dist-tag add @example/ng-library@3.3.0 release-3.x.x --userconfig /tmp/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/.npmrc --registryhttps://private-npm-registry-url/repository/npm-group/ npm ERR! code E400 npm ERR! 400 Bad Request - PUThttps://private-npm-registry-url/repository/npm-group/-/package/@example%2fng-library/dist-tags/release-3.x.x
We use Nexus Repository Manager for our private npm registry. We use the npm-all
endpoint for a normal npm install
, but when we need to publish the package, the registry URL needs to be private, like npm-internal
. For this to work, we needed to configure the registry setting, which takes precedence over the configuration in .npmrc
. This setting was added to ./package.json
:
{
...
"publishConfig": {
"registry" : "https://private-npm-registry-url/repository/npm-internal/"
},
}
Next Major Version
Our last hurdle was this:
SemanticReleaseError: The release `3.4.0` on branch `3.x.x` cannot be published as it is out of range. code: 'EINVALIDNEXTVERSION', details: 'Based on the releases published on other branches, only versions within the range `>=3.3.0 <2.1.3` can be published from branch `3.x.x`.\n' +
The master is on version 2, and when I try to publish a new package from 3.x.x
, it does not allow me to publish it as package 3.4.0
, because the major version is greater than 2. To fix this, there were two options:
- Merge
3.x.x
to master to allow us to publish3.4.0
- Use a branch called
next-major
to release a package with a major version greater than that of master
For this project, I chose the second option: create a new branch named as next-major
from 3.x.x
. After a re-triggered the build, semantic-release published 3.4.0
in our private registry without drama.
One last thing
It’s awesome that @semantic-release/release-notes-generator can generate text to update the changelog file. But the default URL it uses for comparing tags won’t work with Azure repos. For example, in our case it would use the URL /compare/3.3.0...3.4.0
. Instead, we configured it to use the form /branchCompare?baseVersion=GT3.3.0&targetVersion=GT3.4.0&_a=commits
.
Conclusion
In this post I’ve shared the story of our journey from handling releases manually through to automating them using semantic-release. I’ve also described several of the potholes we encountered along the way. I hope that if choose to embark on a similar journey, this post will help you have a smoother ride!
shai
Posted at 17:21h, 14 OctoberThanks!
the problem when working with many developers
things can be broken and when it’s happened you enter to a look to fix the issues ..
shai
Posted at 17:28h, 14 Octoberare you working with Jenkins with Semantic ?
Eric Chen
Posted at 14:31h, 15 OctoberThis post is originally to address semantic release issue in Azure pipeline, and we are also using semantic release in Jenkins recently for different projects.