tl;dr from now on, we release generator in an automated way. We roll-out this setup to the rest when we see it is needed.
Repetitive tasks are tedious. If what you do manually can be automated, then what are you waiting for!
But these tasks take only a couple of minutes from time to time, gimme a break
A couple of minutes here, a couple of minutes there and all of a sudden you do not have time on more important things, on innovation. Automation makes it easier to scale and eliminates errors. Distractions consume time and make you less productive.
We kick ass at AsyncAPI Initiative at the moment. We started to improve our tooling regularly. We are now periodically sharing project status in our newsletter, and host bi-weekly open meetings, but most important is that we just recently updated our roadmap.
Am I just showing off? It sounds like, but that is not my intention. I wish to point out we are productive, and we want to continue this trend and automation helps here a lot. If you have libraries that you want to release regularly and you plan additional ones to come, you need to focus on release automation.
What full automation means
Full automation means that the release process if fully automated with no manual steps. What else did you think?
Your responsibility is just to merge a pull request. The automation handles the rest.
You might say: but I do not want to release on every merge, sometimes I merge changes that are not related to the functionality of the library.
This is a valid point. You need a way to recognize if the given commit should trigger the release and what kind of version, PATCH, or MINOR. The way to do it is to introduce in your project Conventional Commits specification.
Conventional Commits
At AsyncAPI Initiative we use Semantic Versioning. This is why choosing Conventional Commits specification was a natural decision.
Purpose of Conventional Commits is to make commits not only human-readable but also machine-readable. It defines a set of commit prefixes that can be easily parsed and analyzed by tooling.
This is how the version of the library looks like when it follows semantic versioning: MAJOR.MINOR.PATCH
. How does the machine know what release you want to bump because of a given commit? Simplest mapping looks like in the following list:
- Commit message prefix
fix:
indicatesPATCH
release, - Commit message prefix
feat:
indicatesMINOR
release, - Commit message prefix
{ANY_PREFIX}!:
so for examplefeat!:
or evenrefactor!:
indicateMAJOR
release.
It other words, assume your version was 1.0.0, and you made a commit like feat: add a new parameter to test endpoint
. You can have a script that picks up feat:
and triggers release that eventually bumps to version 1.1.0.
Workflow design
At AsyncAPI Initiative where we introduced the release pipeline for the very first time, we had to do the following automatically:
- Tag Git repository with a new version
- Create GitHub Release
- Push new version of the package to NPM
- Push new version of Docker image to Docker Hub
- Bump the version of the package in
package.json
file and commit the change to the repository
This is how the design looks like:
There are two workflows designed here.
The first workflow reacts to changes in the release branch (master
in this case), decides if release should be triggered, and triggers it. The last step of the workflow is a pull request creation with changes in package.json
and package-lock.json
. Why are changes not committed directly to the release branch? Because we use branch protection rules and do not allow direct commits to release branches.
You can extend this workflow with additional steps, like:
- Integration testing
- Deployment
- Notifications
The second workflow is just for handling changes in package.json
. To fulfill branch protection settings, we had to auto-approve the pull request so we can automatically merge it.
GitHub Actions
Even though I have my opinion about GitHub Actions, I still think it is worth investing in it, especially for the release workflows.
We used the GitHub-provided actions and the following awesome actions built by the community:
Release workflow
Release workflow triggers every time there is something new happening in the release branch. In our case, it is the master
branch:
1on:
2 push:
3 branches:
4 - master
GitHub and NPM
For releases to GitHub and NPM, the most convenient solution is to integrate semantic release package and related plugins that support Conventional Commits. You can configure plugins in your package.json
in the order they should be invoked:
1"plugins": [
2 [
3 "@semantic-release/commit-analyzer",
4 {
5 "preset": "conventionalcommits"
6 }
7 ],
8 [
9 "@semantic-release/release-notes-generator",
10 {
11 "preset": "conventionalcommits"
12 }
13 ],
14 "@semantic-release/npm",
15 "@semantic-release/github"
16]
Conveniently, functional automation uses a technical bot rather than a real user. GitHub actions allow you to encrypt the credentials of different systems at the repository level. Referring to them in actions looks as follows:
1- name: Release to NPM and GitHub
2 id: release
3 env:
4 GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
5 NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
6 GIT_AUTHOR_NAME: asyncapi-bot
7 GIT_AUTHOR_EMAIL: info@asyncapi.io
8 GIT_COMMITTER_NAME: asyncapi-bot
9 GIT_COMMITTER_EMAIL: info@asyncapi.io
10 run: npm run release
Aside from automation, the bot also comments on every pull request and issue included in the release notifying subscribed participants that the given topic is part of the release. Isn't it awesome?
Docker
For handling Docker, you can use some community-provided GitHub action that abstracts Docker CLI. I don't think it is needed if you know Docker. You might also want to reuse some commands during local development, like image building, and have them behind an npm script like npm run docker-build
.
1- name: Release to Docker
2 if: steps.initversion.outputs.version != steps.extractver.outputs.version
3 run: |
4 echo ${{secrets.DOCKER_PASSWORD}} | docker login -u ${{secrets.DOCKER_USERNAME}} --password-stdin
5 npm run docker-build
6 docker tag asyncapi/generator:latest asyncapi/generator:${{ steps.extractver.outputs.version }}
7 docker push asyncapi/generator:${{ steps.extractver.outputs.version }}
8 docker push asyncapi/generator:latest
Bump version in package.json
A common practice is to bump the package version in package.json
on every release. You should also push the modified file to the release branch. Be aware though that good practices in the project are:
- Do not commit directly to the release branch. All changes should go through pull requests with proper peer review.
- Branches should have basic protection enabled. There should be simple rules that block pull requests before the merge.
Release workflow, instead of pushing directly to the release branch, should commit to a new branch and create a pull request. Seems like an overhead? No, you can also automate it. Just keep on reading.
1- name: Create Pull Request with updated package files
2 if: steps.initversion.outputs.version != steps.extractver.outputs.version
3 uses: peter-evans/create-pull-request@v2.4.4
4 with:
5 token: ${{ secrets.GH_TOKEN }}
6 commit-message: 'chore(release): ${{ steps.extractver.outputs.version }}'
7 committer: asyncapi-bot <info@asyncapi.io>
8 author: asyncapi-bot <info@asyncapi.io>
9 title: 'chore(release): ${{ steps.extractver.outputs.version }}'
10 body: 'Version bump in package.json and package-lock.json for release [${{ steps.extractver.outputs.version }}](https://github.com/${{github.repository}}/releases/tag/v${{ steps.extractver.outputs.version }})'
11 branch: version-bump/${{ steps.extractver.outputs.version }}
Conditions and sharing outputs
GitHub Actions has two excellent features:
- You can set conditions for specific steps
- You can share the output of one step with another
These features are used in the release workflow to check the version of the package, before and after the GitHub/NPM release step.
To share the output, you must assign an id
to the step and declare a variable and assign any value to it.
1- name: Get version from package.json after release step
2 id: extractver
3 run: |
4 version=$(npm run get-version --silent)
5 echo "version=$version" >> $GITHUB_OUTPUT
You can access the shared value by the id
and a variable name like steps.extractver.outputs.version
. We use it, for example, in the condition that specifies if further steps of the workflow should be triggered or not. If the version in package.json
changed after GitHub and NPM step, this means we should proceed with Docker publishing and pull request creation:
if: steps.initversion.outputs.version != steps.extractver.outputs.version
Full workflow
Below you can find the entire workflow file:
1name: Release
2
3on:
4 push:
5 branches:
6 - master
7
8jobs:
9 release:
10 name: 'Release NPM, GitHub, Docker'
11 runs-on: ubuntu-latest
12 steps:
13 - name: Checkout repo
14 uses: actions/checkout@v2
15 - name: Setup Node.js
16 uses: actions/setup-node@v1
17 with:
18 node-version: 13
19 - name: Install dependencies
20 run: npm ci
21
22 - name: Get version from package.json before release step
23 id: initversion
24 run: npm run get-version --silent
25
26 - name: Set output
27 run: echo "version=$(npm run get-version --silent)" >> $GITHUB_OUTPUT
28
29 - name: Release to NPM and GitHub
30 id: release
31 env:
32 GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
33 NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
34 GIT_AUTHOR_NAME: asyncapi-bot
35 GIT_AUTHOR_EMAIL: info@asyncapi.io
36 GIT_COMMITTER_NAME: asyncapi-bot
37 GIT_COMMITTER_EMAIL: info@asyncapi.io
38 run: npm run release
39 - name: Get version from package.json after release step
40 id: extractver
41 run: echo "::set-output name=version::$(npm run get-version --silent)"
42 - name: Release to Docker
43 if: steps.initversion.outputs.version != steps.extractver.outputs.version
44 run: |
45 echo ${{secrets.DOCKER_PASSWORD}} | docker login -u ${{secrets.DOCKER_USERNAME}} --password-stdin
46 npm run docker-build
47 docker tag asyncapi/generator:latest asyncapi/generator:${{ steps.extractver.outputs.version }}
48 docker push asyncapi/generator:${{ steps.extractver.outputs.version }}
49 docker push asyncapi/generator:latest
50 - name: Create Pull Request with updated package files
51 if: steps.initversion.outputs.version != steps.extractver.outputs.version
52 uses: peter-evans/create-pull-request@v2.4.4
53 with:
54 token: ${{ secrets.GH_TOKEN }}
55 commit-message: 'chore(release): ${{ steps.extractver.outputs.version }}'
56 committer: asyncapi-bot <info@asyncapi.io>
57 author: asyncapi-bot <info@asyncapi.io>
58 title: 'chore(release): ${{ steps.extractver.outputs.version }}'
59 body: 'Version bump in package.json and package-lock.json for release [${{ steps.extractver.outputs.version }}](https://github.com/${{github.repository}}/releases/tag/v${{ steps.extractver.outputs.version }})'
60 branch: version-bump/${{ steps.extractver.outputs.version }}
Automated merging workflow
You may be asking yourself:
Why automated approving and merging is handled in a separate workflow and not as part of release workflow
One reason is that the time between pull request creation and its readiness to be merged is hard to define. Pull requests always include some automated checks, like testing, linting, and others. These are long-running checks. You should not make such an asynchronous step a part of your synchronous release workflow.
Another reason is that you can also extend such an automated merging flow to handle not only pull requests coming from the release-handling bot but also other bots, that, for example, update your dependencies for security reasons.
You should divide automation into separate jobs that enable you to define their dependencies. There is no point to run the automerge job until the autoapprove one ends. GitHub Actions allows you to express this with needs: [autoapprove]
Below you can find the entire workflow file:
1name: Automerge release bump PR
2
3on:
4 pull_request:
5 types:
6 - labeled
7 - unlabeled
8 - synchronize
9 - opened
10 - edited
11 - ready_for_review
12 - reopened
13 - unlocked
14 pull_request_review:
15 types:
16 - submitted
17 check_suite:
18 types:
19 - completed
20 status: {}
21
22jobs:
23 autoapprove:
24 runs-on: ubuntu-latest
25 steps:
26 - name: Autoapproving
27 uses: hmarr/auto-approve-action@v2.0.0
28 if: github.actor == 'asyncapi-bot'
29 with:
30 github-token: '${{ secrets.GITHUB_TOKEN }}'
31
32 automerge:
33 needs: [autoapprove]
34 runs-on: ubuntu-latest
35 steps:
36 - name: Automerging
37 uses: pascalgn/automerge-action@v0.7.5
38 if: github.actor == 'asyncapi-bot'
39 env:
40 GITHUB_TOKEN: '${{ secrets.GH_TOKEN }}'
41 GITHUB_LOGIN: asyncapi-bot
42 MERGE_LABELS: ''
43 MERGE_METHOD: 'squash'
44 MERGE_COMMIT_MESSAGE: 'pull-request-title'
45 MERGE_RETRIES: '10'
46 MERGE_RETRY_SLEEP: '10000'
For a detailed reference, you can look into this pull request that introduces the above-described workflow in the generator.
Conclusions
Automate all the things, don't waste time. Automate releases, even if you are a purist that for years followed a rule of using imperative mood in commit subject and now, after looking on prefixes from Conventional Commits you feel pure disgust.
In the end, you can always use something different, custom approach, like reacting to merges from pull requests with the specific label only. If you have time to reinvent the wheel, go for it.
Cover photo by Franck V. taken from Unsplash.