Tim Klingeleers

Tim Klingeleers

I am a mobile and web developer for APG in the Netherlands. I’m also the owner and founder of Klingeleers VOF.

In my free time I like to run, and work with my dog in search and rescue at Searchdog@work.

Read more

Generating a markdown changelog from git

5 minute read

When you make use commit formats like conventional commits, this standardisation can get you some useful benefits. In this scenario, I wanted to create a changelog in markdown format, that listed all of the bugfixes and features that were created per release and link to a JIRA ticket that is associated with it. Let’s take a look how this was achieved.

Update

I recently found an open source project called git-cliff that does something similar as written in this article, but in a better and more advanced way.

Our requirements

We wanted to generate a markdown file that included all features, bugfixes and chores for every release that we have in our git repository. These releases are marked with a tag. We are using JIRA for issue management and Azure DevOps for our git repository, and we want to link every commit to a JIRA ticket. The changelog should include a link to the JIRA ticket per commit, and a link that displays all code changes for a release to Azure DevOps per tag.

Using git and tags

To start the generation process, a list of commits is needed in groups per tag. In this case it’s a fairly easy one, since we don’t use tags for anything else but to mark a release. Otherwise we could filter it by looking for a specific format (e.g. v#.#.#).

GIT_TAGS=$(git tag -l --sort=-version:refname)

The list of tags that we get here is prepended with a HEAD “tag”, since we also want to list all of the unreleased commits.

TAGS=("HEAD")
TAGS+=($GIT_TAGS)

After we have all the tags, we can get a list of commit hashes between these tags, by making use of the pretty-format option. The commit hashes are the most important for further processing.

if [[ -z $PREVIOUS_TAG ]]; then
    COMMITS=$(git log $LATEST_TAG --pretty=format:"%H")
else
    COMMITS=$(git log $PREVIOUS_TAG..$LATEST_TAG --pretty=format:"%H")
fi

Now that we have this list of commit hashes, we can loop over them and get specific details on the commit. Since we are using conventional commits, every commit title is prepended with a specific keyword depending on the type of change. The most important keywords that we use in the script are:

  • feat: when a new feature is created
  • fix: when a bugfix is created
  • chore: for something else

All of the other keywords from the conventional commit format are discarded in our changelog.

# Get the subject of the current commit
SUBJECT=$(git log -1 ${COMMIT} --pretty=format:"%s")

# Is it marked as a feature commit?
FEATURE=$(grep -Eo "feat:" <<<"$SUBJECT")
# Is it marked as a bugfix commit?
FIX=$(grep -Eo "fix:" <<<"$SUBJECT")
# Is it marked as a chore commit?
CHORE=$(grep -Eo "chore:" <<<"$SUBJECT")

Once we know if it is one of the previous types of commits, we can start dissecting the commit body and figure out if a JIRA ticket is referenced. In our team the convention is that the last JIRA ticket that is referenced in the commit body, is the main ticket for the commit.

# Get the body of the commit
BODY=$(git log -1 ${COMMIT} --pretty=format:"%b")
# Does the body contain a link to a JIRA-number
JIRA_ID=$(grep -Eo "JIRA-[[:digit:]]+" <<<$BODY)
# Get last JIRA-number of the body (body might reference others)
JIRA_ID=$(echo "$JIRA_ID" | tail -1)

If the commit body does not include a JIRA ticket number, we skip it from the changelog. Otherwise we add it to the correct list for the type of change.

The rest of the script is specific styling for our markdown file, that can be manipulated to your liking. In the example code, we start with a title and a latest update date. Then show a list of all unreleased changes, followed by all changes sorted by tag from new to old. The features are listed first, then the bugfixes and then the chores. Every tag is also ended by a link to get the same list in Azure DevOps.

Making use of the script in CI/CD

We can use the script in our CI/CD pipeline and get the markdown file. To make it even more useful for the product stakeholders, this markdown can be automatically converted to PDF for further distribution.

Full script

You’ll have to replace the <…> in the settings section with your own.

#!/bin/bash
IFS=$'\n'

# Settings
PRODUCT_NAME="My app"
REPOSITORY_URL="https://<jira-server>/browse"
AZURE_DEVOPS_GIT_URL="https://<tenant>.visualstudio.com/<project>/_git/<repository>"
INCLUDE_FEATURES=1
INCLUDE_FIXES=1
INCLUDE_CHORES=1

# Get a list of all tags in reverse order
GIT_TAGS=$(git tag -l --sort=-version:refname)

DATE_NOW=$(date '+%d-%m-%Y')
# Add title
MARKDOWN="# Changelog ${PRODUCT_NAME}\n<sup>Last updated: $DATE_NOW</sup>\n\n"

# Make the tags an array and include HEAD as the first (so we can include unreleased changes)
TAGS=("HEAD")
TAGS+=($GIT_TAGS)

for TAG_INDEX in "${!TAGS[@]}"; do
	FEATURES=()
	FIXES=()
	CHORES=()

	LATEST_TAG=${TAGS[TAG_INDEX]}
	PREVIOUS_TAG=${TAGS[TAG_INDEX + 1]}
	TAG_DATE=$(git for-each-ref --format="%(taggerdate:format:%d-%m-%Y)" "refs/tags/${LATEST_TAG}")

	# Get a log of commits that occured between two tags
	# We only get the commit hash so we don't have to deal with a bunch of ugly parsing
	# See Pretty format placeholders at https://git-scm.com/docs/pretty-formats
	if [[ -z $PREVIOUS_TAG ]]; then
		COMMITS=$(git log $LATEST_TAG --pretty=format:"%H")
	else
		COMMITS=$(git log $PREVIOUS_TAG..$LATEST_TAG --pretty=format:"%H")
	fi

	# Loop over each commit and look for feature, bugfix or chore commits
	for COMMIT in $COMMITS; do
		# Get the subject of the current commit
		SUBJECT=$(git log -1 ${COMMIT} --pretty=format:"%s")

		# Is it marked as a feature commit?
		FEATURE=$(grep -Eo "feat:" <<<"$SUBJECT")
		# Is it marked as a bugfix commit?
		FIX=$(grep -Eo "fix:" <<<"$SUBJECT")
		# Is it marked as a chore commit?
		CHORE=$(grep -Eo "chore:" <<<"$SUBJECT")

		# Get the body of the commit
		BODY=$(git log -1 ${COMMIT} --pretty=format:"%b")
		# Does the body contain a link to a JIRA-number
	    JIRA_ID=$(grep -Eo "JIRA-[[:digit:]]+" <<<$BODY)
		# Get last JIRA-number of the body (body might reference others)
		JIRA_ID=$(echo "$JIRA_ID" | tail -1)
		
		# Only include in list if commit contains a reference to a JIRA-number
		if [[ $JIRA_ID ]]; then
			if [[ $FEATURE ]] && [[ $INCLUDE_FEATURES = 1 ]]; then
				search_str="feat:"
				subject="${SUBJECT#*$search_str}"
				FEATURES+=("- [$JIRA_ID]($REPOSITORY_URL/$JIRA_ID):${subject}")
			elif [[ $FIX ]] && [[ $INCLUDE_FIXES = 1 ]]; then
				search_str="fix:"
				subject=${SUBJECT#*$search_str}
				FIXES+=("- [$JIRA_ID]($REPOSITORY_URL/$JIRA_ID):${subject}")
			elif [[ $CHORE ]] && [[ $INCLUDE_CHORES = 1 ]]; then
				search_str="chore:"
				subject=${SUBJECT#*$search_str}
				CHORES+=("- [$JIRA_ID]($REPOSITORY_URL/$JIRA_ID):${subject}")
			fi
		fi
	done

	# Continue to next release if no commits are available since the previous release
	if [[ -z $COMMITS ]]; then
		continue
	fi

	if [[ $LATEST_TAG = "HEAD" ]]; then
		MARKDOWN+="## Unreleased\n\n"
	else
		MARKDOWN+="## Release $LATEST_TAG ($TAG_DATE)\n\n"
	fi

	# List features
	if [[ $FEATURES ]]; then
		FEATURES=($(for l in ${FEATURES[@]}; do echo $l; done | sort -u))
		MARKDOWN+="### ✨ Features\n\n"
		for FEAT in "${FEATURES[@]}"; do
			MARKDOWN+="$FEAT\n"
		done
		MARKDOWN+="\n"
	fi

	# List bugfixes
	if [[ $FIXES ]]; then
		FIXES=($(for l in ${FIXES[@]}; do echo $l; done | sort -u))
		MARKDOWN+="### 🐛 Bugfixes\n\n"
		for FIX in "${FIXES[@]}"; do
			MARKDOWN+="$FIX\n"
		done
		MARKDOWN+="\n"
	fi

	# List chores
	if [[ $CHORES ]]; then
		CHORES=($(for l in ${CHORES[@]}; do echo $l; done | sort -u))
		MARKDOWN+="### 🧹 Chores\n\n"
		for CHORE in "${CHORES[@]}"; do
			MARKDOWN+="$CHORE\n"
		done
		MARKDOWN+="\n"
	fi

	# Append full changelog
	if [[ $LATEST_TAG = "HEAD" ]]; then
		MARKDOWN+="📖 [Full changelog](${AZURE_DEVOPS_GIT_URL}/branchCompare?baseVersion=GT${PREVIOUS_TAG}&targetVersion=GBdevelop)\n\n"
	elif [[ -z $PREVIOUS_TAG ]]; then
		# First release, no way to compare
		echo ""
	else
		MARKDOWN+="📖 [Full changelog](${AZURE_DEVOPS_GIT_URL}/branchCompare?baseVersion=GT${PREVIOUS_TAG}&targetVersion=GT${LATEST_TAG})\n\n"
	fi
	
done

# Save our markdown to a file
printf "%b" "$MARKDOWN" > CHANGELOG.md

Leave a comment

You may also enjoy

Qwik - an HTML first web approach

3 minute read

Meet Qwik, I had never heard of it before, but bumped into it while browsing the web. It instantly sparked my interest by stating that it can deliver an instantly loaded web application, with an HTML-first approach. Another reason to loo...

App wrapping a website with Xamarin

5 minute read

Clients often want their website available in the app store, mostly just for app store presence. I got asked to do app wrapping at work a while ago, because the client purchased a tool that did not have a native app available, only a mob...

Android 64k method limit in Xamarin

2 minute read

Last week, the nightly build of my Android project failed all of a sudden. It was apparently related to the 64k method limit than Android poses for DEX-files. I wasn’t aware how much impact the NuGet packages have on that method count. I...

A look under the hood of Xamarin Forms

4 minute read

Have you always wondered how Xamarin Forms does it’s “magic”? Knowing how things work under the hood is super valuable, especially when things go wrong. This enables you to detect the issue yourself, without having to rely on the knowled...