Here at Headway, we’ve been aiming to better surface metrics from individual projects in a common place for awareness across the team. On the development side, one of these metrics is the amount of code coverage each project’s test suite currently has.
Code coverage gives us project health signals
While code coverage isn’t always a clear indicator of software quality, it can be a helpful pointer to understand a project’s health. High code coverage isn’t a guarantee that the right pieces are tested well, but low code coverage can be a sign that we lack a plan for reliability and maintainability.
Encourage testing practices
Also, we monitor our test coverage to help encourage good testing practices. Not every project needs to have the same percentage of coverage, but watching code coverage metrics can help us to make sure we have a usable and effective test setup for each project. If the configuration and libraries are properly in place, it can be easier to continue writing tests in the future. Without that, it can be a challenge to get started with other project demands at play!
Example of reporting in Slack
For visibility, we report our code coverage to a channel within our team’s Slack. Each update looks something like this:
Let’s walk through how to set this up for a project!
Creating a Slack workflow
First, you’ll need a Slack workflow that can receive events and post messages. I’ll show an overview, but you can follow Slack’s guide for webhooks for more detail.
To start, you’ll want to create a workflow that responds to a webhook. One way to open the workflow builder is to open the settings for a Slack channel, click over to the Integrations tab, and click “Add automation”.
Add two variables to this initial web request step: one for the name of the application (appName) and one for the percentage of code coverage (percent):
Next, add a step to your workflow to post a message.
Select the channel you’d like your messages to appear in and then customize the message text using the variables from the previous step:
When you’re all done, it should look something like this:
Once your workflow is all set, you will need to “Publish” it from Slack’s menu to make it available and ready to use.
Reporting code coverage from GitHub actions
We often use GitHub actions for running our test suite. Other CI tools have their own configuration, but the examples below may still be a helpful starting point.
Depending on your language’s coverage reporting tool, it’s generally straightforward to gather the percentage of code coverage and send that over to Slack.
Here are a few examples steps for adding coverage reporting to your GitHub actions workflow YAML file on merges into main:
Find the final coverage percentage
First, find where your coverage reporter stores the final coverage percentage after test runs. Gather this number and then store it in the output for the given step. We conditionally run this only on merges into main so that we don’t have messages for in-progress PR work that may not reflect the full state of the merged code.
For Coveralls (Elixir), you can gather this from the output of mix coveralls with steps along these lines:
-- CODE line-numbers language-yml --
<!--
- name: Run Test Coverage
if: ${{ github.ref == 'refs/heads/main' }}
run: mix coveralls > coverage.txt
- name: Get Coverage Percent
id: coverage_percent
if: ${{ github.ref == 'refs/heads/main' }}
run: |
percent=`awk '/\\[TOTAL\\]/ { print $2 }' coverage.txt`
echo "value=$percent" >> $GITHUB_OUTPUT
-->
<!--
- name: Run Test Coverage
if: ${{ github.ref == 'refs/heads/main' }}
run: mix coveralls > coverage.txt
- name: Get Coverage Percent
id: coverage_percent
if: ${{ github.ref == 'refs/heads/main' }}
run: |
percent=`awk '/\\[TOTAL\\]/ { print $2 }' coverage.txt`
echo "value=$percent" >> $GITHUB_OUTPUT
-->
For Simplecov (Ruby), the results are stored in the coverage/.last_run.json file:
-- CODE line-numbers language-yml --
<!--
- name: Retrieve coverage percent
if: ${{ github.ref == 'refs/heads/main' }}
id: coverage_percent
run: |
percent=cat coverage/.last_run.json | jq '.result.line'
echo "value=$percent" >> $GITHUB_OUTPUT
-->
<!--
- name: Retrieve coverage percent
if: ${{ github.ref == 'refs/heads/main' }}
id: coverage_percent
run: |
percent=cat coverage/.last_run.json | jq '.result.line'
echo "value=$percent" >> $GITHUB_OUTPUT
-->
Other languages and testing frameworks often provide similar tooling to gather code coverage information.
- For Jest (JavaScript), you can collect coverage results with
jest --collectCoverage. - For an Elixir project that isn’t using Coveralls, mix test can provide coverage information as well.
Sending the final percentage to Slack
After storing the percentage in a variable, send it to Slack. Change the appName variable to whatever name you’d like to show in your Slack message for this project:
-- CODE line-numbers language-yml --
<!--
- name: Send Coverage Results to Slack
uses: slackapi/slack-github-action@v1.23.0
if: ${{ github.ref == 'refs/heads/main' }}
with:
payload: |
{
"appName": "Your application name",
"percent": "${{ steps.coverage_percent.outputs.value }}"
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
-->
<!--
- name: Send Coverage Results to Slack
uses: slackapi/slack-github-action@v1.23.0
if: ${{ github.ref == 'refs/heads/main' }}
with:
payload: |
{
"appName": "Your application name",
"percent": "${{ steps.coverage_percent.outputs.value }}"
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
-->
Lastly, add a SLACK_WEBHOOK_URL secret to your GitHub repository and set it to the webhook URL for the Slack workflow you created earlier.
Merge the PR with these additions into main and watch for the message to arrive in your Slack channel!
