.NET test and coverage reports in GitHub Actions
The template I'm using for creating a new GitHub Actions workflow for .NET projects doesn't create any test and coverage reports, so that's usually the first thing I add to it.
I've documented my approach in the past across two articles I've written for the DotNetCurry Magazine. But since the relevant instructions are interleaved with other topics and some details have changed since those articles were published, I decided to write this blog post with up-to-date information and focus only on this part of the workflow.
I use the template from TimHeuer.GitHubActions.Templates NuGet package as a starting point for my .NET GitHub Actions workflows. First, I make sure that I have the latest version installed:
dotnet new install TimHeuer.GitHubActions.Templates
Then I create a new workflow by running the following command in the repository root folder where the solution file is placed:
dotnet new workflow
To get the data needed for creating a test report, I add the --logger argument to the existing dotnet test command in the workflow:
- name: Test
run: dotnet test --logger trx
The trx format is required by the Test Reporter GitHub action which I'm using to add a test report as a check run (notice the path to the generated .trx file(s) to be included in the report):
- name: Test Report
uses: dorny/test-reporter@v1
if: success() || failure()
with:
name: Tests
path: WebApi.Tests/TestResults/*.trx
reporter: dotnet-trx
The action requires write permissions for creating a check run. You can assign them to the workflow by adding a permissions block to your workflow file. Once you do that, you need to specify all required permissions including the read-only ones that are assigned by default, so don't forget to include those:
permissions:
contents: read
actions: read
checks: write
With this in place, you'll get a nice HTML test report added as a check run to to your workflow run:
To get information about code coverage from the test run, I add the --collect argument to the existing dotnet test command in the workflow:
- name: Test
run: dotnet test --logger trx --collect:"XPlat Code Coverage"
I don't know of a GitHub action which can use this output to add a coverage report as a check run on its own. So I'm using the ReportGenerator .NET local tool first, to create a Markdown report. To make it available in the workflow run, it must be added to the local tool manifest file. If you don't have one yet in your repository, you can create it by running the following command in the repository root folder:
dotnet new tool-manifest
Now, the local tool can be installed:
dotnet tool install dotnet-reportgenerator-globaltool
In the workflow file, the tools must be restored first:
- name: Restore local tools
run: dotnet tool restore
In subsequent steps, the Report Generator can then be used to generate a Markdown report from the generated files with coverage information. Since the dotnet test run puts the same coverage file in two subfolders of TestResults with different depth, I use a file pattern that only grabs one of the two:
- name: Generate coverage report
run: dotnet reportgenerator -reports:WebApi.Tests/TestResults/*/coverage.cobertura.xml -targetdir:./coverage -reporttypes:MarkdownSummary
To add the generated Markdown file to the workflow run as a check run, I then use the checks-action GitHub action:
- name: Create check with coverage report
uses: LouisBrunner/checks-action@v2.0.0
if: always()
with:
token: ${{ secrets.GITHUB_TOKEN }}
name: Code coverage
conclusion: ${{ job.status }}
output: '{"summary":"Code coverage"}'
output_text_description_file: coverage/Summary.md
This action also needs write permissions for creating a check run, but I've assigned those already for the test report check run, so no further changes to the workflow file are necessary. Every workflow run will now also include a coverage report check run:
Unfortunately, the check runs might show up for the wrong workflow run if you have more than one per commit. This is because of a limitation in GitHub Actions that only allows check runs to be attached to a commit, not to a workflow run.
You can find the full workflow file for a sample project in my GitHub repository. Individual commits show how I first created the file from the template, then added the test report and finally added the coverage report.
I find a lot of value in having test and coverage reports available for each build. Unfortunately they aren't a part of the basic .NET workflow file for GitHub Actions created from the template. However, there are tools available which you can use to add them yourself once you know how.
