Fine Code Coverage runs tests on its own
In the sample project for my previous blog post, I noticed that every time I ran my test, its code was executed twice. At first, I thought that I had a bug in my code, but quickly dismissed this option. There was simply no way that the following simple code would insert two records in the database instead of just one:
var newPerson = new Person("John", "Doe");
using (var context = CreateDbContext())
{
context.Add(newPerson);
context.SaveChanges();
}
After further investigation, I finally attributed the strange behavior to the Fine Code Coverage Visual Studio extension. It repeated every test run I did to generate a code coverage report and show coverage information in the code editor gutter.
Ignoring the performance aspect of this behavior for large test suites, this behavior shouldn't be mush of an issue in most cases:
- I only noticed it because my test code had side effects (inserting records into a database table) and I was checking those independently of tests (querying the table with a separate tool). That's not a typical use case for tests.
- The coverage could also be wrong if a test succeeded during the initial test run but failed during the second run initiated by the extension. However, you'd want to fix such flaky tests anyway.
Nevertheless, I decided to check if this behavior is documented. It is, and there is also a way to avoid it. The latest version of the extension can use MS Code Coverage instead of Coverlet to generate coverage information. Since the feature is still in beta, it is disabled by default and must be manually enabled in the extension options by setting RunMsCodeCoverage to Yes.
After doing that, I could confirm that my tests were not being run twice anymore. I'll keep option on unless I encounter any issues.
Despite the described incident, I can still recommend Fine Code Coverage as the best code coverage tool for Visual Studio. I already wrote about this extension in my blog post about code coverage tools for .NET and it only seems to be getting better. Just be careful if it matters to you how many times the tests run. And consider enabling the MS Code Coverage option, although the feature is still in beta.
