Data-Driven or Parameterized Tests in Windows Store apps
Since I usually write my unit tests in NUnit, I got into the habit of using parameterized tests when testing methods for which I need to check the result for many different input values. Instead of having to write many tests for different sets of input values, all of them containing the same core code or calling the same inner method, I can write only a single test, specifying the input values and the expected result:
[TestCase(1, 1, Result = 2)]
[TestCase(1, 3, Result = 4)]
[TestCase(2, 4, Result = 6)]
[TestCase(6, 1, Result = 7)]
public int TestAdding(int a, int b)
{
var calculator = new Calculator();
return calculator.Add(a, b);
}
When you run such a test in ReSharper or Visual Studio test runner with NUnit Test Adapter you even get all of the test cases individually listed to quickly see which one of them failed:
Unfortunately using NUnit is not an option with Windows Store apps. Only MSTest is supported, therefore I had to learn what this testing framework had to offer in such cases. The answer are data-driven unit tests. Similarly to NUnit they allow specifying different test cases for a single test, but the values need to be stored in an external data source instead of with attributes. Having input values for tests in an external database doesn't make much sense to me, so the remaining options are CSV or XML files included in the test project. Still the test code is less obvious, since a lookup in another file is required to see the actual values:
[TestMethod]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV",
"TestCases.csv",
"TestCases#csv",
DataAccessMethod.Sequential)]
[DeploymentItem("TestCases.csv")]
public void TestAdding()
{
var a = Convert.ToInt32(TestContext.DataRow["a"]);
var b = Convert.ToInt32(TestContext.DataRow["b"]);
var result = Convert.ToInt32(TestContext.DataRow["result"]);
var calculator = new Calculator();
Assert.AreEqual(result, calculator.Add(a, b));
}
The test runners also don't handle these tests in the same way. They list only a single test and display individual test outcomes in the output:
Not all that surprising, it turned out that MSTest for Windows Store apps doesn't support such data driven tests. Instead, since Visual Studio 2012 Update 1, an approach very similar to parameterized NUnit tests is available:
[DataTestMethod]
[DataRow(1, 1, 2)]
[DataRow(1, 3, 4)]
[DataRow(2, 4, 6)]
[DataRow(6, 1, 7)]
public void TestAdding(int a, int b, int result)
{
var calculator = new Calculator();
Assert.AreEqual(result, calculator.Add(a, b));
}
For obvious reasons I like this syntax much better. Unfortunately it is only available for Windows Store app and Windows Phone 8 test projects, but not on other platforms. If you're like me and would like to have it supported everywhere, consider voting for the suggestion at User Voice.
Test runners handle these tests similarly to other MSTest data-driven tests, i.e. only a single test is shown in the list. I'd like to give another word of caution at this point. ReSharper as of version 8.0.2 has a bug which causes some of the test cases to be skipped, thus not detecting a failed test case. You either need to run the tests in debug mode or use Visual Studio test runner instead. I've reported the bug and the fix seems to be coming with the next release.
