Damir's Corner.

Type issues with data driven tests in MSTest

Data-driven tests are great for repeating the same unit tests with many different inputs. Here is an example of a test for a method that can convert temperatures between different temperature units:

[DataTestMethod]
[DataRow(Celsius, Celsius, 27, 27)]
[DataRow(Celsius, Fahrenheit, 27, 80.6)]
[DataRow(Celsius, Kelvin, 27, 300.15)]
[DataRow(Fahrenheit, Celsius, 80, 26.67)]
[DataRow(Fahrenheit, Fahrenheit, 80, 80)]
[DataRow(Fahrenheit, Kelvin, 80, 299.82)]
[DataRow(Kelvin, Celsius, 300, 26.85)]
[DataRow(Kelvin, Fahrenheit, 300, 80.33)]
[DataRow(Kelvin, Kelvin, 300, 300)]
public void ConvertsCorrectly(
    TemperatureUnit from,
    TemperatureUnit to,
    double value,
    double expected)
{
    var result = TemperatureConverter.Convert(value, from, to);

    result.Should().BeApproximately(expected, 0.005);
}

To my surprise, a similar test from a project I worked on failed on multiple test cases with the following error message:

Test method DataDrivenTestTyping.MsTest.TemperatureConverterTests.ConvertsCorrectly threw exception: System.ArgumentException: Object of type 'System.Decimal' cannot be converted to type 'System.Double'.

The error was limited to tests with non-integer input values, but the error still did not make much sense because both the literal values in the attributes and the data parameters were of type double.

I could not find a way to get the test framework to pass all numeric values to test parameters with the correct type. Since the DataRowAttribute parameters are of type object, I found a solution that allowed me to explicitly convert the values to the correct type:

[DataTestMethod]
[DataRow(Celsius, Celsius, 27, 27)]
[DataRow(Celsius, Fahrenheit, 27, 80.6)]
[DataRow(Celsius, Kelvin, 27, 300.15)]
[DataRow(Fahrenheit, Celsius, 80, 26.67)]
[DataRow(Fahrenheit, Fahrenheit, 80, 80)]
[DataRow(Fahrenheit, Kelvin, 80, 299.82)]
[DataRow(Kelvin, Celsius, 300, 26.85)]
[DataRow(Kelvin, Fahrenheit, 300, 80.33)]
[DataRow(Kelvin, Kelvin, 300, 300)]
public void ConvertsCorrectly(
    TemperatureUnit from,
    TemperatureUnit to,
    object value,
    object expected)
{
    ConvertsCorrectly(from, to, Convert.ToDouble(value), Convert.ToDouble(expected));
}

public void ConvertsCorrectly(
    TemperatureUnit from,
    TemperatureUnit to,
    double value,
    double expected)
{
    var result = TemperatureConverter.Convert(value, from, to);

    result.Should().BeApproximately(expected, 0.005);
}

I was not happy, but at least it worked.

Then I decided to try upgrading MSTest to the latest version. And it helped: the problem was gone. All test cases worked, even with the original code before my workaround.

With bisection, I determined that the problem was fixed in version 2.2.5. This was the change that fixed the problem.

I have posted a sample project with this problem to my GitHub repository. Various commits show:

• the original problem with MSTest 2.2.4,
• the workaround for the problem with MSTest 2.2.4, and
• the original code that works with MSTest 2.2.5.

I could have solved my problem much sooner if I had updated the library to the latest version before I started doubting the test code that should have worked and looking for workarounds. I'll try to remember that the next time something similar happens to me.