Recently, Microsoft introduced the new version of its test framework, MS-Test 2. With this new version, they introduced a new feature that I was waiting for a long time: parametrized tests (yes, NUnit and XUnit have had this for a long time, I know).

And what are parametrized tests? Let me show you with an example. Let’s say we have this routine to return Fibonacci numbers (source: https://www.dotnetperls.com/fibonacci)

 

public static int Fibonacci(int n)
{
    int a = 0;
    int b = 1;
    // In N steps compute Fibonacci sequence iteratively.
    for (int i = 0; i < n; i++)
    {
        int temp = a;
        a = b;
        b = temp + b;
    }
    return a;
}

And we want to test it. We would like to test it with the numbers 0, 1, 2 and 80 (the first two are special cases, the third is a normal case and 80 is a large number to be sure that the routine works with large numbers). We should create a test like this:

[TestMethod]
public void Given0FibonacciReturns0()
{
   var fib = new Fib();
    var actual = fib.Fibonacci(0);
    Assert.AreEqual(0,actual);
}

This is not a bad test, but we must copy and paste to test the other results. You may argue that we could create a test like this one:

[TestMethod]
public void GivenDataFibonacciReturnsResultsOk()
{
    var numbers = new[] { 0, 1, 2, 80 };
    var results = new[] { 0L, 1L, 1L, 23416728348467685L };
    var fib = new Fib();
    for (int i = 0; i < numbers.Length; i++)
    {
        var actual = fib.Fibonacci(numbers[i]);
        Assert.AreEqual(results[i], actual);
    }
}

But this has some problems:

  • If a test fails, it’s difficult to know which number failed
  • If one number fails, the next ones are not tested
  • You don’t have a clear view of what is being tested

MS-Test has had for a long time Data Driven tests (https://msdn.microsoft.com/en-us/library/ms182527.aspx), but this is very cumbersome. You must create a data file, assign it to the test and run the test using the TestContext. It’s too much work for just four tests, no?

Then it comes MS-Test 2. With it, you can create a DataTestMethod, with DataRows for each test. Let’s see how do you create a test with this new feature.

Creating Parametrized tests with MS-Test 2

In Visual Studio, create a new Console Project. In this project, create a new class and name it Fib.cs. Add this code to the class:

 public class Fib
 {
     public int Fibonacci(int n)
     {
         int a = 0;
         int b = 1;
         // In N steps compute Fibonacci sequence iteratively.
         for (int i = 0; i < n; i++)
         {
             int temp = a;
             a = b;
             b = temp + b;
         }
         return a;
     }
 }

Then, in the solution, add a new Class Library project. Right click the References node in the Solution Explorer and add a reference to the console project. Then right click in the References node again and select “Manage NuGet packages”. Add the packages MsTest.TestAdapter and MsTest.TestFramework.

image

With that, you have a test project with MS-Test 2. If you are using Visual Studio 2017, the Test Project template already includes these two packages, but you must update them to the latest version, as the parametrized tests didn’t run well with the default packages.

Then, we can create our test:

[TestClass]
public class FibonacciTests
{
    [DataRow(0, 0)]
    [DataRow(1, 1)]
    [DataRow(2, 1)]
    [DataRow(80, 23416728348467685)]
    [DataTestMethod]
    public void GivenDataFibonacciReturnsResultsOk(int number, Int64 result)
    {
        var fib = new Fib();
        var actual = fib.Fibonacci(number);
        Assert.AreEqual(result, actual);
    }
}

The test is very similar to the ones we are used to create, it just has some differences:

 

  • Instead of the TestMethod attribute, it is decorated with the DataTestMethod attribute
  • The method receives two parameters
  • Each test has a DataRow attribute associated to it.

 

 

If we run this test, we get these results:

image

As you can see, we have three tests that passed and one that failed. We didn’t take in account in our routine that the results could be very large and overflow. So, we must change the routine to take this in account:

public Int64 Fibonacci(int n)
{
    Int64 a = 0;
    Int64 b = 1;
    // In N steps compute Fibonacci sequence iteratively.
    for (int i = 0; i < n; i++)
    {
        Int64 temp = a;
        a = b;
        b = temp + b;
    }
    return a;
}

Now, when you run the tests, you get this:

image

All tests are passing, and we can have a clear view of which tests were run, without the need of extra files or any other tricks. Cool, no? This was a very welcome addition to MS-Test and can improve a lot our testing.

The source code for this article is in https://github.com/bsonnino/Fibonacci

5 Thoughts on “Parametrized tests with MS-Test

  1. Sameer Arora on February 8, 2018 at 12:53 am said:

    Thanks for sharing more details. However, is there a way to actually reflect the input parameters in the test name that appears in the results. All we see is “Data Row 0”, “Data row 1” suffix, which is quite unhelpful to quickly tell for which parameters the test failed, and how to track such failures across daily run.

    This sounds like a basic usability thing that should be supported AND documented if it supported or not.

    • bsonnino on February 9, 2018 at 1:16 pm said:

      Actually, the tests results show the parameters passed in VS 2017. Take a look at my results:

      Test Name: GivenDataFibonacciReturnsResultsOk
      Test FullName: Fibonacci.Tests.FibonacciTests.GivenDataFibonacciReturnsResultsOk
      Test Source: D:\Documentos\Artigos\Artigos\CSharp\ParametrizedTests\Fibonacci\Fibonacci.Tests\FibonacciTests.cs : line 15
      Test Outcome: Passed
      Test Duration: 0:00:00.0407585

      Result1 Name: GivenDataFibonacciReturnsResultsOk (0,0)
      Result1 Outcome: Passed
      Result1 Duration: 0:00:00.0407386
      Result1 StackTrace:
      Result1 Message:
      Result1 StandardOutput:
      Result1 StandardError:

      Result2 Name: GivenDataFibonacciReturnsResultsOk (1,1)
      Result2 Outcome: Passed
      Result2 Duration: 0:00:00.0000164
      Result2 StackTrace:
      Result2 Message:
      Result2 StandardOutput:
      Result2 StandardError:

      Result3 Name: GivenDataFibonacciReturnsResultsOk (2,1)
      Result3 Outcome: Passed
      Result3 Duration: 0:00:00.0000021
      Result3 StackTrace:
      Result3 Message:
      Result3 StandardOutput:
      Result3 StandardError:

      Result4 Name: GivenDataFibonacciReturnsResultsOk (80,23416728348467685)
      Result4 Outcome: Passed
      Result4 Duration: 0:00:00.0000014
      Result4 StackTrace:
      Result4 Message:
      Result4 StandardOutput:
      Result4 StandardError:

      If you want to know more if the test fails, you can use the assert message, like in

      Assert.AreEqual(result, actual,0.1,$”Test for {number}”);

  2. Javier on April 20, 2018 at 1:27 pm said:

    Sadly, that is not the case if you are using arrays (other collections?), you just get MyTest(A[], B[])

  3. Javier on April 20, 2018 at 1:30 pm said:

    A simple solution is to add a “fake” key tag (string) to each row BTW …

  4. Monger39 on July 16, 2019 at 9:07 am said:

    It would be great to also use the ‘ExpectedException’ for parameterized tests,
    such that for specifically indicated ‘DataRow’s we can check if the expected
    exception is thrown…

Leave a Reply

Your email address will not be published. Required fields are marked *

Post Navigation