Testing

Why Use Random Values For Unit Testing

On a previous post I introduced a library for generating random values for purposes of unit testing.

I received a few comments and questions on my blogs [^][^][^][^].

Simon says that he’d “generally want every instance of the test I run to be repeatable on *every* run” and he’d “inclined to use the data source functionality in VSTS Unit Tests”. I couldn’t agree more.

Hugo pointed me to Pex. Looks nice. I’ll have to take a deeper look at it.

Other commenters were worried that in the case of a test failing to pass they were unable to reproduce the conditions of the test and thus would be unable to understand why the test had failed. As I see it, all data used in a failed test should be in the test output (at least, the data that made the test fail). Being it predictable data or random data.

I guess that most of the comments “against” my idea were due to the fact that I didn’t explain it well. I’ll try to do a better job now.

I use these random values when the value itself is not important, what’s important is its uniqueness.

Imagine you have  Costumer class where it’s enough to tell that the costumer is the same in any two given instances if the costumer ID is the same. In order to test the comparison code, I don’t need any particular reproducible set of values. I just need them to be different or the same depending on the test.

Imagine I have some code that needs to access the database with the connection string in a specific connection string settings element. I don’t care what the connection string is because I won’t be really accessing the database (this is really easy to accomplish with Isolator).

I hope I’ve explained myself better this time.

Using Random Values For Unit Testing

When writing my unit tests I don’t like to use hard coded fixed values because I either end up using the same values or, because of that, tests may succeed by coincidence.

Over time, I have developed an helper class to generate random values for testing.

namespace PauloMorgado.VisualStudio.TestTools.UnitTesting
{
    public static class RandomGenerator
    {
        public static bool Boolean();
        public static string String();
        public static string String(string prefix);
        public static short Int8();
        public static short Int8(short maxValue);
        public static short Int8(short minValue, short maxValue);
        public static short Int16();
        public static short Int16(short maxValue);
        public static short Int16(short minValue, short maxValue);
        public static int Int32();
        public static int Int32(int maxValue);
        public static int Int32(int minValue, int maxValue);
        public static TEnum Enum<TEnum>();
        public static TEnum EnumFlagsWith<TEnum>(TEnum flagsToAdd);
        public static TEnum EnumFlagsWithout<TEnum>(TEnum flagsToRemove);
        public static TEnum Enum<TEnum>(int maxValue);
        public static TEnum Enum<TEnum>(int minValue, int maxValue);
        public static System.Guid Guid();
    }
}

This is something that I would like to find on mock frameworks (like Typemock Isolator, Rhino.Mocks or MoQ).

It’s still a work in progress, but if you want to try it, it’s on my MSDN Code Gallery: Random Generator For Unit Testing

Testing With Multiple Versions Of Internet Explorer

On a previous post I mentioned IETester.

Jorge Moura mentioned TredoSoft’s MultipleIEs and a list of web browsers.

Typemock Developers Community Site Updated

Typemock has updated its Developers Community Site with new sections.

Besides the forums, there's a new add-ons page where anyone can share her/his tools or snippets (I guess I'll have to brush up my Typemock Snippets For Visual Studio to add there) and an experts page (guess who's there).

Typemock fan Meet the experts

Upcoming Typemock Webcast on March 12

Typemock is having a special Webcast on Wednesday March 12, 16:00 GMT

The live demo will include:

  • The Support of the new .NET 3.5 framework and specifically how to mock LINQ statements.
  • The improved IDE based on using colors to emphasize mocked methods.
  • Seamless integration with  Microsoft’s Visual Studio 2008.

It is also an opportunity to meet Typemock’s development team, ask questions online and share some tips.

.....and one of you can win a Nintendo Wii during in the webcast!

Click here to participate in the Webcast

TypeMock Isolator 4.2 Released

TypeMock has released version 4.2 of its mocking framework: TypeMock Insulator. Check out the release notes.

My four favorite new features in this release are:

Improved mock chaining

(If you want to learn more about chains, read this post from Gil)

Take this class to be tested:

public class TestedClass1
{
    public IDictionary<string, object> Dictionary
    {
        get { throw new NotImplementedException(); }
    }

    public object Method(bool cond)
    {
        if (cond)
        {
            return Dictionary["true"];
        }
        else
        {
            return Dictionary["false"];
        }
    }
}

The following test:

[TestMethod]
[VerifyMocks]
public void TestMethod1()
{
    object trueObject = new object();
    object falseObject = new object();

    TestedClass1 target = RecorderManager.CreateMockedObject<TestedClass1>();

    using (RecordExpectations recorder = RecorderManager.StartRecording())
    {
        recorder.DefaultBehavior.CheckArguments();

        recorder.ExpectAndReturn(target.Dictionary["true"], trueObject);
        recorder.FailWhenCalled(target.Dictionary["false"]);
    }

    object value = target.Method(true);
    Assert.AreEqual(trueObject, value);
}

Would simply fail with:

Test method TestProject1.UnitTest1.TestMethod1 threw exception:  TypeMock.VerifyException: 
TypeMock Verification: Method TestProject1.TestedClass1.get_Dictionary() has 1 more expected calls
.

Now, it just passes.

Debugger evaluation doesn't change the test behavior

Now you can debug your code without having to change your tests. You can query mocked values without influencing your tests.

You can do this and still have the test passing:

Debugger Support

The debugger can be run inside a recording block without confusing the test

You can now step into the recording block and debug it without affecting the recording process and the test.

Recording Block

When you step in the debugger into a mocked method, the mocked method will be painted

When you step into a mocked method (or property) I'll see that it's mocked.

Visual Studio Cues

Typemock Insulator Is Typemock Isolator

Yesterday I made a mistake with the name of the next version of TypeMock. The real name is TypeMock Isolator. Whatever the name is, you should try it out.

TypeMock Insulator 4.2 Beta Publicly Available

TypeMock (now called) Insulator 4.2 Beta is publicly available. Check out the release notes.

One of my favorite new features is the improved mock chaining.

Take this class to be tested:

public class TestedClass1
{
    public IDictionary<string, object> Dictionary
    {
        get { throw new NotImplementedException(); }
    }

    public object Method(bool cond)
    {
        if (cond)
        {
            return Dictionary["true"];
        }
        else
        {
            return Dictionary["false"];
        }
    }
}

The following test:

[TestMethod]
[VerifyMocks]
public void TestMethod1()
{
    object trueObject = new object();
    object falseObject = new object();

    TestedClass1 target = RecorderManager.CreateMockedObject<TestedClass1>();

    using (RecordExpectations recorder = RecorderManager.StartRecording())
    {
        recorder.DefaultBehavior.CheckArguments();

        recorder.ExpectAndReturn(target.Dictionary["true"], trueObject);
        recorder.FailWhenCalled(target.Dictionary["false"]);
    }

    object value = target.Method(true);
    Assert.AreEqual(trueObject, value);
}

Would simply fail with:

Test method TestProject1.UnitTest1.TestMethod1 threw exception:  TypeMock.VerifyException: 
TypeMock Verification: Method TestProject1.TestedClass1.get_Dictionary() has 1 more expected calls
.

Now, it just passes.

Visual Studio cues are nice too.

POUT – That’s The Way I Like It

POUT - Plain Old Unit Testing

Read all about it here.

TDD Proven Effective! Or is it? – by Jacob Proffitt

Jacob Proffitt has some comments about the the paper on the effectiveness of test-first approach to programming.

Here are a few excerpts:

If you're like me, you understand the problem better after implementing it and can thus create more relevant tests after the fact than before. I know better where the important edge-cases are and how to expose them and that could mean that my tests are more robust.

Anyway, without question, testing first leads to having more tests per functional unit. The question is if this is valuable.