Using the TestCaseSource attribute in NUnit

I’ve used NUnit for the last few years but I only recently discovered the TestCaseSource attribute which is very useful in certain circumstances.

The scenario

Imagine you have a DeviceInformation class. It’s job is to take a user agent, parse it and provide information about the device. The class could look something like this:

public class DeviceInformation
{
    public bool IsDesktop { get; private set; }
    public bool IsMobile { get; private set; }
    public bool IsTablet { get; private set; }
    public string Manufacturer { get; private set; }

    public DeviceInformation(string userAgent)
    {
        // Some code here which sets the class properties
    }
}

One way to attack this is to create a bunch of user agents and use the TestCase attribute to confirm that multiple user agents return the same result.

private const string USER_AGENT_IPHONE_4 = "...";
private const string USER_AGENT_GALAXY_S5 = "...";
private const string USER_AGENT_LG_G4 = "...";
private const string USER_AGENT_GALAXY_TAB = "...";

[TestCase(USER_AGENT_IPHONE_4)]
[TestCase(USER_AGENT_GALAXY_S5)]
[TestCase(USER_AGENT_LG_G4)]
public void IsMobileDevice_UserAgentIsMobile_ReturnsTrue(string userAgent)
{
    Assert.IsTrue(new DeviceInformation(userAgent).IsMobile);
}

[TestCase(USER_AGENT_GALAXY_TAB)]
[TestCase(USER_AGENT_GALAXY_S5)]
[TestCase(USER_AGENT_LG_G4)]
public void IsAndroidDevice_UserAgentIsAndroid_ReturnsAndroid(string userAgent)
 {
     Assert.That(new DeviceInformation(userAgent).Manufacturer, Is.EqualTo("Android"));
 }

// More tests to cover the IsDesktop, IsTablet and Manufacturer properties here

This is a trivial example and there are already quite a few user agents. You can see that if I were to thoroughly unit test DeviceInformation I would need a lot more user agents and a lot more tests. As the number of users agents increased the number of attributes on each test would increase and the tests would become more difficult to maintain. Each time a user agent was added I’d need to scan every single test and make sure new attributes were added where appropriate.

Using the TestCaseSource attribute

From the NUnit documentation:

TestCaseSourceAttribute is used on a parameterized test method to identify the property, method or field that will provide the required arguments.

This means you can use a property, method or field to hold all of the user agent strings for a particular test rather than specify each one in it’s own attribute. Here I’ve used arrays to organise user agents into groups:

readonly string[] mobileUserAgents =
{
    USER_AGENT_IPHONE_4,
    USER_AGENT_GALAXY_S5,
    USER_AGENT_LG_G4
};

readonly string[] androidUserAgents =
{
    USER_AGENT_GALAXY_TAB,
    USER_AGENT_GALAXY_S5,
    USER_AGENT_LG_G4
};

[Test, TestCaseSource("mobileUserAgents")]
public void IsMobileDevice_UserAgentIsMobile_ReturnsTrue(string userAgent)
{
    Assert.IsTrue(new DeviceInformation(userAgent).IsMobile);
}

[Test, TestCaseSource("androidUserAgents")]
public void IsAndroidDevice_UserAgentIsAndroid_ReturnsAndroid(string userAgent)
{
    Assert.That(new DeviceInformation(userAgent).Manufacturer, Is.EqualTo("Android"));
}

Now each test only has one attribute, much better. Also, to keep the snippet short I’ve only added two groups of user agents, but for a thorough test I’d add a lot more. For example appleUserAgents, tabletUserAgents, desktopUserAgents, backBerryUserAgents, windowsPhoneUserAgents etc. Now when you want to test a new user agent you simply create a new user agent variable and add it to the appropriate arrays. The is no need to change individual tests. Easy.

Why I don't trust code coverage

I had a discussion with a colleague about code coverage the other day where I suggested that 100% code coverage meant very little because you may have covered every line of code, but not necessarily every path through the code. After the conversion ended I thought I'd better check I wasn't talking utter nonsense.

Here is a contrived method and two unit tests.

Here are the code coverage results.

As you can see, 100% code coverage is reported but not all paths through the code have been tested. Yes, the code is dreadful but I think it makes my point. Just because every line of your code has been tested, it doesn't mean that your code has been thoroughly tested.

To make matters worse, lets say 90% code coverage is reported by whatever tool you use, there's no indication of the importance of the code that has been tested. What if the 10% of code that has minimal coverage is the most critical part of your application?

Don't get me wrong, I'm not saying code coverage is a complete waste of time. As a general indication that an area of code needs a few more tests it's great. But other than that, not so much.

Some thoughts on mocking

I find the mocking is one of the most tricky parts of unit testing. Where I work we use Moq which is a nice, simple mocking framework. It may not be as powerful as some of the commercial frameworks but it does the job. 

Roy Osherove suggests that a single unit test should only have one mock but often find myself creating several mocks. Perhaps the class under test has too many dependencies or perhaps I'm creating mocks where they are not needed or perhaps it's simply due to the limitations of the mocking framework. Whichever it is I'd like to get it clear in my head. Take the following example, it has three types of call that you're likely to mock when unit testing a method:

public void DoSomeStuff(User user) 

{    

   this.VerifySomeImportantThings(user);                                // 1

   int returnValue = someRepository.GetImportantValue();      // 2                                 

   someRespository.Save(user);                                                 // 3

}

1. Methods called on the same class under test always have to be mocked when using Moq because you have to tell the framework not to call their code. Also if they were tested it would be duplication as there should be separate unit tests for those methods
2. Method calls on dependencies which return a value only need to be mocked if you care about the return value or any exceptions thrown
3. Method calls on dependencies which do not return a value only need to be mocked if exceptions need to be tested