How to design an NUnit Theory?

Question

I have created an NUnit Theory to help me test some code. The actual code being tested isn't as important to this question as the data that I'm using to test it. Namely, the hours and minutes of time in a 24-hour clock.

I wrote my fixture this way in order to capitalize on the features and conform to the limitations within the NUnit 2.6 Theory feature. In particular, I felt that I had to create classes such as Hour and Minute in order to work-around the feature that Datapoints are matched to arguments by exact type.

[TestFixture]
public class TimeWindowParserTheoryFixture
{
    public class Hour
    {
        public int Value;
    }

    public class Minute
    {
        public int Value;
        public string AsString { get { return Value.ToString("00"); } }
    }

    [Datapoints]
    public IEnumerable<Hour> Hours
    {
        get
        {
            return Enumerable
                .Range(0, 25)
                .Select(v => new Hour() { Value = v })
                .Union(Enumerable.Repeat((Hour)null, 1));
        }
    }

    [Datapoints]
    public IEnumerable<Minute> Minutes
    {
        get
        {
            return Enumerable
                .Range(0, 60)
                .Select(v => new Minute() { Value = v })
                .Union(Enumerable.Repeat((Minute)null, 1));
        }
    }

    [Datapoints]
    public IEnumerable<string> Separators
    {
        get { return new[] { " ", "-" }; }
    }

    [Theory]
    public void ValidHours(Hour startHour,
        Minute startMinute,
        Hour endHour,
        Minute endMinute,
        string separator)
    {
        Assume.That(startHour != null);
        Assume.That(endHour != null);

        var parser = new TimeWindowParser();
        var startMinutesString = String.Format("{0}{1}", startMinute == null ? "" : ":", startMinute == null ? "" : startMinute.AsString);
        var endMinutesString = String.Format("{0}{1}", endMinute == null ? "" : ":", endMinute == null ? "" : endMinute.AsString);
        var pattern = String.Format("{0}{1}{2}{3}{4}{5}{6}", startHour, startMinutesString, "", separator, endHour, endMinutesString, "");
        //Console.WriteLine(pattern);
        var result = parser.Parse(pattern);
        Assert.That(result, Is.Not.Null);
        Assert.That(result.Start, Is.EqualTo(startHour));
        Assert.That(result.End, Is.EqualTo(endHour));
    }
}

What I found is that the size of the data set produced during the default combinatorial logic of NUnit results in a set so large that I run out of memory. It doesn't seem like the way I've set up my test and data should be a problem but since it obviously is so I'm asking for advice on how to think about this problem differently. Here is the OutOfMemoryException stack trace that I get.

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Text.StringBuilder.ExpandByABlock(Int32 minBlockCharCount)
at System.Text.StringBuilder.Append(Char* value, Int32 valueCount)
at System.Text.StringBuilder.AppendHelper(String value)
at System.Text.StringBuilder.Append(String value)
at NUnit.Core.MethodHelper.GetDisplayName(MethodInfo method, Object[] arglist)
at NUnit.Core.Builders.NUnitTestCaseBuilder.BuildSingleTestMethod(MethodInfo method, Test parentSuite, ParameterSet parms)

This exception is odd in itself in that it appears to be generated by simply trying to get the Test method's name (see GetDisplayName). I'm not sure if this is a bug (known or otherwise). BTW, I get a very similar OOM exception when I re-wrote this fixture using the less-experimental Range and Value attributes used in Parameterized tests.

Answer 1

Your question is about the mechanics of using Theories, but I want to first talk about the "theory of Theories" then I'll get back to implementation details.

THEORY OF THEORIES

You should only be using a Theory if you have a theory to start out with. Otherwise, you're just using Datapoints to drive traditional example-driven testing. By "having a theory" I mean having some general statement of truth, which you want to prove over some set of inputs. Generaly the Theory (code) will go through the possible inputs and filter out anything that does not meet your assumptions. An example of a good theory is: For any positive real number, the square root multiplied by itself will give the same number."

Looking at your test, I can't figure out what theory your Theory is seeking to prove. It seems just like a standard parameterized test, with the exception that you are using DataPoints. The only assumption you need to make is that the minuts are not null, which is odd since you provide the minutes in the first place.

Bottom line: I don't think this is a good use of Theory.

MECHANICS

You are generating 24*24*60*60*2 DataPoints. That's a heck of a lot of data. Do you have any reason to believe that your parsing algorithm might work for - say - 24:13 but fail for 24:14 ?

It's true that Datapoints are used combinatorially and it might be better if they were used pairwise. But remember that DataPoints are only one of many ways to throw data at a Theory. The idea is that Theories should handle whatever data you give them.

ALTERNATIVE

I would write this as a test, since I can't think of any theory about your parser or parsers in general that would apply. I would only give the test valid data, probably using TestCaseSourceAttribute pointing to a method that generated valid strings or simply to a big array of strings. I'd have another test that handled various kinds of invalid data.

Charlie