Nunit: TestCaseSource does not expect to generate all test cases?

Question

Hi everyone I have a problem with generate test cases for TestCaseSource. I wrote this code for tests:

using System;
using System.Collections.Generic;
using System.Linq;

using NUnit.Framework;

namespace HeapSort.Tests
{
    [TestFixture]
    public class Tests
    {
        [Test, TestCaseSource(typeof(TestsGenerator),"TestCases")]
        public void IsEqualCollections(int[] received, int[] expected)
        {
            CollectionAssert.AreEqual(received, expected);
        }
    }

    public class TestsGenerator
    {
        public static IEnumerable<TestCaseData> TestCases
        {
            get
            {
                for (var i = 0; i < 25; i++)
                {
                    int[] t1 = GenerateCollection(i), t2 = t1.ToArray();
                    HeapSort.Sort(t1);
                    Array.Sort(t2);

                    yield return new TestCaseData(t1, t2);
                }
            }
        }

        private static int[] GenerateCollection(int seed)
        {
            var rnd = new Random(seed+DateTime.Now.Millisecond);
            int size = rnd.Next(100, 10000); 
            int[] array = new int[size];
                for (var i = 0; i < size; i++)
                    array[i] = rnd.Next(-100, 100);
            return array;

//            return Enumerable
//                .Repeat(100, rnd.Next(100, 10000))
//                .Select(i => rnd.Next(-100, 100))
//                .ToArray();
        }
    }
}

Where is the problem? Rather than get 25 tests, I get from 1 to 8. And often at the starting point of testing it shows that the tests are 7/8 and in the end there is only one test case.

How can I solve this problem?

UPD1: What's interesting is when I run tests through the console I handle all 25 tests how do I achieve the same results through the GUI!?

P.S. sorry for my bad english.

Perhaps I should mention that I'm working under Ubuntu in Rider

Answer 1

DateTime.Now is normally not very accurate. Your loop is generating many identical tests because they are all starting from the same seed. Why are you using a seed rather than simply letting Random work on its own?

Different runners will handle identical test in different ways. If you indicate what runner you are using to execute your tests, I can edit this answer with more information. However, in general, you most certainly don't want to generate a bunch of tests with the same data. They don't do anything for you!

Answer 2

The reason is TestExplorer looks at the name of test case and groups ones with equal names. So you don't see all test cases results. As reason is TestExplorer you don't have the same problem running test using console. Look at this issue for details. According to the issue there are two solutions:

1. create class for your test case and override ToString() method making it return unique values for each test case.
2. use TestCaseData class and its method SetName to set unique name for each test.