Parallel Testing with MBUnit/Gallio using Data Driven Test Cases

Question

I've been doing a lot of reading up on RemoteWebDriver for Selenium 2 using Selenium Grid. At the moment my tests are generated in the Test() using the [Test] Attribute.

I have a TestCriteria class that I fill with information and then use the Selenium WebDriver 'Page Object Design Pattern' as a way of 'controlling' how this data is entered into my tests.

So, I have a simple criteria object, for example:

public class Credentials
{ 
    public string Username { get; set; } 
    public string Password { get; set; }
}

Then use this information in a LoginPage object.

public class LoginPage
{
    [FindsByAnnotation]
    public IWebElement UsernameField { get; set; }

    [FindsByAnnotation]
    public IWebelement PasswordField { get; set; }

    [FindsByAnnotation]
    public IWebelement SubmitButton { get; set; }

    public HomePage Login(Credentials cred)
    {
         UsernameField.SendKeys(cred.user);
         // code for login and return new HomePage object
    }
}

Now, with this sort of structure I am able to create some good method chaining in my Tests once I have the test data, things like my Credentials object, Data that needs to be filled in on other pages etc.

[TestFixture]
public class TestFixture
{
     private IWebDriver driver;
     private TestCaseData data; // Which contains a Credentials object etc etc

     [SetUp]
     public void SetUp()
     {
         data = new TestCaseData() 
         { 
              Credentials = new Credentials() 
              { 
                   Username = "username", 
                   Password = "password"
              }
             // Other test case data objects can be populated here 
         };

         driver = new FireFoxDriver();
     }

     [Test]
     public void Test()
     {
          new LoginPage().Login(data.Credentials)
                         .NavigateToSearchPage()
                         .EnterSearchCriteria(data.SearchCritiera)
          // etc
     }

}

This is all well and good, but...what If i wanted to LOAD in this test data from a serialised TestData object which can be deserialised from XML.

I'm also interested in using RemoteWebDriver, which I have already used however is still flakey compared to using just IWebDriver driver = new FireFoxDriver(); , however ignoring these matters, I would really like to run a TESTCASE more than once...at the same time. Which brings me to the question of parallel testing.

I understand MBUnit can handle this which is part of Gallio, I had also looked into PUnit which is an extension of Nunit but it still has a long way to come. So I have decided to stick with MbUnit.

This gives me the ability to run the TestFixtures with the attribute [Parallelizable]. So if I compile my project and load in the project dll into Gallio TestRunner GUI I can run 4 test cases at the same time, which in return opens up 4 browsers running each test at the same time, also giving the ability to stress the site while all 4 tests are running. (obviously this would be increased to run several hundred browsers on multiple machines using selenium RemoteWebDriver with Selenium Grid.

Does anyone here know a way where I can load in a collection of xml serialised objects, GENERATE new TestFixture objects which can have the [Parallelizable] attribute added to the top of the test fixture and have them all run at the same time after loading in 1 - 10 .xml files from a, for example: C:\TestCase directory?

My idea is to have these all loaded in, and then have Selenium Grid handle the browser sessions with 1 - 5 Selenium nodes running connected to the main selenium grid hub.

I am just having a difficulty really taking advantage of Selenium Grid when I can not find a framework which allows me to generate a Test Fixture, which includes a [SetUp] [TearDown] [Test] and the ability to set certain test conditions to Test Attributes depending on what is loaded from the TestCase .xml file.

For example if a TestCase .xml file had an element which was fail then how can I load this .xml file in and set an attribute on my TestFixture's [Test] for this, or map the TestCase .xml Description of test to the [Test(Description = "")] attribute at run time.

I already know the functionality of selenium webdriver, the page object design pattern, the selenium EventFiringWebDriver functionality for screenshots.

How can i utilize the ability to load in several XML serialised TestCaseData, generate new TestFixture objects after these have been loaded?

Ideally I would like the [SetUp] of each TestFixture to setup the IWebDriver because certain URLs would be different depending on what the TestCase.xml file would contain, for example of this test is to run on a UAT environment, Test environment, Pre-Production environment, I need to set this up before the test is run.

Does anyone have a basic example which uses these main concepts with Selenium Grid/Selenium WebDriver with the ability to run these Test Fixtures in parallel to utilise Selenium Grid functionality of running multiple browsers.

So something I am looking for in pseudo code.

public class Main()
{
   // Load Testfixtures
   List<TestCase> testCases = Deserialise("\\Some\\FolderLocation");

   foreach(test in testCases)
   { 
      // create NEW testFixture, not [Test]
      // ability to attach parallel TestFixture
   }  
}

[Testfixture]
[Parallelizable]
public class TestFixture
{

     public TestCase testCase { get; set; }
     public IWebDriver driver { get; set; }

     [SetUp]
     public void SetUp()
     {
         if(testCase.Environment == "UAT")
         {
             driver = new FireFoxDrive()
             driver.NavigateTo("http://localhost/uat/Login");
         }

         // What if the testCase.ShouldPass is false?

         // How can i generate my [Test] with ExpectedException?

     }

      [Test]
      public void Test()
      {
          // code here with page object method chaining passing in testCase data objects  
      }
}

Maybe I am going about this design the wrong way. Any suggestions please on the best way to parallelize these?

Answer 1

How about this?

public class TestCase
{
    public string Name { get; set; }
}

public static class TestCaseFactory
{
    public static IEnumerable<TestCase> TestCases()
    {
        yield return new TestCase { Name = "one" };
        yield return new TestCase { Name = "two" };
    }
}

[Factory(typeof(TestCaseFactory), "TestCases"), Parallelizable]
public class DataDrivenFixture
{
    private readonly TestCase testCase;

    public DataDrivenFixture(TestCase testCase)
    {
        this.testCase = testCase;
    }

    [SetUp]
    public void SetUp()
    {
        Console.WriteLine("SetUp " + testCase.Name);
    }

    [Test]
    public void Test()
    {
        Console.WriteLine("Test " + testCase.Name);
    }
}

Answer 2

good solution. but what about run tests in parallel different browsers? if I initialize IWebDriver in [FixtureSetUp] then browsers opened and run tests but with errors (basicly NoSuchElement).

public static IEnumerable<TestCase> TestCases()
        {
            yield return new TestCase { Name = "internet explorer" };
            yield return new TestCase { Name = "firefox" };
        }

[FixtureSetUp]
        public void SetUp()
        {
            Console.WriteLine("SetUp " + testCase.Name);
            switch (testCase.Name)
            {
                case "internet explorer":
                    driver = new InternetExplorerDriver();
                    break;
                case "chrome":
                    driver = new ChromeDriver();
                    break;
                case "firefox":
                    driver = new FirefoxDriver();
                    break;
            }
        }