unittest.TestSuite runs previously loaded tests in addition to currently added tests

Question

My code runs tests using the unittest framework. This is the basic idea of what one of my methods looks like:

    def _RunTestsList(self, lTestsPaths):
        """ Runs all tests in lTestsPaths with the unittest module
        """
        for sTestPath in lTestsPaths:
            testLoader = unittest.TestLoader()
            temp_module = imp.load_source('temp_module', sTestPath)
            tstSuite = testLoader.loadTestsFromModule(temp_module)
            unittest.TextTestRunner (verbosity=1).run(tstSuite)

What I'm obviously trying to achieve is to run the tests that are in the lTestsPaths list. For some reason what happens is, instead of running every test in lTestsPaths individually, it runs every test in addition to all the tests that were run previously. This happens also when calling this method from different places in the code. I.e. all the tests that were previously run (in earlier calls) are run again.

When debugging, I see that when tstSuite is initialized, it is initialized with all previously run tests.

Why is this happening? How can I make this code run as expected?

Answer 1

After many debugging hours I got to the root of the problem: The problem seems to be the name of the temp_module, that is, because I give all my temp modules the same name. This has something to do with the built-in dir method, as the dir method, which is called by testLoader.loadTestsFromModule(temp_module) returns test objects names that were run before. I'm not sure why, but this is the reason for code behavior.

To solve this I deleted the module name: 'temp_module' from sys.modules after using the module. There may be a cleaner way, but this works.

Here is the improved code that worked for me:

def _RunTestsList(self, lTestsPaths):
    """ Runs all tests in lTestsPaths with the unittest module
    """
    for sTestPath in lTestsPaths:
        testLoader = unittest.TestLoader()
        temp_module = imp.load_source('temp_module', sTestPath)
        tstSuite = testLoader.loadTestsFromModule(temp_module)
        unittest.TextTestRunner (verbosity=1).run(tstSuite)
        del sys.modules['temp_module']