Kill started Internet Explorer process without closing all IE browsers

Question

My C# program uses Selenium IE web driver to open a websites to do the automated testing (multiple browsers can be open if the websites could not be reached in the first attempt). After doing all the tests, the program will kill started IE processes, but the problem is that it also closes all IE browser that were not opened by the program (like in the case the user opened other websites in IE before running the program.) How could I fix this problem. This is my code:

 public void KillAllBrowsersAndWebDrivers()
    {
        var webDrivers = Process.GetProcessesByName("IEDriverServer").Select(p => p.Id);
        var browsers = Process.GetProcessesByName("iexplore").Select(p => p.Id);
        var processIds = webDrivers.Concat(browsers);
        foreach (var pid in processIds)
        {              
                Process.GetProcessById(pid).Kill();                    
        }
    }

Any help is highly appreciated.