Using multiple Chrome versions with webdrivermanager fails

Question

My issue: chromedriver and chrome versions do not match in my test runs

I use a custom docker image for my test runs (RHEL Linux). In this I install the latest version of Google Chrome rpm, and another older version of Chrome for testing. The main reason I am doing this is the Google Chrome RPM handles all the dependencies for me since the Chrome for testing only comes as a zip, but I also want to have a specific slightly older version for certain purposes (which Google chrome rpm package does not provide, it's always latest).

Google Chrome (RPM) installs in

/usr/bin/google-chrome 
/usr/bin/google-chrome -> /etc/alternatives/google-chrome
/etc/alternatives/google-chrome -> /usr/bin/google-chrome-stable

E.g. when I execute "google-chrome --version" in the docker image I get

122.0.6261.57

Chrome for testing (ZIP) gets installed in

/usr/local/opt/chrome-linux64

with:

/usr/local/bin/chrome -> /usr/local/opt/chrome-linux64/chrome
120.0.6099.109 

Furthermore, I install chromedriver for both versions in my

~/.cache/selenium/chromedriver/

so this has two subfolders with the respective versions mentioned above. And also /usr/local/bin/chromedriver is linked to the latest version (122)

Now when I run selenium with webdrivermanager, trying to use "just google-chrome" I get the following result:

INFO  @ c.d.i.w.c.CustomChromeDriverCreator | Starting the initialization of 'chrome' WebDriver instance
DEBUG @ i.g.b.w.WebDriverManager | Using WebDriverManager 5.5.3
DEBUG @ i.g.b.w.c.ResolutionCache | Created new resolution cache file at /home/jenkins/.cache/selenium/resolution.properties
TRACE @ i.g.b.w.v.VersionDetector | Reading online commands.properties to find out driver version
DEBUG @ i.g.b.w.v.VersionDetector | Detecting chrome version using online commands.properties
DEBUG @ i.g.b.w.v.Shell | Running command on the shell: [google-chrome, --version]
DEBUG @ i.g.b.w.v.Shell | Result: Google Chrome 122.0.6261.57
TRACE @ i.g.b.w.v.VersionDetector | Detected browser version is 122.0.6261.57
INFO  @ i.g.b.w.WebDriverManager | Using chromedriver 122.0.6261.57 (resolved driver for Chrome 122)
DEBUG @ i.g.b.w.c.ResolutionCache | Storing resolution chrome=122 in cache (valid until 11:49:24 22/02/2024 UTC)
DEBUG @ i.g.b.w.c.ResolutionCache | Storing resolution chrome122=122.0.6261.57 in cache (valid until 10:49:24 23/02/2024 UTC)
TRACE @ i.g.b.w.c.CacheHandler | Checking if chromedriver exists in cache
TRACE @ i.g.b.w.c.CacheHandler | Filter cache by chromedriver -- input list [/home/jenkins/.cache/selenium/chromedriver/linux64/120.0.6099.109/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver, /home/jenkins/.cache/selenium/resolution.properties] -- output list [/home/jenkins/.cache/selenium/chromedriver/linux64/120.0.6099.109/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver] 
TRACE @ i.g.b.w.c.CacheHandler | Filter cache by 122.0.6261.57 -- input list [/home/jenkins/.cache/selenium/chromedriver/linux64/120.0.6099.109/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver] -- output list [/home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver] 
TRACE @ i.g.b.w.c.CacheHandler | Filter cache by LINUX -- input list [/home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver] -- output list [/home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver] 
TRACE @ i.g.b.w.c.CacheHandler | Filter cache by 64 -- input list [/home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver] -- output list [/home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver] 
DEBUG @ i.g.b.w.WebDriverManager | Driver chromedriver 122.0.6261.57 found in cache
INFO  @ i.g.b.w.WebDriverManager | Exporting webdriver.chrome.driver as /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver

So webdrivermanager finds the right browser (google-chrome v122) and the matching chromedriver.

But then my tests fail with an error:

1) Error in custom provider, org.openqa.selenium.SessionNotCreatedException: session not created: This version of ChromeDriver only supports Chrome version 122
Current browser version is 120.0.6099.109 with binary path /usr/local/bin/chrome

But according to https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver/01fde32d0ed245141e24151f83b7c2db31d596a4 the default search path should be

/usr/bin/google-chrome

so why does it pick /usr/local/bin/chrome ?

Same goes for https://github.com/bonigarcia/webdrivermanager/blob/7cb9dcf7ee66dec78230454aecac5533b5f7858a/src/main/resources/commands.properties#L11

Note: in my code I am just calling

WebDriverManager.chromedriver().setup();

I am not setting any binary path options in this case.

If the chromedriver instance was set up with the latest version (122), should it not also start that browser when being used?

Answer 1

So I finally found an approach that works:

• install Google chrome in docker image in /usr/bin/google-chrome as normal
• install additional browsers in a non-standard location where the automatic lookup does not find them. This will avoid confusing webdrivermanager as to which browser to use.
• install chromedrivers for all browsers to selenium cache folder
• to use the default Google Chrome just use WebDriverManager.chromedriver().setup(); in the code as normal
• to use other browsers, provide the non-standard path to the binary location in you docker image: WebDriverManager.chromedriver().browserVersionDetectionCommand(customChromePath+ " --version").setup(); chromeOptions.setBinary(customChromePath);

then the selection of both browser and matching chroemdriver will work as expected with webdrivermanager (5.7.0 at least)

Answer 2

Thanks for the detailed question. Indeed, it would be more accurate than WebDriverManager to use the command /usr/bin/google-chrome --version to discover the Chrome version in Linux. Therefore, I changed the commands database to use that command first.

Since WebDriverManager uses the online version of that database, you don't need to do anything. Next time that WebDriverManager tries to discover your Chrome's version, you should see something like this:

2024-02-22 14:11:44 [main] DEBUG i.g.bonigarcia.wdm.WebDriverManager.<init>(226) - Using WebDriverManager 5.7.0
2024-02-22 14:11:44 [main] DEBUG i.g.b.wdm.cache.ResolutionCache.<init>(77) - Created new resolution cache file at /home/user/.cache/selenium/resolution.properties
2024-02-22 14:11:45 [main] DEBUG i.g.b.wdm.versions.VersionDetector.getBrowserVersionFromTheShell(238) - Detecting chrome version using online commands.properties
2024-02-22 14:11:45 [main] DEBUG i.g.bonigarcia.wdm.versions.Shell.runAndWaitArray(65) - Running command on the shell: [/usr/bin/google-chrome, --version]
2024-02-22 14:11:45 [main] DEBUG i.g.bonigarcia.wdm.versions.Shell.runAndWaitArray(69) - Result: Google Chrome 120.0.6099.199
2024-02-22 14:11:46 [main] INFO  i.g.bonigarcia.wdm.WebDriverManager.resolveDriverVersion(1219) - Using chromedriver 120.0.6099.109 (resolved driver for Chrome 120)
2024-02-22 14:11:46 [main] DEBUG i.g.b.wdm.cache.ResolutionCache.putValueInResolutionCacheIfEmpty(119) - Storing resolution chrome=120 in cache (valid until 15:11:46 22/02/2024 CET)
2024-02-22 14:11:46 [main] DEBUG i.g.b.wdm.cache.ResolutionCache.putValueInResolutionCacheIfEmpty(119) - Storing resolution chrome120=120.0.6099.109 in cache (valid until 14:11:46 23/02/2024 CET)
2024-02-22 14:11:46 [main] DEBUG i.g.bonigarcia.wdm.WebDriverManager.manage(1178) - Driver chromedriver 120.0.6099.109 found in cache
2024-02-22 14:11:46 [main] INFO  i.g.bonigarcia.wdm.WebDriverManager.exportDriver(1281) - Exporting webdriver.chrome.driver as /home/user/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver

Answer 3

Update: I also tried the other way around: if I explicitly want to use the older version (by setting a custom shell env variable and passing that to my Java code), then I tried this:

WebDriverManager.chromedriver().browserVersion("120.0.6099.109").setup();

and explicitly pointing to the chrome for testing which is that version, using

chromeOptions.setBinary("/usr/local/bin/chrome");

What then happens is that webdrivermanager (5.5.3 - yeah I know not the latest) fails to get the chromedriver from the website

WARN @ i.g.b.w.v.VersionDetector | Exception reading CfT URL ('null') to get version of chromedriver (For input string: "120.0.6099.109") ERROR @ i.g.b.w.o.HttpClient | Error HTTP 404 executing https://googlechromelabs.github.io/chrome-for-testing/LATEST_RELEASE_120.0.6099.109 WARN @ i.g.b.w.v.VersionDetector | Exception reading https://googlechromelabs.github.io/chrome-for-testing/LATEST_RELEASE_120.0.6099.109 to get latest version of chromedriver (Error HTTP 404 executing https://googlechromelabs.github.io/chrome-for-testing/LATEST_RELEASE_120.0.6099.109)

then it checks the local cache

TRACE @ i.g.b.w.c.CacheHandler | Filter cache by chromedriver -- input list [/home/jenkins/.cache/selenium/chromedriver/linux64/120.0.6099.109/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver, /home/jenkins/.cache/selenium/resolution.properties] -- output list [/home/jenkins/.cache/selenium/chromedriver/linux64/120.0.6099.109/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/LICENSE.chromedriver, /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver]

where the 120 version is actually present! but it still selects version 122

INFO @ i.g.b.w.WebDriverManager | Exporting webdriver.chrome.driver as /home/jenkins/.cache/selenium/chromedriver/linux64/122.0.6261.57/chromedriver

and then of course fails with

1. Error in custom provider, org.openqa.selenium.SessionNotCreatedException: session not created: This version of ChromeDriver only supports Chrome version 122 Current browser version is 120.0.6099.109 with binary path /usr/local/bin/chrome

I am assuming that even though I forcibly set the chromedriver version to "120.0.6099.109" and later explicitly set the binary path, it does a check of the "standard default"

/usr/bin/google-chrome

in between and because that exists, still uses that.

I am aware that my setup is maybe not typical/normal, but I wonder what the correct approach is to have multiple browser versions pre-installed and reliably switch between them AND also fallback on the webdrivermanager automatism?

Reason being: I want to use the same code in different environments. On my Jenkins docker container I do want to pre-install all the binaries (and not download them on-the-fly) to speed up test execution runs. At the same time my QA team runs the same code locally and has varying setup (MacOS, Windows) and as Chrome can update at different times I would like the convenience of automatic chromedrivers installed by webdrivermanager as a default fallback.

So I think, logically, the order should be:

1. Set a binary path for a chrome browser optionally for setup()
2. if no explicit path given, use the search commands.database to find a chrome binary (nice extra: being able to override the list and order locally in a config.properties)
3. get the chrome browser version from that
4. check if the matching chromedriver exists in selenium cache and if yes use that
5. if not try to download the matching chromedriver

I think something like that is essentially this example here: https://github.com/bonigarcia/webdrivermanager/blob/master/src/test/java/io/github/bonigarcia/wdm/test/chrome/ChromeBetaTest.java but would it not make sense to have a method called

WebDriverManager.chromedriver().setCustomBrowserCommand(chromeBetaPath)

that does it all instead of browserVersionDetectionCommand followed by setBinary ?