Debugging with headless browser

Question

I have a WebDriver testsuite, which operates different when I execute it in normal and headless browser. There is an element which is not found when I execute it in headless mode, but found when I use the same code, same driver in normal mode. I use this flag to set headless mode:

chromeOptions.addArguments("--headless");

There is ChromeDriver 2.31 and WebDriver 3.5.2 in use. How could I debug this?

Answer 1

There are two ways to debug. You can get Page Source and check what is different.

Now when you launch a browser using Selenium, it is using the Debugging session to automate chrome. So you can't do a remote debugger to your website using this.

You need to launch chrome manually.

chrome --headless --remote-debugging-port=9222 --disable-gpu http://tarunlalwani.com

Now in open another chrome and debug the site by going to http://127.0.0.1:9222 and inspect the site.

Answer 2

for anyone struggling with this (as i was), there's a powerful method you can use in selenium called execute_script.

as an example, i was finding an element by xpath and clicking on it:

browser.find_element_by_xpath("//li[@id='tabletid']").click()

but this wasn't working in headless. after inspecting the element i found out the button click actually executes a simple javascript line. so i replaced this line with:

js = """drawTable();
            return false;"""
    
browser.execute_script(js)

and i was able to actually pull my data! now my script will work in ssh, docker, etc.

Answer 3

for debugging headless, try to get an screenshot before the error:

in Python:

WINDOW_SIZE = "1200,900" opts.add_argument("--window-size=%s" % self.WINDOW_SIZE) 
if self.HEADLESS:   opts.add_argument('--headless')     
self.driver = webdriver.Chrome(executable_path=chromedriver,options=opts)  


 driver.save_screenshot('./foto.png')