Im in the process of learning how to use Selenium to automate we browsers and working my way through an online course.
Im currently having some problems trying to run a headless browser test using the HtmlUnitDriver
I have a simple test that should load a web page and assert the title, code as follows.
package com.seleniumsimplified.webdriver;
import org.junit.Test;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.htmlunit.HtmlUnitDriver;
import static org.junit.Assert.assertTrue;
public class MyFirstTest {
@Test
public void driverIsTheKing(){
WebDriver driver = new HtmlUnitDriver();
driver.get("http://compendiumdev.co.uk/selenium");
assertTrue(driver.getTitle().startsWith("Selenium Simplified"));
}
}
On attempting to run the test, the test does not appear to run properly, it takes around 4 minuites after which time i receive the following error which im not sure on how to resolve:
"C:\Program Files\Java\jdk1.8.0_131\bin\java" -ea -Didea.launcher.port=7532 "-Didea.launcher.bin.path=C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community` Edition 2016.2\bin" -Didea.junit.sm_runner -Dfile.encoding=UTF-8 -classpath "C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 2016.2\lib\idea_rt.jar;C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 2016.2\plugins\junit\lib\junit-rt.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\jaccess.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\jfxrt.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\localedata.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\nashorn.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunec.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunmscapi.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\zipfs.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\plugin.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\rt.jar;C:\Users\ra.r.hope\IdeaProjects\webdriverbasics\target\test-classes;C:\Users\robert.hope\.m2\repository\junit\junit\4.12\junit-4.12.jar;C:\Users\robert.hope\.m2\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-java\3.5.3\selenium-java-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-api\3.5.3\selenium-api-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-chrome-driver\3.5.3\selenium-chrome-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-edge-driver\3.5.3\selenium-edge-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-firefox-driver\3.5.3\selenium-firefox-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-ie-driver\3.5.3\selenium-ie-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-opera-driver\3.5.3\selenium-opera-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-remote-driver\3.5.3\selenium-remote-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-safari-driver\3.5.3\selenium-safari-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-support\3.5.3\selenium-support-3.5.3.jar;C:\Users\robert.hope\.m2\repository\cglib\cglib-nodep\3.2.4\cglib-nodep-3.2.4.jar;C:\Users\robert.hope\.m2\repository\org\apache\commons\commons-exec\1.3\commons-exec-1.3.jar;C:\Users\robert.hope\.m2\repository\org\apache\commons\commons-lang3\3.5\commons-lang3-3.5.jar;C:\Users\robert.hope\.m2\repository\commons-codec\commons-codec\1.10\commons-codec-1.10.jar;C:\Users\robert.hope\.m2\repository\commons-io\commons-io\2.5\commons-io-2.5.jar;C:\Users\robert.hope\.m2\repository\commons-logging\commons-logging\1.2\commons-logging-1.2.jar;C:\Users\robert.hope\.m2\repository\org\w3c\css\sac\1.3\sac-1.3.jar;C:\Users\robert.hope\.m2\repository\net\sourceforge\cssparser\cssparser\0.9.23\cssparser-0.9.23.jar;C:\Users\robert.hope\.m2\repository\com\google\code\gson\gson\2.8.0\gson-2.8.0.jar;C:\Users\robert.hope\.m2\repository\com\google\guava\guava\23.0\guava-23.0.jar;C:\Users\robert.hope\.m2\repository\com\google\code\findbugs\jsr305\1.3.9\jsr305-1.3.9.jar;C:\Users\robert.hope\.m2\repository\com\google\errorprone\error_prone_annotations\2.0.18\error_prone_annotations-2.0.18.jar;C:\Users\robert.hope\.m2\repository\com\google\j2objc\j2objc-annotations\1.1\j2objc-annotations-1.1.jar;C:\Users\robert.hope\.m2\repository\org\codehaus\mojo\animal-sniffer-annotations\1.14\animal-sniffer-annotations-1.14.jar;C:\Users\robert.hope\.m2\repository\net\sourceforge\htmlunit\htmlunit\2.27\htmlunit-2.27.jar;C:\Users\robert.hope\.m2\repository\net\sourceforge\htmlunit\htmlunit-core-js\2.27\htmlunit-core-js-2.27.jar;C:\Users\robert.hope\.m2\repository\net\sourceforge\htmlunit\neko-htmlunit\2.27\neko-htmlunit-2.27.jar;C:\Users\robert.hope\.m2\repository\org\apache\httpcomponents\httpclient\4.5.3\httpclient-4.5.3.jar;C:\Users\robert.hope\.m2\repository\org\apache\httpcomponents\httpcore\4.4.6\httpcore-4.4.6.jar;C:\Users\robert.hope\.m2\repository\org\apache\httpcomponents\httpmime\4.5.3\httpmime-4.5.3.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\jetty-client\9.4.5.v20170502\jetty-client-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\jetty-http\9.4.5.v20170502\jetty-http-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\jetty-io\9.4.5.v20170502\jetty-io-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\jetty-util\9.4.5.v20170502\jetty-util-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\net\java\dev\jna\jna\4.1.0\jna-4.1.0.jar;C:\Users\robert.hope\.m2\repository\net\java\dev\jna\jna-platform\4.1.0\jna-platform-4.1.0.jar;C:\Users\robert.hope\.m2\repository\com\codeborne\phantomjsdriver\1.4.0\phantomjsdriver-1.4.0.jar;C:\Users\robert.hope\.m2\repository\javax\servlet\javax.servlet-api\3.1.0\javax.servlet-api-3.1.0.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\websocket\websocket-api\9.4.5.v20170502\websocket-api-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\websocket\websocket-client\9.4.5.v20170502\websocket-client-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\websocket\websocket-common\9.4.5.v20170502\websocket-common-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\xalan\serializer\2.7.2\serializer-2.7.2.jar;C:\Users\robert.hope\.m2\repository\xalan\xalan\2.7.2\xalan-2.7.2.jar;C:\Users\robert.hope\.m2\repository\xerces\xercesImpl\2.11.0\xercesImpl-2.11.0.jar;C:\Users\robert.hope\.m2\repository\xml-apis\xml-apis\1.4.01\xml-apis-1.4.01.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\htmlunit-driver\2.27\htmlunit-driver-2.27.jar" com.intellij.rt.execution.application.AppMain com.intellij.rt.execution.junit.JUnitStarter -ideVersion5 com.seleniumsimplified.webdriver.MyFirstTest,driverIsTheKing
Sep 13, 2017 8:24:34 AM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://compendiumdev.co.uk:443: Connection reset
Sep 13, 2017 8:24:34 AM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://compendiumdev.co.uk:443
Sep 13, 2017 8:25:34 AM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://compendiumdev.co.uk:443: Connection reset
Sep 13, 2017 8:25:34 AM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://compendiumdev.co.uk:443
Sep 13, 2017 8:26:43 AM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://compendiumdev.co.uk:443: Connection reset
Sep 13, 2017 8:26:43 AM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://compendiumdev.co.uk:443
org.openqa.selenium.WebDriverException: java.net.SocketException: Connection reset
Build info: version: '3.5.3', revision: 'a88d25fe6b', time: '2017-08-29T12:42:44.417Z'
System info: host: 'LT-LGC07999', ip: '10.44.22.14', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_131'
Driver info: driver.version: HtmlUnitDriver
at org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:686)
at org.openqa.selenium.htmlunit.HtmlUnitDriver.lambda$8(HtmlUnitDriver.java:655)
at org.openqa.selenium.htmlunit.HtmlUnitDriver.lambda$0(HtmlUnitDriver.java:412)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1769)
at sun.security.ssl.HandshakeOutStream.flush(HandshakeOutStream.java:124)
at sun.security.ssl.Handshaker.kickstart(Handshaker.java:1067)
at sun.security.ssl.SSLSocketImpl.kickstartHandshake(SSLSocketImpl.java:1487)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1351)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
at com.gargoylesoftware.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory.connectSocket(HtmlUnitSSLConnectionSocketFactory.java:186)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:194)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1379)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1429)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1298)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:392)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:313)
at org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:668)
... 3 more
I did some research online and found out that javascript is disabled by default when using the HTMLUnitDriver, however im not sure if this is significant to my problem
I experimented with enabling java script by setting the value to true when creating a new HtmlUnitDriver
WebDriver driver = new HtmlUnitDriver(true);
I tried doing this and re running the code but unfortunatley i reveive the same result when running the test......
Has anyone come across this type of issue before and is able to provide any guidance on how i can get the test to run.
For reference im using Selenium 3.53 junit 4.12 and htmlUintDriver 2.27
Additonaly i can run the same test using the chromeDriver or the firefoxDriver and the test runs and passes correctly
Thanks
HtmlUnitDriver was a part of Selenium version 2.53. If you are using Selenium 2.52 or earlier you don't need to download and install HtmlUnitDriver, it is already there.
But still you want to use it download it from here
https://github.com/SeleniumHQ/htmlunit-driver/releases
From 2.53 version, you need to add it separately