AngularJS Protractor tests time out when run through Bamboo

Question

We have an end to end test suite for our AngularJS app. written using Protractor. The tests are run via a Grunt task. When run locally on a windows command line the tests pass. When we run them on the command line on our CI machine that hosts Bamboo they pass. However when Bamboo runs the grunt task "test:e2e", they fail. The failures are various timeouts (see sample of logs below). We've tried increasing some of the timeouts but it does not seem to help. Indeed the 30s timeout seems rather large already.

I'm out of ideas. Does anyone have any other avenues of investigation to suggest? We have temporarily disabled e2e as part of our CI build.

build   17-Jun-2015 23:34:59    [4mRunning "protractor:run" (protractor) task[24m
build   17-Jun-2015 23:35:00    Starting selenium standalone server...
build   17-Jun-2015 23:35:00    [launcher] Running 1 instances of WebDriver
build   17-Jun-2015 23:35:04    Selenium standalone server started at http://10.161.0.147:58338/wd/hub
build   17-Jun-2015 23:35:09    Started
error   17-Jun-2015 23:46:59    A Jasmine spec timed out. Resetting the WebDriver Control Flow.
error   17-Jun-2015 23:46:59    The last active task was: 
error   17-Jun-2015 23:46:59    WebElement.click()
error   17-Jun-2015 23:46:59        at [object Object].webdriver.WebDriver.schedule (C:\bamboo-home\xml-data\build-dir\PA-PPA-JOB1\node_modules\protractor\node_modules\selenium-webdriver\lib\webdriver\webdriver.js:345:15)
error   17-Jun-2015 23:46:59        at [object Object].webdriver.WebElement.schedule_ (C:\bamboo-home\xml-data\build-dir\PA-PPA-JOB1\node_modules\protractor\node_modules\selenium-webdriver\lib\webdriver\webdriver.js:1727:23)
error   17-Jun-2015 23:46:59        at [object Object].webdriver.WebElement.click (C:\bamboo-home\xml-data\build-dir\PA-PPA-JOB1\node_modules\protractor\node_modules\selenium-webdriver\lib\webdriver\webdriver.js:1832:15)
error   17-Jun-2015 23:46:59        at actionFn (C:\bamboo-home\xml-data\build-dir\PA-PPA-JOB1\node_modules\protractor\lib\element.js:75:32)
error   17-Jun-2015 23:46:59        at C:\bamboo-home\xml-data\build-dir\PA-PPA-JOB1\node_modules\protractor\lib\element.js:394:17
error   17-Jun-2015 23:46:59        at Array.forEach (native)
error   17-Jun-2015 23:46:59        at C:\bamboo-home\xml-data\build-dir\PA-PPA-JOB1\node_modules\protractor\lib\element.js:393:9
error   17-Jun-2015 23:46:59        at C:\bamboo-home\xml-data\build-dir\PA-PPA-JOB1\node_modules\protractor\node_modules\selenium-webdriver\lib\goog\base.js:1582:15
error   17-Jun-2015 23:46:59        at [object Object].webdriver.promise.ControlFlow.runInNewFrame_ (C:\bamboo-home\xml-data\build-dir\PA-PPA-JOB1\node_modules\protractor\node_modules\selenium-webdriver\lib\webdriver\promise.js:1654:20)
error   17-Jun-2015 23:47:29    A Jasmine spec timed out. Resetting the WebDriver Control Flow.
error   17-Jun-2015 23:47:29    The last active task was: 
error   17-Jun-2015 23:47:29    Protractor.waitForAngular()
error   17-Jun-2015 23:47:29        at [object Object].webdriver.WebDriver.schedule (C:\bamboo-home\xml-data\build-dir\PA-PPA-JOB1\node_modules\protractor\node_modules\selenium-webdriver\lib\webdriver\webdriver.js:345:15)
error   17-Jun-2015 23:47:29        at [object Object].Protractor.executeAsyncScript_ (C:\bamboo-home\xml-data\build-dir\PA-PPA-JOB1\node_modules\protractor\lib\protractor.js:251:26)
error   17-Jun-2015 23:47:29        at [object Object].Protractor.waitForAngular (C:\bamboo-home\xml-data\build-dir\PA-PPA-JOB1\node_modules\protractor\lib\protractor.js:274:15)

Answer 1

There are many reasons why protractor tests may pass in one place, but fail in other places. Here is a non-exhaustive list of reasons that I have had happen to me:

1. Bamboo server has different binaries than local (eg- different versions of chrome, chrome driver, selenium, protractor, karma, ...)
2. Bamboo server has multiple processes hitting the selenium server at once
3. Bamboo server is slower/faster than locally

Numbers 1 and 2 are easy to diagnose and fix, so I won't go further into detail about them. #3 is more tricky and insidious to diagnose and fix since the errors are time dependent and happen only occasionally.

One thing that I have found that can locate many of these temporal errors is to force your server to be really slow (just for diagnosis, obviously). When we added a 5s wait to all REST calls, this uncovered a lot of flakiness in our tests. It's a long and tedious process to fix all these problems, but it will work.

If you're not feeling like you want a real fix, but just want to patch up the problems, try adding an implicit wait to your WebDriver instance. It would look something like this in your protractor.conf file:

browser.webDriver.implicitlyWait(5000);