How do I properly set up a BrowserMobProxy/Selenium Proxy with RemoteWebDriver/Sauce Labs/Sauce Connect?

Question

I've been looking everywhere and have not found any good documentation on how to properly set up a proxy to run a script on Sauce labs and extract the HAR file. I'm using BMP in an embedded mode https://github.com/lightbody/browsermob-proxy#using-with-selenium, along with https://wiki.saucelabs.com/display/DOCS/Sauce+Connect+Proxy. I found Sauce's documentation on setting up and running scripts through a BMP manually https://wiki.saucelabs.com/display/DOCS/Sauce+Connect+Proxy+with+an+Additional+Proxy+Setup, however their documentation does not show how to set it up in an embedded mode only via standalone mode. Here is my setup:

My PAC file

function FindProxyForURL(url, host) {
    if (shExpMatch(host, "*.miso.saucelabs.com") ||
        shExpMatch(host, "saucelabs.com")) {
        // KGP and REST connections. Another proxy can also be specified.
        return "DIRECT";
    }

    // Test HTTP traffic, route it through the local BrowserMob proxy.
    return "PROXY localhost:9091";
}

BMP Set up

package com.grainger.Framework;

import java.io.File;
import java.io.IOException;
import java.net.Inet4Address;
import java.net.UnknownHostException;

import org.apache.log4j.Logger;
import org.openqa.selenium.Proxy;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;

import com.grainger.Automation.Utilities;
import com.grainger.Build.BuildVariables;

import net.lightbody.bmp.BrowserMobProxy;
import net.lightbody.bmp.BrowserMobProxyServer;
import net.lightbody.bmp.client.ClientUtil;
import net.lightbody.bmp.core.har.Har;
import net.lightbody.bmp.proxy.CaptureType;

public class BrowserMobProxyImpl {
     public static Logger log = Logger.getLogger(BrowserMobProxyImpl.class.getName());
    private static BrowserMobProxy MOB_PROXY_SERVER;
    private static Proxy SELENIUM_PROXY;

    /**
     * @author xsxg091
     * @return
     */
    public static void startBrowserMobProxyServer(){
         // start the proxy
        MOB_PROXY_SERVER = getProxyServer();
        // get the Selenium proxy object
        SELENIUM_PROXY = getSeleniumProxy(MOB_PROXY_SERVER);
    }

    /**
     * @author xsxg091
     * @return
     */
    public static BrowserMobProxy getProxyServer() {
        BrowserMobProxy proxy = new BrowserMobProxyServer();
        proxy.setTrustAllServers(true); 
        proxy.start(9090);
        return proxy;
   }

    /**
     * @author xsxg091
     * @param proxyServer
     * @return
     */
    public static Proxy getSeleniumProxy(BrowserMobProxy proxyServer) {
        Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxyServer);;
        try {
            String hostIp = Inet4Address.getLocalHost().getHostAddress();
            seleniumProxy.setHttpProxy(hostIp + ":" + Integer.toString(9091));
            seleniumProxy.setSslProxy(hostIp + ":" + Integer.toString(9091));
            seleniumProxy.setAutodetect(false);
        } catch (UnknownHostException e) {
            log.error("Error initializing Selenium Proxy");
        }
        return seleniumProxy;
    }

    /**
     * @author xsxg091
     * @param tcName
     * @param capabilities
     */
    public static void setSeleniumProxy(DesiredCapabilities capabilities){
        if(BuildVariables.amICapturingNetworkTraffic()){
            capabilities.setCapability(CapabilityType.PROXY, SELENIUM_PROXY);
        }
    }

    /**
     * @author xsxg091
     * @param tcName
     * @param capabilities
     */
    public static void stopBrowserMobProxyServer(){
        MOB_PROXY_SERVER.stop();
    }

    /**
     * @author xsxg091
     * @return
     */
    public static void getHarFile(String fileName) {
        // enable more detailed HAR capture, if desired (see CaptureType for the complete list)
        MOB_PROXY_SERVER.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT);
        MOB_PROXY_SERVER.newHar(fileName);
        try {
            // get the HAR data
            Har pageHarFile = MOB_PROXY_SERVER.getHar();
            File harFile = new File(Utilities.getWorkSpace()+"//"+fileName+".har");
            pageHarFile.writeTo(harFile);
        } catch (IOException e) {
            log.error("Unable to store Har File");
        }
    }
}

Here is the command I use to kick off my Sauce Tunnel

bin/sc -u ****** -k *********** -i Tunnel_Testing -v --pac file:///<path-to-pac-file>/BrowserMobProxy/browserMob.js

When I run lsof, I can port 9090 is actively listening, but I don't see 9091 in embedded mode. However, when I run it in standalone mode I can see both ports and everything works perfectly on Sauce labs. I see this when I run in embedded mode:

What am I doing wrong? Any help would be greatly appreciated. If anything is unclear, please let me know!

Thanks in advance.

Answer 1

I figured it out. Turns out it was a bug in version 2.1.4. When I upgraded to version 2.1.5, everything worked as it is supposed to.