CONNECT through proxy removes additional headers required for authorization

Question

I am using Jgit library to provide git functionality in our application. When a git operation like clone is executed for cloning HTTPS repo, the request needs to go through an embedded jetty based reverse proxy server which we have created, and is running in the server.

We are using JDK 1.8, Jgit - 5.13.1.202206130422-r and Jetty - 9.4.46.v20220331

We are setting the proxy information before executing the clone request as follows.

             // this custom connection factory sets the proxy 
             CustomJdkHttpConnectionFactory connFactory = new CustomJdkHttpConnectionFactory();

            command.setTransportConfigCallback(transport -> {
               if (transport instanceof TransportHttp) {
                  
                  TransportHttp transportHttp = (TransportHttp) transport;
                  Map<String, String> headers = new HashMap<>();
                  
                  String authHeader = new String(Base64.getEncoder().encode(new String("admin" + ":" + "admin").getBytes()));
                  headers.put("Authorization", "Basic " + authHeader);
                  headers.put("Proxy-Authorization", "Basic " + authHeader);
                  transportHttp.setAdditionalHeaders(headers);
                  transportHttp.setHttpConnectionFactory(connFactory);
                 
               }
            });

Our embedded jetty based proxy server looks like this

public void startHTTPProxyServer() throws Exception {
      server = new Server();

      ServerConnector connector = new ServerConnector(server);
      String hostName = InetAddress.getLocalHost().getCanonicalHostName();
      connector.setHost(hostName);
      connector.setPort(PORT);

      server.setConnectors(new Connector[] { connector });
      // this custom connect handler will check the headers and execute the authorization check
      ConnectHandler proxy = new CustomConnectHandler();
      server.setHandler(proxy);

      server.start();

   }

The Custom connect handler is just enumerating the headers for now. We will add the validation logic once the correct headers are found.

public class CustomConnectHandler extends ConnectHandler {


   protected boolean handleAuthentication(HttpServletRequest request, HttpServletResponse response, String address) {
      logger.info("Request received: " + request.getMethod() + " " + request.getRequestURI());
      Enumeration<String> requestHeaderNames = request.getHeaderNames();
      while (requestHeaderNames.hasMoreElements()) {
         String headerName = requestHeaderNames.nextElement();
         String headerValue = request.getHeader(headerName);
         logger.info("Request Header : " + headerName + " = " + headerValue);
      }
      return false;
   }

}

The problem we are facing is, the authorization headers that we are passing is not getting propagated to our embedded jetty based proxy server. I understand since we are trying to connect to HTTPS repo, the client actually sends the CONNECT request and apparently the additional headers are dropped. But then how can we validate the incoming authorization unless the headers are present?

Any workaround to get these additional headers needed for authorization?

Update: 18th Jan

Based on @Joakim's answer I updated the code to add the header in the CONNECT request but that doesnt seem to work. Authenticator.setDefault(authenticator);

As can be seen from the screenshot, the authenticator is looked up during the call to the end destination and not during the initial CONNECT request to the proxy.

Answer 1

The CONNECT request itself can have authentication, and that's the purpose of overriding the Jetty 12 method on org.eclipse.jetty.server.handler.ConnectHandler with the signature ...

protected boolean handleAuthentication(Request request, Response response, String address)

Once that is approved (you have validated the authentication of the Request and returned true) the CONNECT request is responded to the User Agent (Web Client), and then the tunneled connection is established. If that connection uses TLS, then you cannot see what this tunneled connection contains (the whole point of TLS)

When working with a CONNECT Proxy from an HTTP Client, you have 2 requests.

The first request is talking to the Proxy itself.

This is the CONNECT request, that request can have Authentication. That authentication is strictly for the CONNECT request and the proxy server.

Once the CONNECT response is received by the HTTP Client, and it specifies that the CONNECT is allowed/approved, only then can the HTTP Client initiate the actual request to the destination server.

If you put your authentication headers on the request you initiate on the HTTP Client, then those headers are sent to the destination server, not the CONNECT proxy.

You have to use the Proxy features of your HTTP Client to set the authentication headers on the CONNECT Proxy request in order for the CONNECT proxy to see those headers.

Since your version of jgit uses the old school java.net.HttpURLConnection behaviors (not the newer java.net.http.HttpClient) you are stuck with the unusual behaviors that the old codebase has.

Here's an example of how this is used in the older java.net.HttpURLConnection.

URL url = new URL("http://destination.example.org");

Authenticator authenticator = new Authenticator() {
    @Override
    protected PasswordAuthentication getPasswordAuthentication() {
        // This uses BASIC auth, which is possibly disabled in your JDK
        // See "jdk.http.auth.tunneling.disabledSchemes" system property for details
        return new PasswordAuthentication("joyjit", "password".toCharArray());
    }
};

Authenticator.setDefault(authenticator);

Proxy proxy = new Proxy(Proxy.Type.HTTP,
    new InetSocketAddress("proxy.yoursite.net", 8080));

HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection(proxy);