elasticsearch-py HTTPS_PROXY Variable Ignored - Corporate Proxy

Question

I tried to use this code to utilize the python requests library to connect through a corporate HTTP proxy to elasticsearch by specifying the HTTPS_PROXY environment variable, but it seems like when I try to connect over HTTPS the HTTPS_PROXY variable is getting ignored

from elasticsearch import Elasticsearch, RequestsHttpConnection
es = Elasticsearch([es_url], connection_class=RequestsHttpConnection)

I can't use HTTP because our elasticsearch is hosted on HTTPS over port 443

My expectation is that the library should attempt to connect through the HTTP proxy, however it seems like it is still attempting to connect to the host directly without going through the corporate proxy

Answer 1

• Basically to connect to Elasticsearch Remote server with the SSH (HTTPS) protection you will need SSL certificate file which is usually generated inside the Elasticsearch configuration folder like this: C:\elastic-stack\elasticsearch-8.7.1-windows-x86_64\elasticsearch-8.7.1\config\certs\http_ca.crt

• I don't know how it works with Python, but I can tell how it works with Java. But there is small issue, certificate signed by a Certificate Authority (CA) which is not recognized by the Java truststore.

• To resolve this issue, you need to add the Elasticsearch server's SSL certificate to the truststore used by Java application. To do so, write the following command in Terminal:

  keytool -import -alias http_ca -file C:Users\User\Downloads\http_ca.crt -keystore truststore.jks
  

• And finally Java client looks like this, which makes the connection with Elastic and prints out the relevant connection status:

  public class ElasticsearchConnectionCheckHTTPS {
  
    public static void main(String[] args) {
        String host = "ELASTIC-IP-ADDRESS";
        int port = ELASTIC-PORT;
        String username = "USERNAME";
        String password = "PASSWORD";
        String truststorePath = "\PATH\TO\truststore.jks";
        String truststorePassword = "TRUSTSTORE-PASSWORD";
  
        try {
            KeyStore truststore = KeyStore.getInstance("JKS");
            truststore.load(new FileInputStream(truststorePath), truststorePassword.toCharArray());
  
            TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
            trustManagerFactory.init(truststore);
  
            SSLContext sslContext = SSLContext.getInstance("TLS");
            sslContext.init(null, trustManagerFactory.getTrustManagers(), null);
  
            CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
            credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
  
            RestHighLevelClient client = new RestHighLevelClient(
                    RestClient.builder(new HttpHost(host, port, "https"))
                                    .setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder
                                    .setDefaultCredentialsProvider(credentialsProvider)
                                    .setSSLContext(sslContext)
                                    .setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)));
  
            MainResponse response = client.info(RequestOptions.DEFAULT);
            String clusterName = response.getClusterName().toString();
            String clusterVersion = response.getVersion().toString();
            System.out.println("Connected to Elasticsearch cluster: " + clusterName + " (Version: " + clusterVersion + ")");
  
            client.close();
        } catch (Exception e) {
            System.out.println("Failed to connect to Elasticsearch server. Exception: " + e.getMessage());
        }
    }
  }
  

• If you do everything correct, then you should see similar result like this: