HttpClient 4.5.x multithread request error: has not been leased from this pool

Question

I am writing a test sample for httpclient of apache-httpclient to test the multithread request error.

I ran it in a for loop and create 2 threads in each loop to run the test cases with 2 different hosts setting to pooling manager.

In 10 loops, I always faced 1 error as : Exception in thread "Thread-2" java.lang.IllegalStateException: Entry [id:17][route:{}->http://***:8000][state:null] has not been leased from this pool at org.apache.http.util.Asserts.check(Asserts.java:46)

Does my code has any problem? What should I do?

The sample is below:

private void testHttpClient() {

        HttpHost proxy = new HttpHost("dev.host.com", 8001);
        final DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);

        final PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
        cm.setMaxTotal(20);
        cm.setDefaultMaxPerRoute(1);
        HttpHost localhost1 = new HttpHost("dev.test1.com", 8001);
        cm.setMaxPerRoute(new HttpRoute(localhost1), 2);
        HttpHost localhost2 = new HttpHost("dev.test2.com", 8000);
        cm.setMaxPerRoute(new HttpRoute(localhost2), 2);

        RequestConfig requestConfig = RequestConfig.custom()
            .setConnectTimeout(1000)
            .setSocketTimeout(1000)
            .setConnectionRequestTimeout(1000)
            .build();
        final CloseableHttpClient httpclient1 = HttpClients.custom()
            .setConnectionManager(cm)
            .setRoutePlanner(routePlanner)
            .setDefaultRequestConfig(requestConfig)
            .build();

        RequestConfig requestConfig2 = RequestConfig.custom()
            .setConnectTimeout(1000)
            .setSocketTimeout(1000)
            .setConnectionRequestTimeout(1000)
            .build();
        final CloseableHttpClient httpclient2 = HttpClients.custom()
            .setConnectionManager(cm)
            .setRoutePlanner(routePlanner)
            .setDefaultRequestConfig(requestConfig2)
            .build();

        class HttpClientThead1 implements Runnable {
          public void run() {
            for (int i = 1; i <= 1; i++) {
              System.out.println("HttpClientThead1 Start");
              HttpClientContext context = HttpClientContext.create();
              try {
                HttpGet httpget = new HttpGet("http://dev.test1.com:8001/test/id/10001");
                CloseableHttpResponse response = httpclient1.execute(httpget, context);
                long t = System.currentTimeMillis();
                System.out.println("HttpClientThead1 " + i + ":" + response.getEntity().toString() + " " + t);
                response.close();
              } catch (ClientProtocolException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
              } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
              } finally {
                cm.closeExpiredConnections();
                cm.closeIdleConnections(30, TimeUnit.SECONDS);
              }
            }
          }
        }

    class HttpClientThead2 implements Runnable {
      public void run() {
        for (int i = 1; i <= 1; i++) {
          System.out.println("HttpClientThead2 Start");
          HttpClientContext context = HttpClientContext.create();
          try {
            HttpGet httpget2 = new HttpGet("http://dev.test2.com:8000/test/id/10002");
            CloseableHttpResponse response2 = httpclient2.execute(httpget2, context);
            long t = System.currentTimeMillis();
            System.out.println("HttpClientThead2 " + i + ":" + response2.getEntity().toString() + " " + t);
            response2.close();
          } catch (ClientProtocolException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
          } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
          } finally {
            cm.closeExpiredConnections();
            cm.closeIdleConnections(30, TimeUnit.SECONDS);
          }
        }
      }
    }

    for (int j = 1; j <= 10; j++) {
      HttpClientThead1 t1 = new HttpClientThead1();
      HttpClientThead2 t2 = new HttpClientThead2();

      Thread thread1 = new Thread(t1);
      Thread thread2 = new Thread(t2);
      thread1.start();
      thread2.start();
    }
}

Answer 1

I got the answer, because I set a proxy in this httpclient. So I need to also set it into setMaxPerRoute with the proxy information.

So it should be: cm.setMaxPerRoute(new HttpRoute(localhost2, proxy), 10);

Answer 2

I believe you found a bug in HttpClient; I reported it here: https://issues.apache.org/jira/browse/HTTPCORE-634. It happens only when you are using a connection pool, but do not reuse connections.

Http connections are only reused when the entire response is consumed. I changed your code to read as follows:

(...)
            CloseableHttpResponse response2 = httpclient2.execute(httpget2, context);
            HttpEntity entity = response.getEntity();
            // consume the response
            EntityUtils.consumeQuietly(entity);
            response2.close();
(...)

and no failures since. So, either consume the responses, or do not use connection pooling, and you'll be fine.