How do I get InputStream from HttpClient5 response?

Question

In existing project I have an HTTP service that fetches data using Apache's HttpClient 4 and returns response InputStream like in this code example:

public class HttpClient4Demo {
    public static void main(String[] args) throws IOException {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            try (Scanner scanner = new Scanner(fetchData(httpClient))) {
                while (scanner.hasNextLine()) {
                    System.out.println(scanner.nextLine());
                }
            }
        }
    }

    private static InputStream fetchData(HttpClient httpClient) throws IOException {
        HttpResponse response = httpClient.execute(new HttpGet(
                "https://dummyjson.com/products/1"
        ));
        return response.getEntity().getContent();
    }
}

I want to migrate that service to Apache's HttpClient 5, so I have to rewrite the code:

public class HttpClient5Demo {
    public static void main(String[] args) throws IOException {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            try (Scanner scanner = new Scanner(fetchData(httpClient))) {
                while (scanner.hasNextLine()) {
                    System.out.println(scanner.nextLine());
                }
            }
        }
    }

    private static InputStream fetchData(HttpClient httpClient) throws IOException {
        ClassicHttpResponse response = httpClient.executeOpen(
                null,
                new HttpGet("https://dummyjson.com/products/1"),
                null
        );
        return response.getEntity().getContent();
    }
}

But that code gives me a warning because ClassicHttpResponse is used without try-with-resources.

If I wrap ClassicHttpResponse with try-with-resources, I'll get a closed InputStream, so I won't be able to read response data.

private static InputStream fetchData(HttpClient httpClient) throws IOException {
    try (ClassicHttpResponse response = httpClient.executeOpen(
            null,
            new HttpGet("https://dummyjson.com/products/1"),
            null
    )) {
        return response.getEntity().getContent();
    }
}

I can additionally wrap response.getEntity() into ByteArrayInputStream, and I will get a valid InputStream, but it's not a perfect solution, since the whole response data will be stored in RAM.

private static InputStream fetchData(HttpClient httpClient) throws IOException {
    try (ClassicHttpResponse response = httpClient.executeOpen(
            null,
            new HttpGet("https://dummyjson.com/products/1"),
            null
    )) {
        return new ByteArrayInputStream(EntityUtils.toByteArray(response.getEntity()));
    }
}

So is there any way to get response InputStream without closing it and/or storing it in RAM using HttpClient 5?

Answer 1

Simple way

To get rid of the warning, return ClassicHttpResponse instead of the InputStream, and put it in the try-with-resources clause as below.

public class HttpClient5Demo {
    public static void main(String[] args) throws IOException {
        try (CloseableHttpClient httpClient = HttpClients.createDefault();
             ClassicHttpResponse response = fetchData(httpClient);
             Scanner scanner = new Scanner(response.getEntity().getContent())) {
            while (scanner.hasNextLine()) {
                System.out.println(scanner.nextLine());
            }
        }
    }

    private static ClassicHttpResponse fetchData(HttpClient httpClient) throws IOException {
        return httpClient.executeOpen(null, new HttpGet("https://dummyjson.com/products/1"), null);
    }
}

Since closing the ClassicHttpResponse is equivalent to closing the InputStream, if you can ensure the InputStream is properly closed after returning, the warning could be ignored also.

You may feel a bit uncomfortable, that's why it comes to the recommended way.

Recommended way

Checking HttpClient#executeOpen, which is actually calling a deprecated method HttpClient#execute.

It is strongly recommended to use execute methods with HttpClientResponseHandler such as execute(HttpHost, ClassicHttpRequest, HttpClientResponseHandler) in order to ensure automatic resource deallocation by the client. For special cases one can still use executeOpen(HttpHost, ClassicHttpRequest, HttpContext) to keep the response object open after the request execution.

We can refer to example demonstrates the recommended way of processing the HTTP response, and the processing will be something like below

private static void printData(HttpClient httpClient) throws IOException {
    httpClient.execute(new HttpGet("https://dummyjson.com/products/1"), (response) -> {
        try (Scanner scanner = new Scanner(response.getEntity().getContent())) {
            while (scanner.hasNextLine()) {
                System.out.println(scanner.nextLine());
            }
        }
        return null;
    });
}

return null looks a bit weird here, but in actual scenario, we usually map the response to another object and return it.

Reference: Migration to Apache HttpClient 5.x classic APIs