Why doesn't apache java http library handle sites with Content-Encoding: none?

2.1k views Asked by At

I'm trying to send a GET via a proxy and some sites have the header: Content-Encoding: none, which causes Apache to throw an exception. I'm wondering if this is the intended behavior, and whether I should treat this as a bug or not:

Caused by: org.apache.http.HttpException: Unsupported Content-Coding: none
at org.apache.http.client.protocol.ResponseContentEncoding.process(ResponseContentEncoding.java:98)
at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:139)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:199)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:85)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
... 10 more

My code:

public CloseableHttpResponse getViaProxy(String url, String ip, int port, String username,
                                         String password) {
    CloseableHttpClient httpClient;
    if (username == null) {
        httpClient = HttpClients.custom()
                .setSSLSocketFactory(getCustomSslConnectionSocketFactory())
                .build();
    } else {
        CredentialsProvider credsProvider = new BasicCredentialsProvider();
        credsProvider.setCredentials(
                new AuthScope(ip, port),
                new UsernamePasswordCredentials(username, password));
        httpClient = HttpClients.custom()
                .setDefaultCredentialsProvider(credsProvider)
                .setSSLSocketFactory(getCustomSslConnectionSocketFactory())
                .build();
    }

    RequestConfig config = RequestConfig.custom()
            .setProxy(new HttpHost(ip, port))
            .build();
    HttpGet httpGet = new HttpGet(url);
    httpGet.setConfig(config);

    return httpClient.execute(httpGet);
}

The error I'm referring to is from here. It seems like this method only supports headers with Content-Encoding: gzip, deflate, or identity.

http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/client/protocol/ResponseContentEncoding.html

3

There are 3 answers

0
ok2c On BEST ANSWER

HTTP protocol specification defines gzip, compress, deflate and identity as valid content coding schemes. One should be using identity instead to signal that no content transformation is expected.

1
user3134554 On

If you only want to process the request with the Content-Encoding is none, you can use the custom method of HttpClent like so:

CloseableHttpClient httpClient = HttpClients.custom().disableContentCompression().build();

this code disable the two interceptors RequestAcceptEncoding and ResponseContentEncoding.

0
kazy On

There is a possibility to fix that issue on client-side with interceptors: http://adamscheller.com/java/httpexception-unsupported-content-coding-none-solution/