How to get percentage progress from a BufferedReader?

Question

@Override
    protected StringBuilder doInBackground(String... url) {
        try {
            HttpClient client = new DefaultHttpClient();
            HttpGet request = new HttpGet(url[0]);
            HttpResponse response = client.execute(request);
            BufferedReader rd = new BufferedReader(new InputStreamReader(
                    response.getEntity().getContent()));
            long content = response.getEntity().getContentLength();
//Do not need 'String line'
            String line = "";
            StringBuilder htmlBuilder = new StringBuilder();
            long bytesRead = 0;
            while ((line = rd.readLine()) != null) {
                htmlBuilder.append(line);
                bytesRead = bytesRead + line.getBytes().length + 2;
                publishProgress(new Integer[]{(int) (((double) bytesRead / (double) content) * 100)});
            }
            return htmlBuilder;
        } catch (IOException e) {
            return null;
        }

    }

'long content' is returning a -1. This means that the content is larger than the max value of the long class.

To try and have it return something else I tried to extend DefaultHttpClient=>

public class ImprovedHttpClient extends DefaultHttpClient implements HttpClient {
    @Override
public final HttpResponse execute(HttpUriRequest request)
        throws IOException, ClientProtocolException {
    // TODO Auto-generated method stub
    return super.execute(request);
}

}

The above method is final so I cannot extend it and use polymorphism...I am stuck on how to get the percent progress and I do not want to use another library.

Why do I have to implement HttpClient? According to apache documentation, DefaultHttpClient already implements HttpClient; it makes no sense that I have to implement it. However, I can see how eclipse would prevent me from finding it because final methods cannot be extended.

In conclusion, How do I get the total bytes of content from the DefaultHttpClient without it returning -1?

EDIT This is printed from

System.out.println(Arrays.toString(response.getAllHeaders()));

Stackoverflow successfully gives me back the percentage progress using this url: ("http://stackoverflow.com/questions/20306245/how-to-get-percentage-progress-from-a-bufferedreader#20306306");

[Cache-Control: public, max-age=33, Content-Type: text/html; charset=utf-8, Expires: Sat, 30 Nov 2013 22:37:14 GMT, Last-Modified: Sat, 30 Nov 2013 22:36:14 GMT, Vary: *, X-Frame-Options: SAMEORIGIN, Date: Sat, 30 Nov 2013 22:36:41 GMT, Content-Length: 51468, Via: HTTP/1.1 127.0.0.1:2020 (Brazil/2.0), Server: Brazil/2.0, Connection: Keep-Alive]

Google Returns -1: https://www.google.com/

[Date: Sat, 30 Nov 2013 22:41:44 GMT, Expires: -1, Cache-Control: private, max-age=0, Content-Type: text/html; charset=ISO-8859-1, Set-Cookie: PREF=ID=4c028dc2b639abc2:FF=0:TM=1385851304:LM=1385851304:S=we-TWKR-LtJcomHN; expires=Mon, 30-Nov-2015 22:41:44 GMT; path=/; domain=.google.com, Set-Cookie: NID=67=mId7x6boTt6BYcAWipXzM4Fjt_WGl7KQPgcHdPkA9yiOgHf8pTWl5k38AfFnA68NjL34rDRYsveh-QdLKcyoAzDZRigGe_5ydrwiELRSkW24Q0J7NdtErEnT93WXMjgM; expires=Sun, 01-Jun-2014 22:41:44 GMT; path=/; domain=.google.com; HttpOnly, P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info.", Server: gws, X-XSS-Protection: 1; mode=block, X-Frame-Options: SAMEORIGIN, Alternate-Protocol: 443:quic, Transfer-Encoding: chunked]

Answer 1

There are some situations where an HTTP client can not know the size of the thing you are getting. The HTTP header has a place to declare the content length, but that is not required. In the situation where the server does not specify the content length, the HTTPEntity returns a -1 for length.

In this situation there is no way to report download progress accurately. You could make a "guess" at the content length. Or if there is significant processing after download, you could download the entire thing, and then report progress on the processing.

Check to see if the server is sending the content length for the case you are testing.