Httpfs Create or append file with httpclient

426 views Asked by At

I use HTTPCLIENT to create or append file with rest component HTTPFS.

exemple cmd curl - working with curl

curl -i -X PUT -s --negotiate -u : "http://httpfsServer:14000/webhdfs/v1/user/a_app_kpi/tmp/testAppend.txt?op=CREATE&data=true" --header "Content-Type:application/octet-stream" --header "Transfer-Encoding:chunked" -T testAppend.txt

return query
HTTP/1.1 100 Continue

HTTP/1.1 401 Unauthorized
Server: Apache-Coyote/1.1
WWW-Authenticate: Negotiate
Set-Cookie: hadoop.auth=; Path=/; HttpOnly
Content-Type: text/html;charset=utf-8
Content-Length: 997
Date: Tue, 21 Nov 2017 16:07:07 GMT

HTTP/1.1 100 Continue

HTTP/1.1 201 Created
Server: Apache-Coyote/1.1
Set-Cookie: hadoop.auth="u=a_app_kpi&p=a_app_kpi@Environnement&t=kerberos-dt&e=1511316427865&s=JIJTCZS31D1wphVORe3ygZ1NBuo="; Path=/; HttpOnly
Content-Type: application/json
Content-Length: 0
Date: Tue, 21 Nov 2017 16:07:07 GMT

But when i execute request, the program return ERROR com.vhl.action.WebHdfsAction - null

my code with create :

CloseableHttpClient httpClient = cnxHttp.getHttpClient();
File file = new File(beanActionHttp.getPathLocal());
InputStreamEntity reqEntity = new InputStreamEntity(
                            new FileInputStream(file), -1, ContentType.APPLICATION_OCTET_STREAM);
                    reqEntity.setChunked(true);

HttpUriRequest request = RequestBuilder
                        .put(action_http)
                        .setEntity(reqEntity)
                        .build();

request.addHeader(HttpHeaders.TRANSFER_ENCODING, "chunked");
request.addHeader(HttpHeaders.CONTENT_TYPE, "application/octet-stream");
// HERE EXECUTE RETURN NULL
CloseableHttpResponse response = httpClient.execute(request);
1

There are 1 answers

0
V.HL On

I find my trouble. First i change method execute request to catch a wonderful error - thanks error

CloseableHttpResponse response = httpClient.execute(request); to HttpResponse response = httpClient.execute(request);

Error catching 
    org.apache.http.client.ClientProtocolException
            at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
            at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
            at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
            at com.vhl.action.WebHdfsAction$4.run(WebHdfsAction.java:399)
            at com.vhl.action.WebHdfsAction$4.run(WebHdfsAction.java:326)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:356)
            at com.vhl.action.WebHdfsAction.doCreateAppend(WebHdfsAction.java:326)
            at com.vhl.action.WebHdfsAction.appendFileHdfsHttpfs(WebHdfsAction.java:93)
            at com.vhl.httpclient.HttpClient.main(HttpClient.java:74)
    Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity.

So my trouble it's my request Cannot retry request with a non-repeatable request entity.

I use this follow to solve my trouble Cannot retry request with a non-repeatable request entity

I add my Entity a buffer "BufferedHttpEntity"

Finally the code working :

CloseableHttpClient httpClient = cnxHttp.getHttpClient();
File file = new File(beanActionHttp.getPathLocal());
InputStreamEntity reqEntity = new InputStreamEntity(
                            new FileInputStream(file), -1, ContentType.APPLICATION_OCTET_STREAM);
                    reqEntity.setChunked(true);

BufferedHttpEntity reqEntityBuf = new BufferedHttpEntity(reqEntity);

HttpUriRequest request = RequestBuilder
                        .put(action_http)
                        .setEntity(reqEntityBuf)
                        .build();


// HERE EXECUTE RETURN NULL
CloseableHttpResponse response = httpClient.execute(request);