Trying to use WinInet to upload a file to HDFS

72 views Asked by At

I am using WinInet to try and upload a file to Hadoop. I am doing everything I can think of correctly. IOW, I am following the REST API for WebHDFS including getting the re-direct, then sending a new request with the URI provided in the re-direct. When I make my second request, I can see from spying with Fiddler that it returns an HTTP status code of 201 like it should.

The second request goes something like:

  1. Do WinInet stuff necessary to call HttpOpenRequest() with a verb of PUT (succeeds)
  2. Call HttpSendRequestEx() with all NULLs except hRequest (succeeds)
  3. Write data to file using InternetWriteFile() (succeeds with the correct number of bytes written matching the same as I tried to write)
  4. Call HttpEndRequest() (succeeds)

Everything succeeds, and by all rights the data should be on the server. Instead, there is just an empty file. If I try the same thing using curl, it succeeds. If I spy on WinInet using Fiddle, I see everything succeeding except I never see the data being written to the server.

The headers being returned after I call HttpEndRequest() are:

HTTP/1.1 201 Created
Cache-Control: no-cache
Expires: Fri, 12 Jun 2015 00:22:57 GMT
Date: Fri, 12 Jun 2015 00:22:57 GMT
Pragma: no-cache
Expires: Fri, 12 Jun 2015 00:22:57 GMT
Date: Fri, 12 Jun 2015 00:22:57 GMT
Pragma: no-cache
Location: webhdfs://quickstart.cloudera:8020/user/test.txt
Content-Type: application/octet-stream
Content-Length: 0
Server: Jetty(6.1.26.cloudera.4)

My gut says there is something messed up in the InternetWriteFile() that I'm doing, but all the return codes and write values check out good.

Are there any tricks to doing a PUT with WinInet that I am missing?

0

There are 0 answers