Extract a remote zip file and unzip it to a hdfs in java

Question

What i'm doing is just unzip and upload a zip file ,which can be download from a website, on to a hdfs. And here's the code:

String src="http://corpus.byu.edu/wikitext-samples/text.zip";
String dst = "hdfs://cshadoop1/user/hxy162130/assignment1";
InputStream a = new URL(src).openStream();
System.out.println(a == null);
ZipInputStream in = new ZipInputStream(a);
System.out.println(in == null);
ZipEntry zE = in.getNextEntry();        
System.out.println(zE == null);

As you see, I used openStream method to change the url into inputstream, and then use the inputstream as a parameter of the ZipInputStream.Finally i get an entry from the zipinputStream. But the problem is the getNextEntry method returns a null value, which means the output of my code is false,false,true. And i just can't find where the problem is.

Answer 1

The HTTP request to http://corpus.byu.edu/wikitext-samples/text.zip results in an 301 Moved Permanently giving new Location: https://corpus.byu.edu/wikitext-samples/text.zip. So there is not a ZIP resource avaliable any more using this URL.

To follow redirections you could do:

import java.net.URL;
import java.net.URLConnection;
import java.io.InputStream;
import java.util.zip.*;


class ReadZipInputStream {

 public static void main(String[] args) throws Exception {

  String src="http://corpus.byu.edu/wikitext-samples/text.zip";
  //301 Moved Permanently: Location:https://corpus.byu.edu/wikitext-samples/text.zip

  URL url = new URL(src);
  URLConnection connection = url.openConnection();
  String redirect = connection.getHeaderField("Location");
  if (redirect != null){
   connection = new URL(redirect).openConnection();
  }

  InputStream a = connection.getInputStream();
  System.out.println(a);

  ZipInputStream in = new ZipInputStream(a);
  System.out.println(in);

  ZipEntry zE = in.getNextEntry();        
  System.out.println(zE);

 }
}