I'm new in Java and I'd like to know how to parse data from a webpage (let's say the resulting titles of a specific research in Google). I tried to implement the following async task just to see if I could retrieve the data:
private class DownloadTask extends AsyncTask<String, Void, Void> {
protected Void doInBackground(String... urls) {
HtmlCleaner parser = new HtmlCleaner();
URL url;
try{
url = new URL("https://www.google.com/search?q=java&rct=j&gws_rd=cr&ei=VsWJVcycOMaisAH4wISABA");
URLConnection conn = url.openConnection();
DocumentBuilderFactory factory=DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse(conn.getInputStream());
XPathFactory xfactory = XPathFactory.newInstance();
XPath xpath = xfactory.newXPath();
XPathExpression expr = xpath.compile("//*[@id=\"rso\"]/div[2]/li[1]/div/h3/a");
Object result = expr.evaluate(doc, XPathConstants.NODESET);
NodeList nodes = (NodeList) result;
for (int i = 0; i < nodes.getLength(); i++) {
System.out.println(nodes.item(i).getNodeValue());
Log.d(TAG,nodes.item(i).getNodeValue());
}
}
catch (SAXException e) {
Log.d(TAG, "\necc1");
e.printStackTrace();
} catch (UnsupportedEncodingException e) {
Log.d(TAG, "\necc2");
e.printStackTrace();
} catch (IOException e) {
Log.d(TAG, "\necc3");
e.printStackTrace();
} catch (ParserConfigurationException e) {
Log.d(TAG, "\necc4");
e.printStackTrace();
} catch (XPathExpressionException e) {
Log.d(TAG, "\necc5");
e.printStackTrace();
}
return null;
}
}
But I keep having SAXParseExcpetion like this:
org.xml.sax.SAXParseException: Unexpected <! (position:START_DOCUMENT null@1:1 in java.io.InputStreamReader@425a7e80)
Referring to the line: Document doc = builder.parse(conn.getInputStream());
.
I suppose that there could be errors in the document creation, but I don't really know how to handle it.