SAXParseException in Google scraping

132 views Asked by At

I'm new in Java and I'd like to know how to parse data from a webpage (let's say the resulting titles of a specific research in Google). I tried to implement the following async task just to see if I could retrieve the data:

private class DownloadTask extends AsyncTask<String, Void, Void> {
        protected Void doInBackground(String... urls) {
            HtmlCleaner parser = new HtmlCleaner();
            URL url;
            try{
                url = new URL("https://www.google.com/search?q=java&rct=j&gws_rd=cr&ei=VsWJVcycOMaisAH4wISABA");
                URLConnection conn = url.openConnection();
                DocumentBuilderFactory factory=DocumentBuilderFactory.newInstance();
                DocumentBuilder builder = factory.newDocumentBuilder();
                Document doc = builder.parse(conn.getInputStream());
                XPathFactory xfactory = XPathFactory.newInstance();
                XPath xpath = xfactory.newXPath();
                XPathExpression expr = xpath.compile("//*[@id=\"rso\"]/div[2]/li[1]/div/h3/a");

                Object result = expr.evaluate(doc, XPathConstants.NODESET);
                NodeList nodes = (NodeList) result;
                for (int i = 0; i < nodes.getLength(); i++) {
                    System.out.println(nodes.item(i).getNodeValue());
                    Log.d(TAG,nodes.item(i).getNodeValue());
                }
            }
            catch (SAXException e) {
                Log.d(TAG, "\necc1");
                e.printStackTrace();
            } catch (UnsupportedEncodingException e) {
                Log.d(TAG, "\necc2");
                e.printStackTrace();
            } catch (IOException e) {
                Log.d(TAG, "\necc3");
                e.printStackTrace();
            } catch (ParserConfigurationException e) {
                Log.d(TAG, "\necc4");
                e.printStackTrace();
            } catch (XPathExpressionException e) {
                Log.d(TAG, "\necc5");
                e.printStackTrace();
            }
        return null;
        }
    }

But I keep having SAXParseExcpetion like this:

org.xml.sax.SAXParseException: Unexpected <! (position:START_DOCUMENT null@1:1 in java.io.InputStreamReader@425a7e80)

Referring to the line: Document doc = builder.parse(conn.getInputStream());. I suppose that there could be errors in the document creation, but I don't really know how to handle it.

0

There are 0 answers