Resolve HtmlCleaner issue of getting HTTP respond code 403

Question

I'm using html cleaner to get data from a website...but I keep getting this error.

Server returned HTTP response code: 403 for URL: http://www.groupon.com/browse/chicago?z=skip

I'm not sure what I do wrong because I've use the same code before and its work perfectly. is anyone able to help me please?.

Code is below:

public ArrayList ParseGrouponDeals(ArrayList arrayList) {
    try {
        CleanerProperties props = new CleanerProperties();

        props.setTranslateSpecialEntities(true);
        props.setTransResCharsToNCR(true);
        props.setOmitComments(true);

        TagNode root = new HtmlCleaner(props).clean(new URL("http://www.groupon.com/browse/chicago?z=skip"));

        //Get the Wrapper.
        Object[] objects = root.evaluateXPath("//*[@id=\"browse-deals\"]");
        TagNode dealWrapper = (TagNode) objects[0];

        //Get the childs
        TagNode[] todayDeals = dealWrapper.getElementsByAttValue("class", "deal-list-tile grid_5_third", true, true);
        System.out.println("++++ Groupon Deal Today: " + todayDeals.length + " deals");
        for (int i = 0; i < todayDeals.length; i++) {
            String link = String.format("http://www.groupon.com%s", todayDeals[i].findElementByAttValue("class", "deal-permalink", true, true).getAttributeByName("href").toString());
            arrayList.add(link);
        }
        return arrayList;
    } catch (Exception e) {
        System.out.println("Error parsing Groupon:" + e.getMessage());
        e.printStackTrace();
    }
    return null;
}

Answer 1

For me adding the 'User-Agent' solves the problem; use it like this snippet:

        final URL urlSB = new URL("http://www.groupon.com/browse/chicago?z=skip");
        final URLConnection urlConnection = urlSB.openConnection();
        urlConnection.addRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0");
        urlConnection.connect();
        final HtmlCleaner cleaner = new HtmlCleaner();
        final CleanerProperties props = cleaner.getProperties();
        props.setNamespacesAware(false);
        final TagNode tagNodeRoot = cleaner.clean(urlConnection.getInputStream());