MalformedByteSequenceException parsing RSS feed with ROME

Question

Im getting a MalformedByteSequenceException running a JSF application which makes use of the Rome Rss-parsing Lib:

[2017-08-31T15:54:01.241+0200] [glassfish 5.0] [SEVERE] [] [] [tid: _ThreadID=101 _ThreadName=Thread-9] [timeMillis: 1504187641241] [levelValue: 1000] [[
  com.sun.org.apache.xerces.internal.impl.io.MalformedByteSequenceException: Ungültiges Byte 1 von 1-Byte-UTF-8-Sequenz.
    at com.sun.org.apache.xerces.internal.impl.io.UTF8Reader.invalidByte(UTF8Reader.java:701)
    at com.sun.org.apache.xerces.internal.impl.io.UTF8Reader.read(UTF8Reader.java:567)
    at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(XMLEntityScanner.java:1793)
    at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipChar(XMLEntityScanner.java:1463)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2824)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
    at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)
    at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339)
    at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121)
    at main.java.tools.RssAggregator.searchAggregatedFeeds(RssAggregator.java:119)
    at main.java.jobs.ApplicationRunnable.lambda$execute$0(ApplicationRunnable.java:45)
    at java.lang.Iterable.forEach(Iterable.java:75)
    at main.java.jobs.ApplicationRunnable.execute(ApplicationRunnable.java:43)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
]]

The error is thrown in RssAggregator.searchAggregatedFeeds() line 119:

public List<Feed> searchAggregatedFeeds(String keyword) {
    keyword = keyword.toLowerCase();
    List<Feed> results = new LinkedList<>();
    Document document = null;
    try {
        DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
        DocumentBuilder builder = factory.newDocumentBuilder();

        //document = builder.parse(IOUtils.toInputStream(this.aggregatedRssString);
        InputStream stream = new ByteArrayInputStream(this.aggregatedRssString.getBytes());
        document = builder.parse(stream);  //TODO HERE We have the Problem
        stream.close();

        //Normalize the XML Structure; It's just too important !!
        document.getDocumentElement().normalize();

    } catch (ParserConfigurationException | SAXException | IOException e) {
        e.printStackTrace();
    }

    //Here comes the root node
    if (document != null) {

      //Element root = document.getDocumentElement();
        //Get all
        NodeList nList = document.getElementsByTagName("item");

        for (int i = 0; i < nList.getLength(); i++) {

            Node node = nList.item(i);
            Element ele = (Element) node;

            Feed feed = new Feed();
            feed.setDate(ele.getElementsByTagName("dc:date").item(0).getTextContent());
            feed.setDescription(ele.getElementsByTagName("description").item(0).getTextContent());
            feed.setLink(ele.getElementsByTagName("link").item(0).getTextContent());
            feed.setSubject(ele.getElementsByTagName("title").item(0).getTextContent());
            feed.setId(getIdCounter());  //set the id. This increments also

        /* now the searching ... */
            if (feed.getSubject().toLowerCase().contains(keyword) || feed.getDescription().toLowerCase().contains(keyword))
                results.add(feed);
        }
    }
    return results;
}

I can't figure out what should be wrong with the line document = builder.parse(stream); //TODO HERE We have the Problem ?

What strikes me here is that when testing it with a junit-test, all goes well:

@Test
public void searchTest() throws Exception {
    List<Feed> results = rssAggregator.searchAggregatedFeeds("a");
    System.out.println("Hits: " + results.size());

    List<Feed> results2 = rssAggregator.searchAggregatedFeeds("z");
    System.out.println("Hits: " + results2.size());

    List<Feed> results3 = rssAggregator.searchAggregatedFeeds("x");
    System.out.println("Hits: " + results3.size());

}

resulting as expected:

... while calling it from the cyclic job doesn't:

    List<Feed> allResults = new LinkedList<>();

    /* for each WORD search the aggregatedfeed for matches and add them to allResults */
    configBean.words.forEach(w -> {
        System.out.println("current searched for Word: " + w);
        List<Feed> wResults = aggregator.searchAggregatedFeeds(w);  // :(
        allResults.addAll(wResults);
    });

Question: why does the junit-test work, while the call from within the application doesn't? The aggregated feeds are exactly the same.

Answer 1

Is your rss data UTF-8 encoded? Calling the no parameter version of String.getBytes() will use platform's default charset and it might not be UTF-8. That is why your test might work on Linux but fail on Windows. Try String.getBytes(Charset charset) instead.