Subentity SolrEntityProcessor stops working since SolR 5.x

Question

I use a data import like this

<dataConfig>
    <document name="products">
        <entity
            name="outer"
            dataSource="my_datasource"
            pk="id"
            query="..."
            deltaQuery="..."
            deltaImportQuery="..."
        >
            <entity
                name="solr"
                processor="SolrEntityProcessor"
                url="http://127.0.0.1:8983/solr/${solr.core.name}"
                query="Xid:${outer.Xid}"
                rows="1"
                fl="Id,FieldA,FieldB"
                wt="javabin"
            />
        </entity>
    </document>
</dataConfig>

The interesting part is the sub entity, which uses SolrEntityProcessor. Until (including) SoLR 4.10 everything worked fine, but since 5.1 it doesn't work anymore. It doesn't fail in the meaning, that it tells me, it failed, but it "successfully" stops during the import of the second document.

• DIH import one document
• DIH fetches the second (outer) entity
• DIH stops

In the logs the following Exception appears. It looks like DIH intentionally closes the connection of the SolrEntityProcessor and crashes as soon as it tries to fetch the sub entity for the second document.

java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
    at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270)
    at org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:444)
    at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:482)
    at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)
    at org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:363)
    at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:224)
    ... 3 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
    at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:62)
    at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:246)
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:514)
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
    ... 5 more
Caused by: java.lang.IllegalStateException: Connection pool shut down
    at org.apache.http.util.Asserts.check(Asserts.java:34)
    at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
    at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:217)
    at org.apache.http.impl.conn.PoolingClientConnectionManager.requestConnection(PoolingClientConnectionManager.java:184)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
    at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
    at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:466)
    at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
    at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
    at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
    at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
    at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
    at org.apache.solr.handler.dataimport.SolrEntityProcessor.doQuery(SolrEntityProcessor.java:198)
    at org.apache.solr.handler.dataimport.SolrEntityProcessor.buildIterator(SolrEntityProcessor.java:147)
    at org.apache.solr.handler.dataimport.SolrEntityProcessor.nextRow(SolrEntityProcessor.java:128)
    at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
    ... 8 more

Answer 1

This is a known bug in Solr. Even I have come across! I posted this as an answer because, this is a bug and there's no solution from the author! We have actually downgraded the version in order to get rid of this bug. I am not sure if this helps, but this is what we did at our end.

Update: This may or may not help you, but we resolved the issue by downgrading.