Solr custom UpdateRequestProcessorFactory fails with "Error Instantiating UpdateRequestProcessorFactory"

Question

I have a custom class extending UpdateRequestProcessorFactory doing some work on a document when it gets added to the index. This was working fine in v4.10.3 in standalone Solr. I moved to SolrCloud v5.2 and it throws this error when adding the Collection (node):

ERROR - 2015-06-14 12:25:11.071; [   docs_shard1_replica1] org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: Error CREATEing SolrCore 'docs_shard1_replica1': Unable to create core [docs_shard1_replica1] Caused by: class com.example.solr.update.processor.SelfTaggerUpdateProcessorFactory
    at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:661)
    at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:213)
    at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
    at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:646)
    at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:417)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
    at org.eclipse.jetty.server.Server.handle(Server.java:497)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
    at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: Unable to create core [docs_shard1_replica1]
    at org.apache.solr.core.CoreContainer.create(CoreContainer.java:651)
    at org.apache.solr.core.CoreContainer.create(CoreContainer.java:611)
    at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:628)
    ... 27 more
Caused by: org.apache.solr.common.SolrException: Error Instantiating UpdateRequestProcessorFactory, com.example.solr.update.processor.SelfTaggerUpdateProcessorFactory failed to instantiate org.apache.solr.update.processor.UpdateRequestProcessorFactory
    at org.apache.solr.core.SolrCore.<init>(SolrCore.java:815)
    at org.apache.solr.core.SolrCore.<init>(SolrCore.java:658)
    at org.apache.solr.core.CoreContainer.create(CoreContainer.java:637)
    ... 29 more
Caused by: org.apache.solr.common.SolrException: Error Instantiating UpdateRequestProcessorFactory, com.example.solr.update.processor.SelfTaggerUpdateProcessorFactory failed to instantiate org.apache.solr.update.processor.UpdateRequestProcessorFactory
    at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:587)
    at org.apache.solr.core.SolrCore.createInitInstance(SolrCore.java:622)
    at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:2281)
    at org.apache.solr.update.processor.UpdateRequestProcessorChain.init(UpdateRequestProcessorChain.java:126)
    at org.apache.solr.core.SolrCore.createInitInstance(SolrCore.java:624)
    at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:2265)
    at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:2259)
    at org.apache.solr.core.SolrCore.loadUpdateProcessorChains(SolrCore.java:1069)
    at org.apache.solr.core.SolrCore.<init>(SolrCore.java:766)
    ... 31 more
Caused by: java.lang.ClassCastException: class com.example.solr.update.processor.SelfTaggerUpdateProcessorFactory
    at java.lang.Class.asSubclass(Class.java:3208)
    at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:475)
    at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:422)
    at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:566)
    ... 39 more

In solrconfig.xml I have configured like this:

<updateRequestProcessorChain>
     <processor class="com.example.solr.update.processor.SelfTaggerUpdateProcessorFactory" lib="custom-libs" version="1">
        <arr name="source">
            <str>title</str>
            <str>desc</str>
            <str>subject</str>
            <str>content</str>
        </arr>
        <str name="dest">category</str>
     </processor>
     ...
</updateRequestProcessorChain>

I have tried to add my custom jar in two ways:

1. Add it to the server/lib/ folder where all the Jetty jars are located. This fails with Error Instantiating UpdateRequestProcessorFactory
2. Add it to the .system collection as suggested and then make it available to the <processor /> using the lib and version attributes, which fails with a Caused by: java.lang.ClassNotFoundException: com.example.solr.update.processor.SelfTaggerUpdateProcessorFactory

Does this Error Instantiating UpdateRequestProcessorFactory exception tell something I don't see? Does the move from standalone to Solrcloud require something for my custom code I missed?

Answer 1

The issue is a classloader issue. I had added my custom jar into the server/lib/ folder.

When I added the Collection, it would instantiate my custom class which needs the UpdateRequestProcessorFactory class but is not available in that classloader.

I solved this by rmoving my jar from server/lib/ and adding this line to solrconfig.xml:

<lib dir="${solr.install.dir:../../..}/custom-libs/" regex="solr-.*\.jar" />

and putting my custom jar in a custom-libs/ folder in ${SOLR_HOME}. This way I make sure that my custom classes are only loaded when the Collection is added, and not before, so they are in the correct classloader.

Answer 2

I think the issue is moving from standalone Solr to a Cloud one. Specifically, about where you store your jar file with the custom component. It is probably not visible to all the nodes and therefore fails.

However, Solr 5.2 does have new functionality related to that you may want to use. It allows to use the Config API to add the library, assuming you enable that functionality.