I have the following fields:
- In the database I have the field
property_industry_sector
which is a list of comma separated ints, null or empty string. - In the Solr schema configuration I have the same field
property_industry_sector
of type int and multivalued.
My problem is that I have to handle to difference in the DataImportHandler
configuration, and my attempt looks like this:
<entity
name="property_industry_sector_extractor"
transformer="script:SplitIndustrySector"
query="
SELECT property_industry_sector
FROM job
WHERE job.id = ${job.id}
">
<field column="property_industry_sector" name="property_industry_sector" />
</entity>
Where the ScriptTransformer
has the following definition:
function SplitIndustrySector(row) {
//var logger = java.util.logging.Logger.getLogger("org.apache.solr");
if(row.get('property_industry_sector') !== null) {
if(false === row.get('property_industry_sector').isEmpty()) {
var pieces = row.get('property_industry_sector').split(',');
var arr = new java.util.ArrayList();
for(var i=0, len=pieces.length; i<len; i++) {
arr.add(new java.lang.Integer(pieces[i]));
}
row.put('property_industry_sector', arr);
return row;
}
}
var arr = new java.util.ArrayList();
arr.add(new java.lang.Integer(0));
row.put('property_industry_sector', arr);
return row;
}
The problem is with the general case, when the value is null or empty string, because no matter what the transformer does, I still get the following Exception
property_industry_sector=property_industry_sector(1.0)={[, 0]}}]
java.lang.NumberFormatException: For input string: ""
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:493)
at java.lang.Integer.parseInt(Integer.java:514)
at org.apache.solr.schema.TrieField.createField(TrieField.java:374)
at org.apache.solr.schema.SchemaField.createField(SchemaField.java:97)
at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:203)
at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:276)
at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60)
at org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:73)
at org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:294)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:631)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:267)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:186)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:353)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:411)
at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:392)
I do not understand where the empty string comes from (which it tries to convert to Integer) while also being confused by the values it tries to insert above the exception:
property_industry_sector=property_industry_sector(1.0)={[, 0]}}]
I've tried clearing the row prior to the put()
call. Return null
, or just as with the current example return the row with a single value of 0
.
Haven't found a way to work it out, but managed to solve the issue with an alternative solution. Instead of using the
ScriptTransformer
I was able to achieve the same goal with SQL transformations.