I am facing an issue while creating an elastic search index by passing a list of custom object. Following is the bean which I populate based on certain conditions -
public class DataBean {
private String request_id;
private String clicks;
private String impressions;
private String conversion;
public String getRequest_id() {
return request_id;
}
public void setRequest_id(String request_id) {
this.request_id = request_id;
}
public String getClicks() {
return clicks;
}
public void setClicks(String clicks) {
this.clicks = clicks;
}
public String getImpressions() {
return impressions;
}
public void setImpressions(String impressions) {
this.impressions = impressions;
}
public String getConversion() {
return conversion;
}
public void setConversion(String conversion) {
this.conversion = conversion;
}
@Override
public String toString() {
return "DataBean [ clicks=" + clicks + ", impressions=" + impressions + ", conversion=" + conversion + ", request_id=" + request_id + "]";
}
}
and following is the gist of code which loops over the records and populates the above bean and stores it in a list which is used to create an index -
....
if (result.isSucceeded()) {
final List<DataBean> dataBeanList = new ArrayList<>();
final List<SearchResult.Hit<DataBean, Object>> hits = result.getHits(DataBean.class, Object.class);
for (SearchResult.Hit<DataBean, Object> hit : hits) {
if (!dataBeanList.stream().anyMatch(
dataBean -> Objects.equals(dataBean.getRequest_id(), hit.source.getRequest_id()))) {
DataBean source = hit.source;
dataBeanList.add(source);
}
}
MetricAggregation aggregations = result.getAggregations();
TermsAggregation termsAggregation = aggregations.getTermsAggregation("request_id");
for (TermsAggregation.Entry entry : termsAggregation.getBuckets()) {
for (DataBean dataBean : dataBeanList) {
if (entry.getKey().equalsIgnoreCase(dataBean.getRequest_id())) {
Double impression = entry.getSumAggregation("imp").getSum();
Double click = entry.getSumAggregation("click").getSum();
Double action = entry.getSumAggregation("action").getSum();
dataBean.setImpressions(impression.toString());
dataBean.setClicks(click.toString());
dataBean.setConversion(action.toString());
break;
}
}
}
Index index = new Index.Builder(new Gson().toJson(dataBeanList)).index(destinationIndex).type("hour")
.build();
try {
client.execute(index);
} catch (IOException ioException) {
LOGGER.debug("Failed to execute the search request on indices");
throw new JestClientExecutionException("Failed to execute the search request on indices - ",
ioException);
}
}
And following is the index template file -
{
"order": 0,
"template": "hour_*",
"settings": {
},
"mappings": {
"hour": {
"properties": {
"conversion": {
"index": "not_analyzed",
"store": true,
"type": "integer"
},
"impressions": {
"index": "not_analyzed",
"store": true,
"type": "integer"
},
"clicks": {
"index": "not_analyzed",
"store": true,
"type": "integer"
},
"request_id": {
"analyzer": "keyword",
"index": "not_analyzed",
"store": true,
"type": "text"
}
}
}
},
"aliases": {}
}
Now when I try to execute this code, it gives we following error -
[2017-09-04T06:10:29,365][INFO ][o.e.c.m.MetaDataCreateIndexService] [node-1] [hour_11_2017-09-04] creating index, cause [auto(bulk api)], templates [hour], shards [5]/[1], mappings [hour]
[2017-09-04T06:10:30,622][DEBUG][o.e.a.b.TransportShardBulkAction] [node-1] [hour_11_2017-09-04][1] failed to execute bulk item (index) BulkShardRequest [[hour_11_2017-09-04][1]] containing [index {[hour_11_2017-09-04][hour][AV5LgwJct35uj5JLu_X5], source[_na_]}]
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
at org.elasticsearch.index.mapper.DocumentParser.wrapInMapperParsingException(DocumentParser.java:176) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:69) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:277) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:529) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:506) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.action.bulk.TransportShardBulkAction.prepareIndexOperationOnPrimary(TransportShardBulkAction.java:450) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequestOnPrimary(TransportShardBulkAction.java:458) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:143) [elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:113) [elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:69) [elasticsearch-5.5.1.jar:5.5.1]
I tried replacing list of objects with a map to figure out whether list is a problem. I observed that map works but list doesn't. So the problem is with the list of objects but I am unable to figure out how to solve this. If anyone has any insights please let me know.