Camel SQL - Individual Insert to Batch

Question

I am using Camel SQL. I am trying to gather data from Oracle and insert it into Snowflakes. Below is the code snippet.

from(sql:select*...?dataSource=#oracleDataSource&repeatcount=1&batch=true)
.filter(ex -> filterDataMethod(ex))
.process(ex -> addHeadersForInsert(ex))
to(sql:insert into tableName (loadDate,col1 , col2 , col3) values (:#col1 , :#col2 , :#col3 , :#col4 )?dataSource=#snowflakesDataSource&repeatcount=1)

I tried adding batch=true in the From and To URIs but the code is not running in batch, rather it is inserting only single rows. Please let me know if there are any options to convert this to batch so that single data insertion can be avoided.

Note: I had gone through few posts related to this, but could not find anything related to this.

I followed instructions to use Iterator as mentioned. It created duplicate records.

Answer 1

After trying things found out the following Batch = true in route will not enable Batch mode.

1. Data needs to be batched out by manual process. I used aggregator and returned the list of Data from aggregator. Used the same sample as mentioned in camel page. Route :-

   from() .aggregate(constant(1), new ArrayListAggregationStrategy()).completionSize(batchsize you prefer).completionTimeout(500) .to()

Aggregation Class class ArrayListAggregationStrategy implements AggregationStrategy {

    public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
        Object newBody = newExchange.getIn().getBody();
        ArrayList<Object> list = null;
        if (oldExchange == null) {
            list = new ArrayList<Object>();
            list.add(newBody);
            newExchange.getIn().setBody(list);
            return newExchange;
        } else {
            list = oldExchange.getIn().getBody(ArrayList.class);
            list.add(newBody);
            return oldExchange;
        }
    }
}

2. Once the data is returned as list , had another processor which would return the data as HashMap<> of iterators which is set in the body. Refer the Iterator Snippet

With this process, when batch = true is enabled in route, the data would be inserted in Batch.