Write multiple groups/batch with header and trailer using spring batch

25 views Asked by At

I want to generate a file with financial transactions similar to the following format. Basically transactions are grouped by merchant and currency. Each group has a separate header and trailer.

FILE_HEADER
       MERCHANT_AND_CURRENCY_HEADER
         TRANSACTION_WITH_AMOUNT_AND_SAME_CURRENCY_INDICATED_IN_HEADER
       MERCHANT_FOOTER_WITH_COUNTER_AND_TOTAL_AMOUNT_OF_TRANSACTIONS_FOR_CURRENT_CURRENCY
       MERCHANT_AND_CURRENCY_HEADER
         TRANSACTION_WITH_AMOUNT_AND_SAME_CURRENCY_INDICATED_IN_HEADER
       MERCHANT_FOOTER_WITH_COUNTER_AND_TOTAL_AMOUNT_OF_TRANSACTIONS_FOR_CURRENT_CURRENCY
       MERCHANT_AND_CURRENCY_HEADER
         TRANSACTION_WITH_AMOUNT_AND_SAME_CURRENCY_INDICATED_IN_HEADER
       MERCHANT_FOOTER_WITH_COUNTER_AND_TOTAL_AMOUNT_OF_TRANSACTIONS_FOR_CURRENT_CURRENCY
FILE_FOOTER

what I do now is, load transactions from data base using following ItemReader

@Bean
    @StepScope
    public RepositoryItemReader<EvryTransactionFileEod> transactionRepositoryReader(EvryTransactionFileEodRepository evryTransactionFileEodRepository
            , @Value("#{jobParameters[" + PARAM_CUTOFF_DATE_TIME + "]}") LocalDateTime cutoffDate
            , @Value("#{stepExecution.jobExecution.id}") long batchId){
        try {
            return new RepositoryItemReaderBuilder<EvryTransactionFileEod>()
                    .methodName("findByBatchId").name("transactionFileReader")
                    .arguments(Arrays.asList(batchId))
                    .sorts(Map.of("merchantId", Sort.Direction.ASC,
                            "transactionCurrencyCode", Sort.Direction.ASC, "trxnNo", Sort.Direction.ASC))
                    .repository(evryTransactionFileEodRepository)
                    .pageSize(FG_CHUNK_SIZE)
                    .build();
        }catch (Exception e) {
            log.error(e.getMessage(), e);
            throw e;
        }
    }

then in the ItemProcessor put items into a list that have same merchant and currency. if the current item is different return a 'EodFileGroup' as follows

@Override
    public EodFileGroup process(EvryTransactionFileEod item) {
        try {

            TransactionDto transactionDto = new TransactionDto();
            transactionDto.setMsgType(item.getMsgType());
            ...
            ...
            transactionDto.setP63(item.getP63());
            incrementCountsInContext(item);
            if(transactionDtoList.isEmpty()){
                merchantCurrency = transactionDto.getMerchantId()+transactionDto.getTransactionCurrencyCode();
                transactionDtoList.add(transactionDto);
                return null;
            }
            else if((item.getMerchantId()+item.getTransactionCurrencyCode()).equals(merchantCurrency)){
                transactionDtoList.add(transactionDto);
                return null;
            }
            else
            {
                EodFileGroup eodFileGroup = new EodFileGroup();
                eodFileGroup.setTransactions(List.copyOf(transactionDtoList));
                eodFileGroup.setHeader("---H---");
                eodFileGroup.setTrailer("---T---");
                transactionDtoList.clear();
                //add new merchantCurrency dto s
                merchantCurrency = item.getMerchantId()+item.getTransactionCurrencyCode();
                transactionDtoList.add(transactionDto);
                return eodFileGroup;
            }
        }catch (Exception e) {
            log.error(e.getMessage(), e);
            throw e;
        }
    }

EodFileGroup

public class EodFileGroup {
    private String header;
    private List<TransactionDto> transactions;
    private String trailer;
}

ItemWriter expect EodFileGroup objects and write to the file

 @Bean
    @StepScope
    public ItemWriter<EodFileGroup> eodFileDtoItemWriter(
            @Value("#{jobParameters[" + PARAM_CURRENT_DATE_TIME + "]}") LocalDateTime currentDate){
        try{
            String folderPath = FileUtils.getUserDirectoryPath();
            String dateTime = GeneralUtils.formatDate("yyyyMMdd_HHmmss", currentDate);
            String fileName = fileNamePrefix + dateTime;

            GroupItemWriter groupItemWriter = new GroupItemWriter(folderPath + File.separator + fileName);
return groupItemWriter;
        }catch (Exception e) {
            log.error(e.getMessage(), e);
            throw e;
        }
    }

write method in GroupItemWriter

 @Override
    public void write(Chunk<? extends EodFileGroup> groups) throws Exception {
        try (Writer writer = new BufferedWriter(new FileWriter(outputFile))) {
            for (EodFileGroup group : groups) {
                // Write the header to the file
                writer.write(group.getHeader());

                // Write the transactions to the file
                for (TransactionDto transaction : group.getTransactions()) {
                    writeTransaction(writer, transaction);
                }

                // Write the trailer to the file
                writer.write(group.getTrailer());
                //writeTrailer(writer);
            }
        } catch (IOException e) {
            throw new Exception("Error writing to the output file", e);
        }
    }

Is there a better approach than this to achieve my requirement using spring batch? Also n the above solution I have a problem of identifying the last element at the process method of ItemProcessor in order to return the EodFileGroup object.

0

There are 0 answers