Using Cobrix with Java spark program to parse cobol like file

Question

Has anyone tried running Cobrix in java spark program? I seem to be running into some class/version mismatch issue while calling Cobol datasource. I have a sample app at in github

when I run spark-submit like below,

spark-submit spark-cobol_2.11-2.6.9-bundle.jar --class com.test.cobol.App --master local target/spark-cobol-jay-1.0-SNAPSHOT.jar

I get following errors,

23/12/27 16:45:50 INFO DefaultSource: Cobrix 'spark-cobol' build 2.6.9 (2023-10-16T10:52:47) 
Exception in thread "main" java.lang.NoClassDefFoundError: scala/Product$class
    at za.co.absa.cobrix.cobol.parser.recordformats.RecordFormat$FixedLength$.<init>(RecordFormat.scala:22)

Answer 1

. was getting into a version mismatch between the spark and scala versions. The build file I downloaded used 2.11 scala version, while the rest of my program used 2.12. I have modified the Readme on how to fix this issue at my github page at https://github.com/jaysara/spark-cobol-jay