I am trying to run an example Java/Spark code from IntelliJ Idea on MacBook pro.
My java:
12.0.2
Also, I have run:
mvn dependency:copy-dependencies
in the folder with pom.xml.
It works well.
My code:
SparkConf conf = new SparkConf()
.setAppName("Word Count").setMaster("local");
JavaSparkContext sc = new JavaSparkContext(conf);
JavaDoubleRDD doubleRDD = sc.parallelizeDoubles(Arrays.asList(1.0, 1.0, 2.0, 3.0, 5.0, 8.0));
JavaRDD<String> stringRDD = sc.parallelize(Arrays.asList("Hello", "World"));
JavaPairRDD<String, Double> cartesian = stringRDD.cartesian(doubleRDD);
cartesian.first(); // error!
I got error:
java.lang.IllegalArgumentException
When I debugged into this, I found:
the source code does not match the bytecode.
I am not sure why I got this. Did I do something wrong ? or I missed something to set up something for Spark ?
thanks