How To get the values returned from Spark Job class into Spark Launcher class

334 views Asked by At

I am using the below piece of code to launch the Spark Job.

I want to get the result from com.test.Foo Class into the Launcher class which starts the spark application.

I don't want to write the values to a temporary file and read it from the Launcher Class. Is there a way to get the required values returned from the spark main class com.test.Foo .

Is there a way to set the values to current session and get it wherever required?

SparkAppHandle handler = new SparkLauncher()
                            .setAppResource(<path to jar>)
                            .setMaster("yarn-cluster") 
                            .setDeployMode("cluster")
                            .setVerbose(true)
                            .setMainClass("com.test.Foo")               
                            .addAppArgs(args[0],args[1])
                            .startApplication();
0

There are 0 answers