I am trying to build a Movie Recommender System Using Apache Spark MLlib.
I have written a code for recommender in java and its working fine when run using spark-submit
command.
My run command looks like this
bin/spark-submit --jars /opt/poc/spark-1.3.1-bin-hadoop2.6/mllib/spark-mllib_2.10-1.0.0.jar --class "com.recommender.MovieLensALSExtended" --master local[4] /home/sarvesh/Desktop/spark-test/recommender.jar /home/sarvesh/Desktop/spark-test/ml-latest-small/ratings.csv /home/sarvesh/Desktop/spark-test/ml-latest-small/movies.csv
Now I want to use my recommender in real world scenario, as a web application in which I can query recommender to give some result.
I want to build a Spring MVC web application which can interact with Apache Spark Context and give me results when asked.
My question is that how I can build an application which interacts with Apache Spark which is running on a cluster. So that when a request comes to controller it should take user query and fetch the same result as the spark-submit
command outputs on console.
As far as I have searched, I found that we can use Spark SQL, integrate with JDBC. But I did not find any good example.
Thanks in advance.
For isolating the user sessions and showing the results in an isolated manner, you may need to use queues with a binded user identity. Incase the the results takes time, with this identity you can show the respective results to the user.