Install SparkR that comes with Spark 1.4

Question

The newest version of Spark (1.4) now comes with SparkR. Does anyone know how to go about installing the SparkR implementation on Windows? The sparkR.R script is currently located in C:/spark-1.4.0/R/pkgs/R/

This appears to be a step in the right direction, but the instructions don't work for Windows as there is no sparkR directory as it relates to.

Answer 1

@DavidArenburg put me on the right track.

Following the Windows documentation in the C:\spark-1.4.0\R\WINDOWS.md, I installed RTools and added R.exe and RTools to my computers PATH.

Then, I ran install-dev.bat in C:\spark-1.4.0\R This added the lib\SparkR\ installation that I was missing.

Then, from the command prompt, I ran

mklink /D "C:\Program Files\R\R-3.1.3\library\SparkR" "C:\spark-1.4.0\R\lib\SparkR"

This added a link in my R packages directory to the installation in the spark folder.

library(SparkR) # this should run now.

Answer 2

Frustrated with finding a solution myself, I put together a quick write up here, which addresses the install on Mac, but it shouldn't be far from implementation in Windows. Hope this helps.

Answer 3

For some reason I could not get it to work with the mklink.

So I very simply copy-pasted the SparkR folder from under my Spark installation (on my computer it's found at: C:\spark\spark\R\lib) to my R library folder (on my computer it's found at: C:\Program Files\R\R-3.1.1\library.

And now it's working!