Performing Parallel computation using doSNOW and multiple servers in R

643 views Asked by At

I am trying to do a multi-server (not multi-core) computation using doSNOW and foreach packages. I have 2 Windows servers and I want to start a parallel computation on both of these Windows machines.

I have the following code:

library(foreach)
library(doSNOW)

winOptionsServer1 <-
  list(host="Server1",
       rscript="C:/Program Files/R/R-3.1.2/bin/Rscript.exe",
       snowlib="C:/Program Files/R/R-3.1.2/library")

winOptionsServer2 <-
  list(host="Server2",
       rscript="C:/Program Files/R/R-3.1.2/bin/Rscript.exe",
       snowlib="C:/Program Files/R/R-3.1.2/library")


cl <- makeCluster(c(rep(winOptionsServer1, 2), rep(winOptionsServer1, 2)), type="SOCK")

After calling makeCluster my machine does somrthing, but never actually completes the call. When I hit Stop in RStudio I get the following error message:

running command 'ssh -l mypc Server1 C:/PROGRA~1/R/R-31~1.2/bin/Rscript.exe "C:/Program Files/R/R-3.1.2/library/snow/RSOCKnode.R" MASTER=MY-PC PORT=11764 OUT=/dev/null SNOWLIB=C:/Program Files/R/R-3.1.2/library' had status 127 

Does it mean that I have to configure something on these remote servers? What exactly should I configure? ssh? And how do I do it? Maybe I should open some ports on my remote machines? Which ones?

Does anyone have a full example of steps I need to do to run R across 2 or more machines.

P.S. doSnow works really well with multi-core running, no problem with that. I have problems with multi-servers running

0

There are 0 answers