Export objects in global and function environment to a cluster

638 views Asked by At

I am using foreach and doParallel on below sample code.

library(foreach)
  library(doParallel)
  clusters =  makeCluster(4)
  registerDoParallel(clusters)


  fun1 <- function(param1, param2, param3)
  {
     param4 = param1+param2
     param5 = param2+param3
     param6 = param3+param1
     print(ls(envir = environment()))
     print(ls(envir = .GlobalEnv))
     # clusterExport(cl = clusters, varlist = ls(), envir = environment())
     clusterExport(cl = clusters, varlist = ls(), envir = .GlobalEnv)
     mmm = foreach(i = 1:length(param1), .combine = rbind) %dopar% fun2(i, param4, param5, param6)
     print(mmm)
  }

  fun2 <- function(i, param4, param5, param6)
  {
  j = param4[i] * param5[i] * param6[i]
  }

  param1 = 1:10
  param2 = 2:11
  param3 = 3:12
  fun1(param1, param2, param3)

In clusterExport's envir argument, when I use envir = .GlobalEnv, I get the error

**"Error in get(name, envir = envir) : object 'param4' not found"**.

And when I use envir = environment(), error is

**"Error in fun2(i, param4, param5, param6) : task 1 failed - "could not find function "fun2"""**

My question is how can I export all objects in global as well as function environment to a cluster.

1

There are 1 answers

0
F. Privé On

When you are in a function, you don't need to export manually the objects you need. But you have to pass fun2 to fun1 if you want it to be accessed by the foreach in fun1. So, just do:

fun1 <- function(param1, param2, param3, fun2)
{
  param4 = param1+param2
  param5 = param2+param3
  param6 = param3+param1
  mmm = foreach(i = 1:length(param1), .combine = rbind) %dopar% {
    fun2(i, param4, param5, param6)
  } 
  print(mmm)
}

fun1(param1, param2, param3, fun2)