I am using a shared library written in C in my R code. I load the compiled shared library using dyn.load command. I am going to call a shared library function in a parallelized foreach loop. Here is my code:
library(foreach)
library(doParallel)
totalCores = detectCores()
cluster <- makeCluster(totalCores[1]-1)
registerDoParallel(cluster)
dyn.load("package.so")
run <- function(i) {
row <- data[i,]
res <- .Call("c_function", as.double(row))
return(res)
}
result <- foreach(i=1:nrow(data), .combine = rbind) %dopar% {
run(i)
}
I get the following error:
Error in { :
task 1 failed - "C symbol name "c_function" not in load table"
Although I have loaded the shared library, it seems c_function is not recognized in the parallel tasks. Of course when I let the dyn.load command in the foreach loop the problem is solved:
result <- foreach(i=1:nrow(data), .combine = rbind) %dopar% {
dyn.load("package.so")
run(i)
}
But I am not sure if this is the best practice since at each iteration the shared library (package.so) is loaded and it may be not efficient. Any ideas?
Edit:
Regarding to r2even's answer I tested the following code:
foreach(i=1:50,.packages='rootSolve') %dopar% {
print(is.loaded("c_function"))
}
My PC has 10 CPU cores (20 threads) so the value of totalCores variable was 20 when I executed this code. Here is the result:
[[1]]
[1] TRUE
[[2]]
[1] TRUE
[[3]]
[1] TRUE
[[4]]
[1] TRUE
[[5]]
[1] TRUE
[[6]]
[1] TRUE
[[7]]
[1] TRUE
[[8]]
[1] TRUE
[[9]]
[1] TRUE
[[10]]
[1] TRUE
[[11]]
[1] FALSE
[[12]]
[1] FALSE
[[13]]
[1] FALSE
[[14]]
[1] FALSE
[[15]]
[1] FALSE
[[16]]
[1] FALSE
[[17]]
[1] FALSE
[[18]]
[1] FALSE
[[19]]
[1] FALSE
[[20]]
[1] TRUE
[[21]]
[1] TRUE
[[22]]
[1] TRUE
[[23]]
[1] TRUE
[[24]]
[1] TRUE
[[25]]
[1] TRUE
[[26]]
[1] TRUE
[[27]]
[1] TRUE
[[28]]
[1] TRUE
[[29]]
[1] TRUE
[[30]]
[1] TRUE
[[31]]
[1] TRUE
[[32]]
[1] TRUE
[[33]]
[1] TRUE
[[34]]
[1] TRUE
[[35]]
[1] TRUE
[[36]]
[1] TRUE
[[37]]
[1] TRUE
[[38]]
[1] TRUE
[[39]]
[1] TRUE
[[40]]
[1] TRUE
[[41]]
[1] TRUE
[[42]]
[1] TRUE
[[43]]
[1] TRUE
[[44]]
[1] TRUE
[[45]]
[1] TRUE
[[46]]
[1] TRUE
[[47]]
[1] TRUE
[[48]]
[1] TRUE
[[49]]
[1] TRUE
[[50]]
[1] TRUE
It raises several questions. Is the value of is.loaded("c_function") always False only in the the iterations 11 to 20? Is it guaranteed that from the iteration 21 to the rest it is always true?
Try this *hack*, almost certainly less inefficient than repeated calls to
dyn.load:(Untested.)