Parallel erroring out with `withCallingHandlers`

116 views Asked by At

I have a parallel process that looks like this:

library(foreach)
library(doSNOW)

cl = makeCluster(parallel::detectCores() - 1, type = "SOCK", outfile = "out.txt") 
registerDoSNOW(cl)

l = foreach(i = 1:100) %dopar% { 
  res = withCallingHandlers(
    read.csv("somefilethatdoesntexist.csv"), error = function(e) e)
  if(inherits(res, "error")) res = NULL
  res
}

My expectation is that even if there is an error in "expression" the loop should continue, but it exits with an error and the resulting "l" variable is not created.

This seems to happen especially related to missing files. But if I'm wrapping it in a tryCatch and handling appropriately within "expression" how can it error out?

1

There are 1 answers

3
Roland On BEST ANSWER

Maybe this (adapted from here):

library(foreach)
library(doSNOW)

cl = makeCluster(parallel::detectCores() - 1, type = "SOCK", outfile = "out.txt") 
registerDoSNOW(cl)

l = foreach(i = 1:2) %dopar% { 
  withCallingHandlers({
      res <- withRestarts( read.csv("somefilethatdoesntexist.csv"),
                          skipError=function() return(NULL))
    },
    error=function(e) {saveRDS(e, paste0("E:/temp/", i, ".rds")); invokeRestart("skipError")})
  res
}

l
#[[1]]
#NULL
#
#[[2]]
#NULL

e <- readRDS("E:/temp/1.rds")
e
#<simpleError in file(file, "rt"): cannot open the connection>