Output list of two rbinded data frames with foreach in R

4.9k views Asked by At

Let's say I want to use foreach in the doParallel package to return a list of two data frames of different dimensions like the following:

a<-NULL
b<-NULL
for(i in 1:100){
  a<-rbind(a,data.frame(input=i,output=i/2))
  if(i > 5){
    b<-rbind(b,data.frame(input=i,output=i^2))
  }
}
list(a,b)

Sinceforeachreturns an object, there's no (at least to me) obvious way to do the above with foreach.

NOTE: this is a much simplified version of the problem I'm actually working with so solving the problem by using lapply (or something along those lines) won't work. The spirit of my question is how to do this with foreach.

2

There are 2 answers

2
NewNameStat On BEST ANSWER

I figured it out. You have to define your own function that combines the lists in exactly the way you want.

#takes an arbitrary number of lists x all of which much have the same structure    
comb <- function(x, ...) {  
      mapply(rbind,x,...,SIMPLIFY=FALSE)
}

foreach(i=1:10, .combine='comb') %dopar% {
      a<-rbind(a,data.frame(input=i,output=i/2))
      if(i > 5){
        b<-rbind(b,data.frame(input=i,output=i^2))
      }
      list(a,b)
}
0
Adrian Antico On

Adding a data.table rbindlist version to NewNameStat's answer:

#takes an arbitrary number of lists x all of which much have the same structure    
comb <- function(x, ...) {  
      mapply(rbind,x,...,SIMPLIFY=FALSE)
}

foreach(i=1:10, 
        .combine=function(x,...) mapply(function(...) data.table::rbindlist(list(...), fill = TRUE),x,...,SIMPLIFY=FALSE)) 
      %dopar% {
      a<-rbindlist(list(a,data.table(input=i,output=i/2)))
      if(i > 5){
        b<-rbindlist(list(b,data.table(input=i,output=i^2)))
      }
      list(a,b)
}