How to import and save multiple data files with a for loop in R?

Question

I'm trying to import data for multiple rounds using the package "openair". This is the code I'm using

install.packages("devtools")
devtools::install_github("r-lib/conflicted")


#openair
library(openair)
library(tidyverse)

#import
sites_2005_2019 <- importMeta(
  source = "aurn",
  year = 2005:2019
)

 defra_aurn[[y]] <- for (y in 2005:2019) {
  
importAURN(year = y,
                         data_type = "daily",
                         site=sites_2005_2019$code,
                         meta = TRUE,

 save(defra_aurn[[y]], file = "C:/Users/..../defra_aurn.Rdata")                        
)
}

this gives the error:

Error in save(defra_aurn[[y]], file = "C:/Users/..../defra_aurn[[y]].Rdata") : 
  object ‘defra_aurn[[y]]’ not found

I've tried putting the save command outside the loop, inside the loop but outside the argument,but I nevertheless get error that object not found.

I cannot simply use 2005:2019 in the argument to import as that would only import sites that were open during the whole period, but I need all available sites in each year, so I think the loop is the only way to go forward.

Would appreciate any help, thanks!

I described the problem in the previous section. Hope that helps, I'd be happy to add more details if needed

adding more details here:

thanks this worked, but unable to save it in r and then write it to stata.

this code is giving error in the last line: 

Error in write.dta(defra_aurn, "defra_aurn.dta") : 
  The object "dataframe" must have class data.frame

To correct this I tried numerous options like

defra_aurn_combined <- do.call(rbind, defra_aurn)

defra_aurn_combined <- bind_rows(defra_aurn)

full_join()

and several variants of these.

I have added some details to the original post One challenge of combining these data frames is that number of sites vary over dates, so either I get errors that numbers of columns of arguments do not match, or the combined data has only sites common to all years, which is not what I'm looking for.

Answer 1

There's a couple of problems here. The save function can't access the object because it doesn't exist yet, and you cannot assign using y because y only exists in the loop.

What you want is lapply and to save afterwards. Here's a reduced example:

# openair
library(openair)
library(tidyverse)

# import
sites_2005_2019 <- importMeta(
  source = "aurn",
  year = 2005:2019
)

sites_2005_2019 <- head(sites_2005_2019, 3)

defra_aurn <- lapply(setNames(c(2005:2006), c(2005:2006)), function(y) {
  importAURN(
    year = y,
    data_type = "daily",
    site = sites_2005_2019$code,
    meta = TRUE
  )
})

save(defra_aurn, file = "defra_aurn.Rdata")

Note that I used setNames so that you get back a named list. Also, you only need to save once because you are saving the whole object (the named list), rather than each item in the list.

Edit: In the comments you say you want to then save this to read into stata. To do this you need a data.frame not a named list. One option would be to add this to the end:

full_dataset <- do.call(rbind, defra_aurn)
# full_dataset <- purrr::list_rbind(defra_aurn) # Or purrr alternative
foreign::write.dta(full_dataset, "full_dataset.dta")
save(defra_aurn, file = "defra_aurn.Rdata")