How to append data through for loops

134 views Asked by At

I am trying to recursively collect data through loops. I wrote this command and it works for collecting information from 2 pages. For example,

library(jsonlite)   
data1 <- fromJSON("https://www.example.com/?page=1", flatten = TRUE)
data2 <- fromJSON("https://www.example.com/?page=2", flatten = TRUE)
filings<- rbind.pages(list(data1, data2))

I was wondering if i can do this operation recursively for 300 pages. Let me know any suggestions.

library(jsonlite)
for (i in 1:300) {
datai <- fromJSON("https://www.example.com/?page=i", flatten = TRUE)
}
filings<- rbind.pages(list(data[1:300]))
2

There are 2 answers

2
keegan On BEST ANSWER

To use the looping variable in a string, you'll need to use this rather unpleasant combination of eval, parse and paste. Here's a simple example

for (i in 1:10){
    eval(parse(text=paste(
    'print ("iteration number ',i,'")'
    ,sep='')))
}

Your example is probably something like

for (i in 1:300) {
eval(parse(text=paste(
'data_',i, '<- fromJSON("https://www.example.com/?page=',i,'", flatten = TRUE)'
,sep='')))
}
0
Cron Merdek On

Loop will be much slower then apply. See lapply, sapply. paste0 just concatenate two strings.

GetJSON = function(id_)
{
    return(fromJSON(paste0("https://www.example.com/?page=", id_), flatten = TRUE))
}
lapply(1:300, GetJSON)