I have setup RSelenium for scrapping data from several pages. However, some pages do not exist anymore or are very slow to load, and after a while of sitting idle, Rselenium closes Chrome.
I found that one suggested solution to this in Selenium is to use detach=TRUE. I set such a command in Rselenium, but chrome still keeps closing in cases where there is a long wait.
Any ideas how to prevent Rselenium from closing Chrome?
eCaps <- list(
chromeOptions =
list(prefs = list("download.prompt_for_download" = FALSE,
"download.directory_upgrade" = TRUE,
"plugins.always_open_pdf_externally"= TRUE,
"download.default_directory" = URLFilesFolder,
"detach" = TRUE
)
)
)
rD <- rsDriver(browser = "chrome", port = free_port(), chromever = "latest", extraCapabilities = eCaps)
remDr <- rD$client