R Shiny run task/script in different process

Question

In my Shiny app users can generate heavy powerpoint report. When it contains a lot of slides it could take > 30 minutes to be done. And therefore I'd like to process those tasks in independent processes/tasks which could work even when app is closed - e.g. user clicks button to generate report, closes app and when report is ready app informs user by email. Are there any good practices or proven solutions to do this?

My first thought was using future package with plan(multisession) set - but I'm not sure what happens when user closes the app - future session closes too or not?

Answer 1

I was lucky enough to be at London EARL this week and I think one of the best presentations I saw there was about exactly this (by Joe Cheng). You would need the promises package for this to work and as it says on the documentation a special version of shiny devtools::install_github("rstudio/shiny@async") that supports asynchronous programming.

You can find a first documentation here on how this works by using dplyr and promises (future is also compatible).

As a small example (taken from the documentation), running an intensive calculation using the following:

read.csv.async("data.csv") %...>%
  filter(state == "NY") %...>%
  arrange(median_income) %...>%
  head(10) %...>%
  View()

would essentially return the console cursor back, allowing you to run any other command you want and would automatically open the View tab once this was finished. I might be able to dig out a shiny example in a bit, but keep in mind this is still under development and will be released before the end of the year (with a more comprehensive documentation I would imagine).

Answer 2

So I made some example workaround using future package. Code executes in separate session (cluster) even when app is closed. I think the next step is just to figure out how app should check if process is still running or is finished. Any ideas?

library(future)
cl <- parallel::makeCluster(2L)
plan(cluster, workers = cl)

server <- function(input, output) {
  observeEvent(input$run, {

    iteration <- as.numeric(input$iteration)
    path <- input$path

    future::future({
      writeLog <- function(n, path) {
        file.remove(path)
        for (i in 1:n) {
          cat("#", i, "-",  as.character(Sys.time()), "\n", file = path, append = TRUE)
          Sys.sleep(1)
        }
      }
      writeLog(iteration, path)
    }, globals = c("iteration", "path"))
  })
}

ui <- fluidPage(
  sidebarLayout(
    sidebarPanel(
      tags$div("This app writes to file in cluster which means it is computed in parallel to this session. 
               It will execute even when app is closed.")
      , br()
      , shiny::textInput("path", "Path to log file", value = "/src/dev/export_performance/future.log")
      , shiny::textInput("iteration", "Iteration number", value = 60)    
    ),
    mainPanel(
      br()
      , actionButton("run", "Run future")
    )
  )
)

shinyApp(ui = ui, server = server)