Number of cores used doing parallel programming in R

Question

I use the following code in order to run my code in different cores at the same time.

I use 7 workers and as you can see I have to run my code for 90 (number of rows of grid_models) different simulation designs.

While 94% of the 90 designs have been completed, I have more than one remaining. However, I see that only one core is used, and so the remaining six cores are not used. This happens when I have some remaining desings (~6% of 90). For the 94% of the the 90 designs all the cores are used. Do you know why? How can I fix this?

In other words, for the last remaining of the 90 designs, even if more than one designs remain, only one core used.

Here is the code:

#   Libraries                                                              

packages <- c(
  "doFuture",
  "parallel",
  "doParallel",
  "foreach",
  "future",
  "progressr"
)

# Install packages not yet installed
installed_packages <- packages %in% rownames(installed.packages())
if (any(installed_packages == FALSE)) {
  install.packages(packages[!installed_packages])
}

# Packages loading
invisible(lapply(packages, library, character.only = TRUE))


# Models Specification

grid_models <- expand.grid(n_obs = c(1:5), cor_mat = c(1:3), beta = c(1:2), active_var = c(1:3))

vasilis <- list()

# Register the parallel backend
plan(multisession, workers = 7)

# Run the loop in parallel
opts <- list(progress = progress,
             packages = c("glmnet"),
             seed = TRUE)
options(future.globals.maxSize= 891289600)


# Parallel Code
with_progress({
  Progress <- progressor(along = 1:nrow(grid_models))
system.time( vasilis <- foreach(J = 1:nrow(grid_models),.options.future = opts) %dofuture% {
      
# some code used here that ends creating a matrix named Results

Results

Progress()

})
  })