Async calls to CLI tool in separate folders

I think I found a simple way that works for me.

I set up the persisent daemons with mirai, ask for their pids once, assign them to my individual folders and then for each task I can ask the pid and assign it to the correct folder to call the tool from.

Something like this:

library(mirai)
#> Warning: package 'mirai' was built under R version 4.4.3

#set 2 workers
daemons(2)
#> [1] 2

#fn to get pids of workers
myfunc <- function(folder){
  
  pid <- Sys.getpid()
  
  res <- setNames(folder, pid)
  
  return(res)
  
}

#folders to call the tool from
folders <- c("dir1", "dir2")

#dictionary to assign worker pid to specific folder
dict <-
  mirai_map(folders, myfunc)[.flat]

dict
#>   7760  13868 
#> "dir1" "dir2"

#fn to perform task with variable computing times
mytask <- function(id, arg){
  
  pid <- as.character(Sys.getpid())
  folder <- dict[pid]
  
  Sys.sleep(arg)
  
  print(paste0("task", id, " done in folder ", folder))
  
}

#task IDs and corresponding arguments for mytask fn
#task1 will run 5 sec, others 1 sec only
tasks <-
  data.frame(id = 1:5, arg = c(5, rep(1, 4)))

#run all on 2 workers
#5 sec total run time, i.e. load balancing works
#folders correct as well
system.time(print(mirai_map(tasks, mytask, dict = dict)[]))
#> [[1]]
#> [1] "task1 done in folder dir1"
#> 
#> [[2]]
#> [1] "task2 done in folder dir2"
#> 
#> [[3]]
#> [1] "task3 done in folder dir2"
#> 
#> [[4]]
#> [1] "task4 done in folder dir2"
#> 
#> [[5]]
#> [1] "task5 done in folder dir2"
#>    user  system elapsed 
#>     0.0     0.0     5.1

Created on 2025-06-28 with reprex v2.1.1