Linked Questions

Popular Questions

Worker log file gets mangled when log_directory is set

Asked by At

I used to have the worker logs as such:


So I wanted to have SLURMCluster writes the worker logs in a separate directory (as opposed to the current working dir), so I provided "log_directory" as an input argument as such.

from dask_jobqueue import SLURMCluster
c = SLURMCluster(queue_name, log_directory="dirA")

And the logs end up like this:


And all the worker logs are piped to the .err file while the .out file is empty.

What am I missing in the SLURMCluster configuration to set it correctly, such that the .out should contain the logs and the .err should contain worker errors.

Related Questions