YARN log aggregation on a per job basis

75 views Asked by At

Can properties such as yarn.log-aggregation-enable and yarn.log-aggregation.retain-seconds applied on a per job basis? I would not like this to be enabled at a cluster-wide scale but only for a few tasks.

1

There are 1 answers

0
Sumit Nigam On BEST ANSWER

Currently, there is no way to aggregate logs for specific YARN applications. https://issues.apache.org/jira/browse/YARN-85 seems to have attempted to provide this feature but the issue is still unresolved.