Can properties such as yarn.log-aggregation-enable and yarn.log-aggregation.retain-seconds applied on a per job basis? I would not like this to be enabled at a cluster-wide scale but only for a few tasks.
Can properties such as yarn.log-aggregation-enable and yarn.log-aggregation.retain-seconds applied on a per job basis? I would not like this to be enabled at a cluster-wide scale but only for a few tasks.
Currently, there is no way to aggregate logs for specific YARN applications. https://issues.apache.org/jira/browse/YARN-85 seems to have attempted to provide this feature but the issue is still unresolved.