How do I kill a YARN container to test failure scenarios

Question

I'm building an application on AWS EMR using YARN (and Dask) version Hadoop 2.7.3-amzn-1. I'm trying to test various failure scenarios and I'm wanting to simulate a container failure. I can't seem to find an easy way to kill a YARN container - only the whole application. Is there a command-line utility for this?

Answer 1

[root@node1 lillcol]# yarn container -help
20/04/24 15:04:14 INFO client.AHSProxy: Connecting to Application History server at node1/127.0.0.1:10200
usage: container
 -help                                     Displays help for all commands.
 -list <Application Attempt ID>            List containers for application
                                           attempt.
 -signal <container ID [signal command]>   Signal the container. The
                                           available signal commands are
                                           [OUTPUT_THREAD_DUMP,
                                           GRACEFUL_SHUTDOWN,
                                           FORCEFUL_SHUTDOWN] Default
                                           command is OUTPUT_THREAD_DUMP.
 -status <Container ID>                    Prints the status of the
                                           container.

Through the command yarn container -signal [container-ID] GRACEFUL_SHUTDOWN to achieve.

i've tried and int works,I hope that will be helpful.

Answer 2

Seems like it's exposed in API starting from version 2.8.0 https://hadoop.apache.org/docs/r2.8.0/api/org/apache/hadoop/yarn/client/api/YarnClient.html#signalToContainer(org.apache.hadoop.yarn.api.records.ContainerId,%20org.apache.hadoop.yarn.api.records.SignalContainerCommand)

Answer 3

YARN has no CLI or REST API that kills a container.

The simplest way to create a container failure is to login to a NodeManager host and kill the process (which would be a container) spawned by the NodeManager.