AWS Fault Injection Simulator returning 'Not authorized to perform the required action.'

Question

I am trying to use AWS FIS to stress CPU on an ubuntu VM thru ssm send command (aws:ssm:send-command/AWSFIS-Run-CPU-Stress).

When I ssh to the VM and use 'top' command I see the CPU shooting up successfully from 0.3% to 99.9% but surprisingly the FIS experiment ends up in 'failed' state with a tag:'Not authorized to perform the required action'.

Since the cpu definitely increased and I even see COMMAND name 'stress-ng-cpu' against the 99.9% cpu entry, any idea why it is still saying 'not authorized' and ending up in failed status?

Thanks.

Answer 1

It worked after I completely removed mine and applied the permissions as shown here: https://chaos-engineering.workshop.aws/en/030_basic_content/030_basic_experiment/10-permissions.html

Posting this for the benefit of others who might run into similar issues.

Answer 2

The issue was probably a missing permission in the role, most likely ssm:ListCommands or ssm:CancelCommand, which are required by the aws:ssm:send-command action (https://docs.aws.amazon.com/fis/latest/userguide/fis-actions-reference.html#ssm-actions-reference)

If you only have ssm:SendCommand permission, the action will be able to start (and inject the fault), but then it will fail either when checking the state of the running experiment, or when trying to stop the experiment.

The FIS experiment will end up in failed state, but the SSM document will keep running until the provided duration is reached.