Cloud watch agent metrics for AWS ASG getting grouped event after instance deletion?

Question

Background:

I am creating monitoring alarms for my AWS auto-scaling group. I have created the CPU utilization alarm using Namespace: "AWS/EC2" and MetricName: "CPUUtilization", which is a built-in metric and alarm created on this metric works fine. However, for RAM(memory) utilizations I have installed Cloud Watch agent on the EC2 instances using the launch template. and using Namespace: "CWAgent" and MetricName: "mem_used_percent".

Issue

My issue is that cloud watch agent metrics stays in cloud watch metrics even after auto scaling group terminates the instances and since my alarms are on the basis the dimension of auto scaling group, I do not get the metrics because it tries to group the metrics even for the instances that have been deleted already. Below screenshot would help you to understand better. I know we cannot delete cloud watch metrics. However is there a way to ask the cloud watch alarms to only group the metrics for currently running instances? Or any other possible solutions? Thank you for your time and help.

Cloud Watch agent metrics

RAM/Memory Alarm

CPU Utilization Alarm

amazon-cloudwatch-agent.json

{
  "metrics": {
    "aggregation_dimensions": {
      "AutoScalingGroupName": "\${aws:AutoScalingGroupName}"
    },
    "metrics_collected": {
        "disk":{  
        "resources":[  
          "/"
        ],
        "measurement":[  
          "used"
        ],
        "metrics_collection_interval": 60
      },
      "mem": {
        "measurement": [
          "mem_used_percent"
        ],
        "metrics_collection_interval": 60
      }
    }
  },
  "logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/var/log/messages",
            "log_group_name": "$log_group_name",
            "log_stream_name": "$log_stream_name",
            "timezone": "UTC"
          },
          {
            "file_path": "/var/www/logs/app.log",
            "log_group_name": "$app_log_group_name",
            "log_stream_name": "$app_log_stream_name",
            "timezone": "UTC"
          }
        ]
      }
    },
    "log_stream_name": "cloudwatch-agent",
    "force_flush_interval": 5
  }
}