Which permissions for a cross-accounts transfert across two s3 with aws-datasync

Question

I'm currently following the AWS official documentation tutorial for transferring a large volume of data between two S3 buckets, which can be found at this link: https://docs.aws.amazon.com/datasync/latest/userguide/tutorial_s3-s3-cross-account-transfer.html#s3-s3-cross-account-required-permissions-source-account

I'm attempting to adapt this tutorial for Terraform by using two separate AWS providers, one for the source account and another for the destination account. However, I've encountered an issue related to the permissions on the bucket. I'm not entirely comfortable with AWS permissions and roles, and this is where I'm facing a roadblock.

I need to add a role on the bucket :

{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Sid": "DataSyncCreateS3LocationAndTaskAccess",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::source-account:role/source-datasync-role"
      },
      "Action": [
        "s3:GetBucketLocation",
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads",
        "s3:AbortMultipartUpload",
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:ListMultipartUploadParts",
        "s3:PutObject",
        "s3:GetObjectTagging",
        "s3:PutObjectTagging"
      ],
      "Resource": [
        "arn:aws:s3:::destination-bucket",
        "arn:aws:s3:::destination-bucket/*"
      ]
    },
    {
      "Sid": "DataSyncCreateS3Location",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::source-account:role/source-user-role"
      },
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::destination-bucket"
    }
  ]
}

However, I'm uncertain about the source-user-role mentioned in the policy. I have tried using the role that I use as a user (I'm using an SSO login and connected as AWSReservedSSO_AWSAdministratorAccess_xxxxxx/myUserName). But when I apply my Terraform configuration, I encounter an AccessDenied error.

Here's a snippet of my Terraform configuration for reference:

 resource "aws_datasync_task" "example" {
  provider                 = aws.source
  destination_location_arn = aws_datasync_location_s3.destination.arn
  name                     = "example"
  source_location_arn      = aws_datasync_location_s3.source.arn
}

resource "aws_datasync_location_s3" "destination" {
  provider      = aws.source
  s3_bucket_arn = aws_s3_bucket.destination-bucket.arn
  subdirectory  = "/"

  s3_config {
    bucket_access_role_arn = aws_iam_role.datasync_source_role.arn
  }
}

resource "aws_datasync_location_s3" "source" {
  provider      = aws.source
  s3_bucket_arn = aws_s3_bucket.source-bucket.arn
  subdirectory  = "/"

  s3_config {
    bucket_access_role_arn = aws_iam_role.datasync_source_role.arn
  }
} 

Unfortunately, this configuration results in an

Message_: "DataSync bucket access test failed: could not perform s3:HeadBucket on bucket destination-bucket-test. Access denied. Ensure bucket access role has s3:ListBucket permission."

Any assistance or guidance on resolving this issue would be greatly appreciated.

Answer 1

I think on the high level you will need something like this for the S3 bucket policy on the Destination Account :

  name = "datasync-destination-bucket-policy"
  bucket = aws_s3_bucket.destination_bucket.id

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = ["s3:PutObject", "s3:ListBucket"],
        Effect = "Allow",
        Principal = {
          AWS = "arn:aws:iam::<source_account_id>:role/datasync-source-role"
        },
        Resource = [
          aws_s3_bucket.destination_bucket.arn,
          "${aws_s3_bucket.destination_bucket.arn}/*",
        ]
      }
    ]
  })
}

For the the Source Account you will need something like this :

resource "aws_s3_bucket_policy" "datasync_source_bucket_policy" {
  name = "datasync-source-bucket-policy"
  bucket = aws_s3_bucket.source_bucket.id

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = ["s3:GetObject", "s3:ListBucket"],
        Effect = "Allow",
        Principal = {
          AWS = "arn:aws:iam::<destination_account_id>:role/datasync-destination-role"
        },
        Resource = [
          aws_s3_bucket.source_bucket.arn,
          "${aws_s3_bucket.source_bucket.arn}/*",
        ]
      }
    ]
  })
}

Of Course don't forget to create the IAM Roles for each one Source --> Destination .

These permissions should be associated with the corresponding AWS DataSync roles and the S3 bucket policies in both the source and destination accounts. Make sure to replace <source_account_id> and <destination_account_id> with the actual AWS account IDs.