Creation of data repository association of Lustre Scratch 2 file system to S3 via Terraform AWS provider

Question

I can create Amazon FSx Lustre SCRATCH2 file system with a data repository association (DRA) with an S3 bucket in one action if I do this via AWS Management Console. It also works if I first create a SCRATCH2 and then add an S3 DRA to it. Via Terraform AWS provider I try to create an FSx Lustre Scratch 2 file system and an S3 bucket and link them via creation of a DRA. I get an error message:

Error: creating FSx for Lustre Data Repository Association: UnsupportedOperation: This file system does not support data repository associations.

In Amazon FSx Lustre documentation I cannot find anything about inability to add a DRA later. There's only a notice about inability to use DRAs with Scratch 1. See for example: https://docs.aws.amazon.com/fsx/latest/LustreGuide/overview-dra-data-repo.html https://docs.aws.amazon.com/fsx/latest/LustreGuide/create-dra-linked-data-repo.html

Looks like Amazon FSx Lustre documentation is incomplete on DRAs and Scratch 2. Or maybe I simple have not found the right place.

Terraform AWS provider documentation says it is only PERSISTENT_2 that DRA resource (aws_fsx_data_repository_association) is compatible with. https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fsx_data_repository_association

Which seems strange as it's possible to create Scratch 2 with S3 DRA via AWS Management Console and the DRA import/export works fine with such Scratch 2 file systems.

Seems that at least Terraform AWS provider documentation doesn't contradict what the provider can do.

I use Terraform 1.1.5 and AWS Provider 5.26.0 (checked on 4.65.0 as well) but I could upgrade Terraform to the latest if it might help.

My Terraform resources I have already tried are:

resource "aws_fsx_lustre_file_system" "smpl_lstr_scr2" {
  deployment_type       = "SCRATCH_2"
  storage_capacity      = 1200
  data_compression_type = null

  subnet_ids = [ var.mysubnet ]
}

resource "aws_s3_bucket" "a_buck" {
  bucket = "my_buck"
}

resource "aws_fsx_data_repository_association" "scr2_s3_dra" {
  file_system_id       = aws_fsx_lustre_file_system.a_scr2.id
  data_repository_path = s3://${aws_s3_bucket.a_buck.id}"
  file_system_path     = "/"

  s3 {
    auto_export_policy {
      events = ["NEW", "CHANGED", "DELETED"]
    }
    auto_import_policy {
      events = ["NEW", "CHANGED", "DELETED"]
    }
  }
}

Is there a way to mimic AWS Management Console Scratch 2 file system with a DRA with an S3 bucket behaviour in Terraform to create an FSx Lustre Scratch 2 file system and its association to an S3 bucket? Or any other Terraform way to create Scratch 2 to S3 DRAs?

Answer 1

I have found that Terraform AWS Provider 5.26.0 creates SCRATCH 2 FSx for Lustre file systems with Lustre version 2.10 by default. It explains why I get the error message when using Terraform. Amazon FSx for Lustre documentation explicitly tells it's impossible to use DRAs with 2.10. https://docs.aws.amazon.com/fsx/latest/LustreGuide/overview-dra-data-repo.html https://docs.aws.amazon.com/fsx/latest/LustreGuide/create-dra-linked-data-repo.html

Correct SCRATCH 2 FSx for Lustre resource to use DRA is:

resource "aws_fsx_lustre_file_system" "smpl_lstr_scr2" {
  file_system_type_version = "2.15"

  deployment_type       = "SCRATCH_2"
  storage_capacity      = 1200
  data_compression_type = null

  subnet_ids = [ var.mysubnet ]
}