Terraform JSON requires an Azure subnet that is nested in an Azure Vnet Object to have an ID but will not allow it to be set

Question

TL;DR: JSON Terraform Azure Vnet requires nested subnets to have an ID but errors when provided an ID

Background

I have an Azure account that has a bunch of resources that were created out of band, but now the customer wants us to managed the resources all in terraform. Luckily terraform has a way of importing resources, and even a lovely tutorial the demonstrates how to import the resources and generate the necessary config (https://learn.hashicorp.com/tutorials/terraform/state-import).

Once I have imported the resources I run the command tf show -no-color -json > main.tf.json to generate the body of the config. I dump it out in JSON because once tf show is run you need to then modify some of the output in order to have the results to be a valid terraform configuration file.

After the tf show is run, I have a script that munges the data making some changes to the structure of the JSON and eliminating attributes that to be removed or changed.

The Problem

The problem that is faced is that after sanitizing the output generated by tf show when I run, tf validate or any other command that requires validation I get an error that says that the id attribute is required for the elements in the subnet JSON array within the azurerm_virtual_network JSON object. When I add the id though, I then get an error saying that an id cannot be specified for an Azure subnet JSON object.

Note importantly this only occurs when I am using a .tf.json file if I use a regular tf file no issue arises.

The Code

Terraform Version: Terraform v0.13.5

Provider Version: azurerm v2.38.0

Here is a terraform JSON config that demonstrates the issue

    {
        "resource": {
            "azurerm_virtual_network": {
                "gork_vnet": {
                    "address_space": [
                        "10.0.0.0/16"
                    ],
                    "location": "westus2",
                    "name": "gork_vnet",
                    "resource_group_name": "mork_rg",
                    "subnet": [
                        {
                            "address_prefix": "10.0.0.0/24",
                            "name": "cunning_brutality",
                            "security_group": ""
                        }
                    ]
                }
            }
        }
    }

Error Produced:

Error: Incorrect attribute value type

  on main.tf.json line 22, in [1].resource.azurerm_virtual_network.gork_vnet:
  22:                     "subnet": [
  23:                         {
  24:                             "address_prefix": "10.0.0.0/24",
  25:                             "name": "cunning_brutality",
  26:                             "security_group": ""
  27:                         }
  28:                     ]

Inappropriate value for attribute "subnet": element 0: attribute "id" is
required.

If I add the id I get this error

Error: "subnet.0.id": this field cannot be set

  on main.tf.json line 30, in [1].resource.azurerm_virtual_network.gork_vnet:
  30:                 }

Additionally it should be noted that according to the TF docs, the JSON should actually be structured as

                    "subnet": [
                        {
                            "cunning_brutality": {
                                "address_prefix": "10.0.0.0/24",
                                "name": "cunning_brutality",
                                "security_group": ""
                            }
                        }
                    ]

But that generates the error

Inappropriate value for attribute "subnet": element 0: attributes
"address_prefix", "id", "name", and "security_group" are required.

Conclusion

Unfortunately my code spelunking hasn't turned up anything particularly enlightening, according to all the documentation this should work. So any direction or guidance would be much appreciated.

Answer 1

Unless the configuration will be maintained by automated systems on an ongoing basis, I'd caution that someone asking for something to be maintained with Terraform is probably expecting Terraform's native syntax, rather than JSON, because that syntax is often considered easier to read and maintain by humans in the future.

With that said, if you do intend to use JSON then there are various rules for how the alternative JSON syntax corresponds with the native syntax, because JSON has fewer constructs and therefore a particular JSON structure can be interpreted as one of a number of different native syntax structures depending on the context.

Unfortunately in this case it seems like the documentation for azurerm_virtual_network is incorrect, or at least incomplete. From referring to the implementation of that resource type I can see that this subnet argument has the special legacy "Attributes as Blocks" mode turned on, which forces Terraform to interpret it in a different way that is backwards compatible with some unexpected patterns from older Terraform versions.

Because you are using JSON syntax, the subsection Attributes as Blocks In JSON Syntax is relevant in your case. We can see there that attributes with this legacy processing mode are represented in JSON using the JSON expression mapping rules rather than the block mapping rules, which means that each object in your JSON array must be valid value of the object type the provider chose for that block. The schema for that object type includes an attribute called id which can normally be omitted in the native syntax, but must be explicitly set to null in the JSON syntax so that the resulting value will be of the correct type:

{
   ...
   "subnet": [
     {
        "address_prefix": "10.0.0.0/24",
        "name": "cunning_brutality",
        "security_group": "",
        "id": null
     }
   ]
   ...
}

Provider documentation is supposed to mention when an argument is using the legacy parsing mode and link to the relevant documentation, so not mentioning this is arguably a bug in the azurerm_virtual_network documentation which you might consider reporting to the provider developers in the provider's GitHub repository. But that aside, I hope this extra hint above is helpful to get this working, if you aren't able to use the more common native syntax where this special quirk doesn't apply.

Answer 2

After my validation, I also face the same result. As a workaround, you can create the subnet as a separate block instead of a nested argument in a JSON file.

For example,

{
    "resource": {
      "azurerm_virtual_network":{
        "example":{
            "address_space": [
                "10.0.0.0/16"
              ],

              "dns_servers": [],
 
              "location": "eastus",
              "name": "wer5rnis6vnet",
              "resource_group_name": "nancyarm"

        }
      },
      "azurerm_subnet":{
        "example":{
            "name": "wer5rnis6subnet",
            "resource_group_name": "nancyarm",
            "virtual_network_name" : "wer5rnis6vnet",
            "address_prefixes": ["10.0.0.0/24"]
        }
    }
    }
      
}

From Terraform docs about Terraform 0.12 and later. Everything that can be expressed in native syntax can also be expressed in JSON syntax, but some constructs are more complex to represent in JSON due to limitations of the JSON grammar. Read HCL JSON Syntax Specification.

In this case, you also could select the Cherry-pick configuration. You can add the missing required attributes that caused the errors in your plan according to the terraform show output.

For example,

resource "azurerm_virtual_network" "example" {


  address_space         = [
        "10.0.0.0/16",
    ]
  
    location              = "eastus"
    name                  = "gork_vnet"
    resource_group_name   = "mork_rg"
    subnet                {
            address_prefix = "10.0.0.0/24"
            name           = "cunning_brutality"
            security_group = ""
        }

}