Cannot convert Terraform remote state in S3 backend to local state with `terraform init -reconfigure`

Question

I'm using Terraform v1.0.0 and created a remote backend using AWS S3 and AWS DynamoDB as explained in Terraform Up & Running by Yevgeniy Brikman:

1. I wrote the code for the S3 bucket and the DynamoDB table and created the resources via terraform apply
2. I added terraform { backend "S3" {} } to my code
3. I created a backend.hcl file with all the relevant parameters
4. I moved my local state to S3 by invoking terraform init -backend-config=backend.hcl

Now I want to convert the remote state back to local state so I can safely delete the remote backend. Brikman explains to do this, one has to remove the backend configuration and invoke terraform init. When I try this, I see this:

$ terraform init
Initializing modules...

Initializing the backend...
╷
│ Error: Backend configuration changed
│ 
│ A change in the backend configuration has been detected, which may require migrating existing state.
│ 
│ If you wish to attempt automatic migration of the state, use "terraform init -migrate-state".
│ If you wish to store the current configuration with no changes to the state, use "terraform init -reconfigure".
╵

I figured the correct approach is to use -reconfigure which seems to work at first glance:

$ terraform init -reconfigure
Initializing modules...

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/aws from the dependency lock file
- Using previously-installed hashicorp/aws v3.47.0
- Using previously-installed hashicorp/random v3.1.0

Terraform has been successfully initialized!

However, executing terraform plan reveals that the initialization did not succeed:

$ terraform plan
╷
│ Error: Backend initialization required, please run "terraform init"
│
│ Reason: Unsetting the previously set backend "s3"
│
│ The "backend" is the interface that Terraform uses to store state,
│ perform operations, etc. If this message is showing up, it means that the
│ Terraform configuration you're using is using a custom configuration for
│ the Terraform backend.
│
│ Changes to backend configurations require reinitialization. This allows
│ Terraform to set up the new configuration, copy existing state, etc. Please run
│ "terraform init" with either the "-reconfigure" or "-migrate-state" flags to
│ use the current configuration.
│
│ If the change reason above is incorrect, please verify your configuration
│ hasn't changed and try again. At this point, no changes to your existing
│ configuration or state have been made.
╵

The only way to unset the backend seems to be via terraform init -migrate-state:

$ terraform init -migrate-state
Initializing modules...

Initializing the backend...
Terraform has detected you're unconfiguring your previously set "s3" backend.
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "s3" backend to the
  newly configured "local" backend. No existing state was found in the newly
  configured "local" backend. Do you want to copy this state to the new "local"
  backend? Enter "yes" to copy and "no" to start with an empty state.

  Enter a value: yes
    
Successfully unset the backend "s3". Terraform will now operate locally.

Is it not possible to convert the state via terraform init -reconfigure despite Terraform explicitly telling me so? If so, what does terraform init -reconfigure do exactly?

Answer 1

Below work around solved this problem for me.

Add below and run terraform init

terraform { backend "local" { } }

Answer 2

From the official docs1, it seems -reconfigure is a bit destructive in the sense that it disregards the existing config. I would think that if you did changes to the backend, and then ran the command, it would only work from the assumption that this is a new config. I just recently read the docs myself, and I did not know that this was the behavior.

So, back to your question, I would assume -migrate-state is the desired option to use when migrating states between different backends. I understand from your issue that this was the case using terraform init -migrate-state?

Answer 3

As said by MindTooth, command init -migrate-state does exactly what you want to do. It migrates the state unchanged when a different backend is configured.

init -reconfigure will initialise the new backend with a clean empty state.

Another way to do it is by pulling the state from s3 backend to a json file. Then initialising an empty local backend using init -reconfigure, and pushing the state back in.

terraform state pull > state.json
terraform init -reconfigure
terraform state push state.json