Failed to get existing workspaces: containers.Client#ListBlobs: Code="ContainerNotFound"

Question

During a Git-Ops execution using terraform script, always getting bellow error.

Initializing the backend...
Initializing modules...
- application in modules/app-service
- application-insights in modules/application-insights
- cosmosdb-mongodb in modules/cosmosdb-mongodb
- key-vault in modules/key-vault
- redis in modules/redis
- storage-blob in modules/storage-blob
╷
│    Error: Failed to get existing workspaces: containers.Client#ListBlobs: Failure 
     responding to request: StatusCode=404 -- Original Error: autorest/azure: 
     Service returned an error. Status=404 Code="ContainerNotFound" Message="The 
     specified container does not exist.\nRequestId:3d028e99-601e-0063-7325- 
     0b834d000000\nTime:2023-10-30T11:39:20.9706758Z"

I try run to all the recommended fixed like "terraform init" with either the "-reconfigure" or "-migrate-state" flags

Cli Response: Terraform initialized in an empty directory!

Terraform config

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.72.0"
    }
    azurecaf = {
      source  = "aztfmod/azurecaf"
      version = "1.2.26"
    }
  }

  backend "azurerm" {    
    resource_group_name  = "rg-cloud-sample-us"
    storage_account_name = "st001sampleus"
    container_name       = "sample-container-us-1"
    key                  = "terraform.tfstate"
    access_key           = "Je8/L4LapIu7htf+4vrKbTG6+AStiCC0Jw=="
    
}

provider "azurerm" {
  subscription_id      = "********"
  features {}
}

I give all the recommended premising like contributor, owner, Storage Blob Data Contributor / Owner , but nothing work as expected.

Need a solution to resolve this issue!!! Any help will be appreciated.

Answer 1

In the gitops action template I am using, Container name hard coded as tfsate

So if I use the bellow configuration it is working as expected.

    terraform {
     required_providers {
      azurerm = {
       source  = "hashicorp/azurerm"
       version = "3.72.0"
    }
    azurecaf = {
      source  = "aztfmod/azurecaf"
      version = "1.2.26"
    }
   }

   backend "azurerm" {    
     resource_group_name  = "rg-cloud-sample-us"
     storage_account_name = "st001sampleus"
     container_name       = "tfstate"
     key                  = "terraform.tfstate"
     access_key           = "Je8/L4LapIu7htf+4vrKbTG6+AStiCC0Jw=="
    
}

provider "azurerm" {
  subscription_id      = "********"
  features {}
}

Note: Actual issue is in the gitops action template code, if it is take the container name dynamically from backend configuration. It should work as expected. But for that we need to change in the action template code. So for now if I use tfsate as a container name, it is working perfectly.

Answer 2

Error: Failed to get existing workspaces: containers.Client#ListBlobs: Failure 
     responding to request: StatusCode=404 -- Original Error

The error message you're encountering indicates that the specified container does not exist in the specified storage account. There is no issue with permissions, as being a contributor should be sufficient to upload a file to Azure Blob storage

I have contributor role on subscription level and uploaded Terraform backend file to Azure Blob.

    provider "azurerm" {
      features {}
    }
    terraform {
      required_providers {
        azurerm = {
          source  = "hashicorp/azurerm"
          version = "3.72.0"
        }
      }
    
      backend "azurerm" {    
        resource_group_name  = "venkat"
        storage_account_name = "venkatdemo"
        container_name       = "venkat"
        key                  = "terraform.tfstate"
        access_key           = "sfetNjI1uvkWFCjZ+zAkR5OV1nxqvngjgoLneI4EdBGT7rCZIqFk+NxtPLUePv3taifhRThuNB1+AStjU+zYw=="
        
    }
    }

Terraform init Once I ran the terraform init, the .tfstate file has been uploaded to storage account as shown below.

Refer: Stack link related to same issue answered by me