Terraform Error: cannot create metastore: Invalid JSON received

Question

Terraform Error: cannot create metastore: Invalid JSON received

223 views Asked by Aniket Karajgikar At 02 November 2024 at 07:28

I am trying to create databricks metastore using Terraform by following documentation.

I tried below code:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.74.0"
    }
    databricks = {
      source  = "databricks/databricks"
    }
  }
}
provider "databricks" {
   profile = var.databricks_connection_profile (DEFAULT conn profile was valid)
   host = data.azurerm_databricks_workspace.dbwdata.workspace_url
   #token= "*****" (I tried this)
   #config_file = <path of .databrickscfg>  (& this as well)
}
    


resource "databricks_metastore" "this" {
       provider = databricks.accounts
       name     = "mymeta"
       storage_root = format("abfss://%s@%s.dfs.core.windows.net/sandbox",
         "container",
         "samplestorage")
       force_destroy = true
       region        = var.azure_region
       depends_on = [
         module.databricks
        ]
       }

But I am getting below error:

Error: cannot create metastore: Invalid JSON received (1040 bytes): <!doctype html>Databricks - Sign inYou need to enable JavaScript to run this app.

Original Q&A

There are 1 answers

**Venkat V** · Answer 1 · 2023-11-21 09:49:16

The error message you're encountering suggests an issue related to authentication or access to the Databricks environment. Make sure it has the required permissions for the Service Principal (Account Admin) to create a Meta Store. follow the Ms doc about assigning account admin to SP.

Here is the updated code to create Metastore using Service Principal

    terraform {
      required_providers {
        databricks = {
          source = "databricks/databricks"
        }
      }
    }
    provider "azurerm" {
      features {}
    
      client_id       = "SP-App-ID"
      client_secret   = ",SP-secret"
      tenant_id       = "Tenant-ID"
      subscription_id = "Subscription-ID"
    }
    
    data "azurerm_resource_group" "this" {
      name = "Venkat"
    }
    
    data "azurerm_databricks_workspace" "this" {
      name                = "sample-databricks"
      resource_group_name = "Venkat"
    }
    
    provider "databricks" {
      alias               = "accounts"
      host                = "https://accounts.azuredatabricks.net"
      account_id          = "Workspace-account-ID"
      azure_tenant_id     = "Tenant-ID"
      azure_client_id     = "Client-ID"
      azure_client_secret = "secret"
    }
    
    resource "azurerm_databricks_access_connector" "unity" {
      name                = "venkatdatabricksmi"
      resource_group_name = data.azurerm_resource_group.this.name
      location            = data.azurerm_resource_group.this.location
      identity {
        type = "SystemAssigned"
      }
    }
    resource "azurerm_storage_account" "unity_catalog" {
      name                     = "thejadatabricksdemo"
      resource_group_name      = data.azurerm_resource_group.this.name
      location                 = data.azurerm_resource_group.this.location
      account_tier             = "Standard"
      account_replication_type = "GRS"
      is_hns_enabled           = true
    }
    
    resource "azurerm_storage_container" "unity_catalog" {
      name                  = "venkat-container"
      storage_account_name  = azurerm_storage_account.unity_catalog.name
      container_access_type = "private"
    }
    
    resource "azurerm_role_assignment" "example" {
      scope                = azurerm_storage_account.unity_catalog.id
      role_definition_name = "Storage Blob Data Contributor"
      principal_id         = azurerm_databricks_access_connector.unity.identity[0].principal_id
    }
    
    resource "databricks_metastore" "this" {
      provider = databricks.accounts
      name     = "primary"
      storage_root = format("abfss://%s@%s.dfs.core.windows.net/",
        azurerm_storage_container.unity_catalog.name,
        azurerm_storage_account.unity_catalog.name)
      force_destroy = true
      region        = "eastus"
    }

Terraform apply:

enter image description here

After running the code, the meta store has been created in workspace as below.

enter image description here

TechQA.

Terraform Error: cannot create metastore: Invalid JSON received

There are 1 answers

Related Questions in AZURE

Related Questions in AZURE-DATABRICKS

Related Questions in DATABRICKS-UNITY-CATALOG

Related Questions in TERRAFORM-PROVIDER-DATABRICKS

Popular Questions

Popular Tags

Trending Questions