Terraform/GCP Kubernetes error when trying to create namespace

Question

getting the below error with my terraform config.

    Error: Post "https://35.224.178.141/api/v1/namespaces": x509: certificate signed by unknown authority

  on main.tf line 66, in resource "kubernetes_namespace" "example":
  66: resource "kubernetes_namespace" "example" {

Here is my config, all I want to do for now is create a cluster auth with it, and create a namespace. I have searched everyone and cant see where anyone else has run into this problem. It is most likely something stupid I am doing. I thought this would be relatively simple, but its turning out to be a pain. I dont want to have to wrap gcloud commands in my build script.

provider "google" {
  project = var.project
  region  = var.region
  zone    = var.zone
  credentials = "google-key.json"
}


terraform {
  backend "gcs" {
    bucket = "tf-state-bucket-devenv"
    prefix = "terraform"
    credentials = "google-key.json"
   }
}

resource "google_container_cluster" "my_cluster" {
  name     = var.kube-clustername
  location = var.zone
  remove_default_node_pool = true
  initial_node_count       = 1

  master_auth {
    username = ""
    password = ""

    client_certificate_config {
      issue_client_certificate = false
    }
  }
}

resource "google_container_node_pool" "primary_preemptible_nodes" {
  name       = var.kube-poolname
  location   = var.zone
  cluster    = google_container_cluster.my_cluster.name
  node_count = var.kube-nodecount

  node_config {
    preemptible  = var.kube-preemptible
    machine_type = "n1-standard-1"
    disk_size_gb = 10
    disk_type = "pd-standard"


    metadata = {
      disable-legacy-endpoints = "true",
    }

    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]
  }
}
data "google_client_config" "provider" {}

provider "kubernetes" {
  load_config_file = false
  host = "https://${google_container_cluster.my_cluster.endpoint}"
  cluster_ca_certificate = "{base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}"
  token = "{data.google_client_config.provider.access_token}"
}


resource "kubernetes_namespace" "example" {
  metadata {
    name = "my-first-namespace"
  }
}

Answer 1

TL;DR

Change the provider definition to:

provider "kubernetes" {
  load_config_file = false
  host = "https://${google_container_cluster.my_cluster.endpoint}"
  cluster_ca_certificate = base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)
  token = data.google_client_config.provider.access_token
}

What changed?

The "{}" was deleted from the cluster_ca_certificate and token values

I included the explanation below.

---

I used your original terraform file and I received the same error as you. I modified (simplified) your terraform file and added the output definitions:

resource "google_container_cluster" "my_cluster" {
  OMMITED 
}

data "google_client_config" "provider" {}

provider "kubernetes" {
  load_config_file = false
  host = "https://${google_container_cluster.my_cluster.endpoint}"
  cluster_ca_certificate = "{base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}"
  token = "{data.google_client_config.provider.access_token}"
}


output "cert" {
  value = "{base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}"
}

output "token" {
  value = "{data.google_client_config.provider.access_token}"
}

Running above file showed:

• $ terraform apply --auto-approve

data.google_client_config.provider: Refreshing state...
google_container_cluster.my_cluster: Creating...
google_container_cluster.my_cluster: Creation complete after 2m48s [id=projects/PROJECT-NAME/locations/europe-west3-c/clusters/gke-terraform]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

cert = {base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}
token = {data.google_client_config.provider.access_token}

As you can see the values were interpreted as strings from the provider and not "processed" to get the needed values. To fix that you will need change the provider definition to:

  cluster_ca_certificate = base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)
  token = data.google_client_config.provider.access_token

Running $ terraform apply --auto-approve once again:

data.google_client_config.provider: Refreshing state...
google_container_cluster.my_cluster: Creation complete after 3m18s [id=projects/PROJECT-NAME/locations/europe-west3-c/clusters/gke-terraform]
kubernetes_namespace.example: Creating...
kubernetes_namespace.example: Creation complete after 0s [id=my-first-namespace]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

cert = -----BEGIN CERTIFICATE-----
MIIDKzCCAhOgAwIBAgIRAO2bnO3FU6HZ0T2u3XBN1jgwDQYJKoZIhvcNAQELBQAw
<--OMMITED-->
a9Ybow5tZGu+fqvFHnuCg/v7tln/C3nVuTbwa4StSzujMsPxFv4ONVl4F4UaGw0=
-----END CERTIFICATE-----

token = ya29.a0AfH6SMBx<--OMMITED-->fUvCeFg

As you can see the namespace was created. You can check it by running:

• $ gcloud container clusters get-credentials CLUSTER-NAME --zone=ZONE
• $ kubectl get namespace my-first-namespace

Output:

NAME                 STATUS   AGE
my-first-namespace   Active   3m14s

---

Additional resources:

• Terraform.io: Docs: Configuration: Outputs
• Terraform.io: Docs: Configuration: Variables