DataProc Jupyter

17 views Asked by At

I have a terrform script which is used to install Dataproc using GKE, I am struggling how to install Jupyter inside the dataproc cluster.

I tried the following

virtual_cluster_config {
    staging_bucket = google_storage_bucket.staging_bucket.name
    kubernetes_cluster_config {
        kubernetes_namespace = "foobar"

        kubernetes_software_config {
            component_version = {
                "SPARK" : "3.1-dataproc-7"
            }

            properties = {
                "spark:spark.eventLog.enabled": "true"
                "dataproc:jupyter.enable": "true"
            }
        }
    }
}

But the properties did not effect after apply.

I am not seeing a way I can use initilization actions if I am using GKE for dataproc according to TF Docs https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/dataproc_cluster#virtual_cluster_config

0

There are 0 answers