I have a terrform script which is used to install Dataproc using GKE, I am struggling how to install Jupyter inside the dataproc cluster.
I tried the following
virtual_cluster_config {
staging_bucket = google_storage_bucket.staging_bucket.name
kubernetes_cluster_config {
kubernetes_namespace = "foobar"
kubernetes_software_config {
component_version = {
"SPARK" : "3.1-dataproc-7"
}
properties = {
"spark:spark.eventLog.enabled": "true"
"dataproc:jupyter.enable": "true"
}
}
}
}
But the properties did not effect after apply.
I am not seeing a way I can use initilization actions if I am using GKE for dataproc according to TF Docs https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/dataproc_cluster#virtual_cluster_config