Databricks Connect is not yet supported on the cluster with process isolation enabled

Question

I'm using Azure Databricks (6.5 Runtime) and trying to hook it up to an IDE. However, in order to enable AAD credential passthrough in our cluster we need to set the following properties to true

spark.databricks.passthrough.enabled true
spark.databricks.pyspark.enableProcessIsolation true

But this causes databricks-connect test to fail with the error

Caused by: java.lang.IllegalArgumentException: requirement failed: Databricks Connect is not yet supported on the cluster with process isolation enabled

Are there any workarounds for this? If not, then are there plans to support this use case anytime soon?

Answer 1

Make sure the below configuration is set to true.

 spark.databricks.libraryIsolation.processIsolation.enabled = true