Spark Remote error when connecting to cluster

[ff-paulo-barbos]

Hi, I am using the latest version of pyspark and I am trying to connect to a remote cluster with runtime 13.3.

My doubts are:

- Do i need databricks unity catalog enabled?

- My cluster is already in a Shared policy in Access Mode, so what other configuration is missing? 

Error callstack:

status = StatusCode.FAILED_PRECONDITION
details = "INVALID_STATE: cluster xxx-xxx-xxx is not Shared or Single User Cluster.

[Debayan]

Hi, Is your workspace is already unity catalog enabled? Also, did you go through the considerations for enabling workspace for unity catalog? 

https://docs.databricks.com/en/data-governance/unity-catalog/enable-workspaces.html#considerations-b...

Please let us know if this helps, also, please tag @Debayan with your next comment which will notify me. Thanks!

[ZivadinM]

Do I need unity catalog to use databricks connect?