Python Code not working in DBR 10.4 LTS - Shared Mode

[parthsalvi]

Hello,

We're trying to use DBR 10.4 LTS in Shared Mode.

But our python code is not working in that cluster.

We're running into following message

Your administrator has only allowed sql commands on this cluster. This execution contained at least one disallowed language 

As suggested, we also tried setting spark conf "spark.databricks.repl.allowedLanguages"

in cluster but still we get same message.

In Admin login both workspace and account level we're not able to locate any such option of setting language.

Note: We've a customer managed VPC installation and using Unity Catalog as well.

Python code is working in No isolation mode but we cannot use it as it doesn't support unity catalog.

Is this bug or a set up issue?

[Cedric]

Hi Parthsalvi,

Could you please try using 11.1+ and see if that works?

[parthsalvi]

Yes it does work on 11.1+ . But we're running into few issues on 11.2

like these So we wanted to fix the issue in 10.4 itself. Is this a known issue ?

[Cedric · Accepted solution]

Hey @Parth Salvi​,

Got it, could you please try adding the following Spark config to your cluster running DBR 10.4?

spark.databricks.unityCatalog.userIsolation.python.preview true

[parthsalvi]

Python code now working on 10.4 LTS shared mode. Thanks a lot, Cedric !!!

But still facing same issues in 10.4 as well. I've added a separate post for it

https://community.databricks.com/s/question/0D58Y00009DDRguSAH/few-sparks-apis-not-working-in-dbr-11...

Does it seem like custom vpc set up issue ?