Ravi
Databricks Employee
Databricks Employee

@Cassio Eskelsen​  Using secret in select query with $ syntax is now blocked due to security reasons in the new DBRs(10.2+) and we will soon block it in all supported DBRs with future releases. This is the reason why we are getting the error in DBR 10.2 but not on previous DBR versions.

You can add the following Spark configuration to your cluster setting and it will disable the validation that has been added to new DBRs. This has to be added at the cluster level.

  • spark.databricks.secureVariableSubstitute.enabled false

View solution in original post