topic Re: How do we concatenate some fixed string with a secret value in Spark Config in Databricks Job Cluster? in Data Engineering

How do we concatenate some fixed string with a secret value in Spark Config in Databricks Job Cluster?

debanjan89 — Thu, 25 Aug 2022 07:03:55 GMT

Hi Team,

I am trying to configure access to adls through Service Principal through Spark Config in Databricks job cluster. like,

fs.azure.account.oauth2.client.id.<adls_account_name>.dfs.core.windows.net {{secrets/scopeName/clientID}}

The above statement works fine..

How do we concatenate some fixed string with a secret value?

For example,

fs.azure.account.oauth2.client.endpoint.<adls_account_name>.dfs.core.windows.net "https://login.microsoftonline.com/"+{{secrets/scopeName/tenantID}}+/"oauth2/token"

fs.azure.account.oauth2.client.endpoint.<adls_account_name>.dfs.core.windows.net https://login.microsoftonline.com/{{secrets/scopeName/tenantID}}/oauth2/token

Re: How do we concatenate some fixed string with a secret value in Spark Config in Databricks Job Cluster?

Vidula — Mon, 12 Sep 2022 13:35:24 GMT

Hi @Debanjan Ray

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: How do we concatenate some fixed string with a secret value in Spark Config in Databricks Job Cluster?

Manimkm08 — Tue, 03 Jan 2023 12:52:09 GMT

@Debanjan Ray @Vidula Khanna Am also facing the issue. Can someone please share the solution or workaround for the issue. Would be helpful for others like me who are facing the issue in future.

Re: How do we concatenate some fixed string with a secret value in Spark Config in Databricks Job Cluster?

Manimkm08 — Wed, 04 Jan 2023 13:16:00 GMT

@Kaniz Fatma We are blocked on this issue. Can you please look into the thread and give your suggestion to workaround it.