topic Re: How to retrieve Auto Loader client secret from Azure Key Vault? in Data Engineering

How to retrieve Auto Loader client secret from Azure Key Vault?

Magnus — Wed, 23 Nov 2022 13:57:05 GMT

I'm using Auto Loader in a SQL notebook and I would like to configure file notification mode, but I don't know how to retrieve the client secret of the service principal from Azure Key Vault. Is there any example notebook somewhere? The notebook is part of a Delta Live Tables workflow.

Re: How to retrieve Auto Loader client secret from Azure Key Vault?

Geeta1 — Thu, 24 Nov 2022 15:43:18 GMT

Hi @Magnus Johannesson , you must use the Secrets utility (dbutils.secrets) in a notebook or job to read a secret.

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-utils#dbutils-secrets

Hope it helps!

Re: How to retrieve Auto Loader client secret from Azure Key Vault?

Magnus — Mon, 28 Nov 2022 07:03:38 GMT

Thanks @Geeta Sai Boddu! It worked when I converted the notebook to Python and used the Secrets utility. I was hoping that it could be done in a SQL notebook, but maybe that's not supported.

Re: How to retrieve Auto Loader client secret from Azure Key Vault?

Geeta1 — Tue, 29 Nov 2022 10:37:55 GMT

Hey @Magnus Johannesson , great! Yeah, I don't think we can do it using SQL.