Can I use databricks principals on databricks connect 12.2?

Surajv — Fri, 16 Feb 2024 08:32:10 GMT

Hi community,

Is it possible to use Databricks service principals for authentication on Databricks connect 12.2 to connect my notebook or code to Databricks compute, rather than using personal access token?

I checked the docs and got to know that upgrading Databricks version to v13+ gives us access to using Databricks service principals in Databricks connect, but that also requires setting up Unity catalogue.

As per our current use case, we are restricting to using Databricks connect 12.2 only (unless no other way out) but that is limiting the authentication features.

Hence wanted to ask, if there's a way out or any piece of document which shows a way to use service principals in Databricks connect 12.2?

Re: Can I use databricks principals on databricks connect 12.2?

Surajv — Fri, 16 Feb 2024 11:21:42 GMT

Hi @Retired_mod

Thanks for your response. I was able to generate the token of the service principal following this doc, later saved it in the <Databricks Token> variable prompted when running databricks-connect configure command in terminal. And was able to run the pyspark code as expected.

But as I see read through the docs, the databricks principals token expires in 1 hr. I have few doubts:

Is there a way to increase the expiration time of the token?
Where is this export variable (i.e Databricks Token) or other databricks-connect variables (like Databricks Host, Cluster ID, Org ID) stored in environment?
As I was trying to build a script which hits the service principals APIs to get the token and later exports the token value in databricks-connect configure command (doing something similar given in this link, so that I won't have to keep generating a new token everytime)
I tried finding the variables using: <grep -iR "Databricks Token"> and other commands, but didn't get any useful results.

topic Re: Can I use databricks principals on databricks connect 12.2? in Get Started Discussions

Can I use databricks principals on databricks connect 12.2?

Re: Can I use databricks principals on databricks connect 12.2?