Setup dbt-core with Azure Entra ID

[Shazaamzaa]

Hey team, I'm trying to standardize the development environment setup in our team. I've written up a shell script that I want our devs to run in WSL2 after setup. The shell script does the following:

1. setup Azure CLI - install and authenticate

2. Install Databricks CLI

3. Configures `databrickscfg` file with a `dev` Databricks profile that uses Azure CLI to authenticate

4. Installs dbt-core

5. Configures dbt-core profiles.yml <-- This is where my problems arise.

I've run this setup in a virtual Python environment on my WSL2 setup and everything works fine. I've setup dbt-core to authenticate to a SQL warehouse using OAuth and using the Client ID from a registered app in Azure. This setup works for me, but for whatever reason my colleague isn't able to get the dbt-core profile to authenticate. Every attempt to validate the connection using `dbt debug` command we get the error `Runtime error> No data received in callback`. We use a dbt project as part of a Databricks Asset Bundle and the colleague is able to successfully authenticate to Databricks, connect to their Personal Cluster etc. But dbt-core will not connect successfully. I followed the directions per the links below but no success. Both documents just say that run `dbt debug` and you will authenticate and it will work. The prompt to open a browser window works but then loads to a failure. This user definitely has access to the workspace and the SQL warehouse as they can connect via the GUI. Is there something wrong with the profile (see below)? Better even, is there a standard way to setup dbt-core with Azure Databricks and OAuth U2M that I'm missing?
Configure Azure Databricks sign-on from dbt Core with Microsoft Entra ID - Azure Databricks | Micros...
Secure OAuth Configuration for Azure Databricks and dbt: A Guide to Enhance Your Data Workflows
```

gold:

  outputs:

    dev:

      host: "<host_id>.azuredatabricks.net"

      http_path: "<http_path>"

      catalog: dev_catalog

      schema: default

      threads: 4

      type: databricks

      auth_type: oauth

      client_id: "{{ env_var('DATABRICKS_CLIENT_ID') }}" <-- tested as a local environment variable and hard coded

  target: dev

```
Any help is greatly appreciated.

[Kaniz_Fatma · Accepted solution]

Hi @Shazaamzaa, Ensure that the OAuth configuration in Azure is correct and that all environment variables like `client_id` are properly set. Verify that the browser used for authentication isn't blocking necessary pop-ups or redirects. Double-check the `profiles.yml` file for errors and confirm that network settings or firewalls aren’t obstructing the process. Running `dbt debug` with verbose logging might reveal more details about the failure. For a proper setup, consult resources like [Using dbt core with OAuth on Azure Databricks](https://community.databricks.com/t5/technical-blog/using-dbt-core-with-oauth-on-azure-databricks/ba-...), [Configure Azure Databricks sign-on from dbt Core](https://learn.microsoft.com/en-us/azure/databricks/integrations/configure-oauth-dbt), and [Set up Databricks OAuth](https://docs.getdbt.com/docs/cloud/manage-access/set-up-databricks-oauth). 

Let me know if you need further assistance or if there’s anything else I can help with!

[Shazaamzaa]

Hey @Kaniz_Fatma thanks for the response. I persisted a little more with the logs and the issue appears to be related to WSL2 not having a backend credential manager to handle management of tokens supplied by the OAuth process. To be honest, this is my interpretation of the below error message
`RuntimeError: No recommended backend was available. Install a recommended 3rd party backend package; or, install the keyrings.alt package if you want to use the non-recommended backends. See https://pypi.org/project/keyring for details.`
I've seen some other online posts about this issue (see here). I finally attempted the same setup script in a fresh WSL2 install of Ubuntu-24.04 (was previously using 22.02) and it seems to now work, even though the message about the no recommended backend still appears in the log, the OAuth process to authenticate via browser works fine. I guess it was a local issue all along, thanks again.