topic Re: Warning in dbt task in Warehousing & Analytics

Warning in dbt task

martindlarsson — Thu, 21 Nov 2024 14:40:22 GMT

We are using dbt core and running our transformations in a dbt task. Since a few months ago we have started to see a warning message in our runs. This started happening after an update of dbt core and it seems that Databricks needs to update the definition of the generated prifiles file. Right now we are running version 1.8.7 of dbt core.

This is the warning:

+ dbt deps
12:48:49 Running with dbt=1.8.7
12:48:49 [WARNING]: Deprecated functionality User config should be moved from the 'config' key in profiles.yml to the 'flags' key in dbt_project.yml.
12:48:49 Installing brooklyn-data/dbt_artifacts

Re: Warning in dbt task

bhanu_gautam — Thu, 28 Nov 2024 10:45:07 GMT

@merry867 , Thanks for sharing , yes it looks like warning message indicates that user configuration should be moved from the config key in the profiles.yml file to the flags key in the dbt_project.yml file.

Re: Warning in dbt task

martindlarsson — Thu, 28 Nov 2024 13:01:13 GMT

Looks like you had help from ChatGPT?

Re: Warning in dbt task

martindlarsson — Thu, 28 Nov 2024 13:05:31 GMT

Both of you missed the sentence "Databricks needs to update the definition of the generated profiles file."

The warning is regarding the structure of the profiles.yml which is generated by Databrick. This is not anything I as a user can change accept follow the guide to run with a custom profile which I do not have a need for.

So I wand to reach somebody working at Databricks and can give this information to the team that works on the dbt task.