Databricks Community

Also, you need to pass the parameters using the --vars flag like that: dbt run --vars '{"start_time": "{{job.start_time.[timestamp_ms]}}"}' You will need to modify the 3rd dbt command in your job.

Can you try creating another profile instead of the Default one and try with it, it seems that what it is not collecting is the cluster details but wanted to check with a new profile  

What would be the result if instead of using ' ' you use ` `? 

Got it, allow me with some time to look for information around this one

Hi @jeremy98, I think it has to do with the serverless version being used outside the workflow versus in DABs, since python version changes. please see: https://docs.databricks.com/en/release-notes/serverless/index.html both the versions have differe...

Hi @lauraxyz, I think you can use the system table and audit logs to achieve that monitoring:  https://docs.databricks.com/en/admin/account-settings/audit-logs.html

Hi @yvishal519 Did you end up finding that the " __CreatedDateTime and __UpdatedDateTime columns are managed as expected within the apply_changes function to ensure correct handling of timestamps." as  @NandiniN had replied originally OR are you mana...

Where you able to get this issue resolved after looking at the permissions level on your schema and tables?

I'm not actually getting a failure, but I want to see that the value of a parameter I set up in the workflow is being set correctly where it is used. In this case I'd like to parameterize the Kinesis stream I'll be polling from, so I'd like to pass t...

Any fix here?

@Alberto_Umana Thank you for sharing. However, I have already followed the steps mentioned in the article, and I’m still unable to establish a connection.When using AWS access keys, the connection is successful, which confirms that there are no issue...

To address the issue of needing to run initialization code that reads file content during the load of a UDF (User Defined Function) in Databricks, you should avoid performing file operations in the constructor due to security restrictions. Instead, y...