Databricks Community

augustsc · 01-05-2024

I'm running a scheduled workflow with a dbt task on Azure Databricks. We want to export the dbt-output from the dbt task to a storage container for our Slim CI setup and data observability. The issue is, that the Databricks API (/api/2.1/jobs/runs/ge...

augustsc

Hi! We're also facing the same problem. We don't have any materialized views or streaming tables, but we are still seeing PREMIUM_JOBS_SERVERLESS_COMPUTE_EU_NORTH with billing_origin_product = SHARED_SERVERLESS_COMPUTE generated each day at a time wh...

augustsc · 03-14-2024

Thank you for your reply. Yes, we are using the Task run ID, not the Job or Job run. I am able to get the output from the task, it is just that the dbt_output part is always empty.

augustsc · 01-19-2024

Hi, thank you for your reply!Is there a way to check if dbt-core/databricks is compatible with the Databricks runtime?We are currently using dbt-core and dbt-databricks 1.7.3 with Databricks runtime 14.2. Sill everything runs as expected, but dbt_out...

Databricks Community

User Stats

User Activity

Running dbt in a Databricks task, the dbt_output from the Databricks jobs api is empty

Re: Serverless Compute Cost Monitoring (System Tables)

Re: Running dbt in a Databricks task, the dbt_output from the Databricks jobs api is empty

Re: Running dbt in a Databricks task, the dbt_output from the Databricks jobs api is empty