topic Running DBT via Jobs Compute in Data Engineering

Running DBT via Jobs Compute

brendanc19 — Tue, 15 Aug 2023 15:47:32 GMT

Is it possible to run a DBT job using Jobs Compute?

If not, is there a reason for that, and is there any plans to change that?

Re: Running DBT via Jobs Compute

User16502773013 — Thu, 17 Aug 2023 21:51:57 GMT

Hello @brendanc19 ,

Running DBT with Jobs compute cluster is actually supported

You will need to create a job task with type DBT

Kindly check documentation here for step by step on how to run DBT transformation Databricks jobs

Regards

Re: Running DBT via Jobs Compute

brendanc19 — Mon, 21 Aug 2023 18:35:38 GMT

In the link you gave, it says you must target a SQL Warehouse or an all purpose compute cluster. Is that accurate? If so, why isn't a Job compute supported and is there any plans to support it?

Re: Running DBT via Jobs Compute

User16502773013 — Wed, 23 Aug 2023 21:10:50 GMT

Hello @brendanc19 ,

Currently only available integration for this case is by using serverless or pro SQL warehouses, the current job clusters can't be used.

Regards