topic dbx with serverless clusters in Data Engineering

dbx with serverless clusters

lbdatauser — Thu, 22 Aug 2024 12:42:18 GMT

With dbx, is it impossible to create tasks that run on serverless clusters? Is it necessary to use Databricks bundles for it?

https://dbx.readthedocs.io/en/latest/reference/deployment/

https://learn.microsoft.com/en-us/azure/databricks/jobs/run-serverless-jobs

Re: dbx with serverless clusters

Satyadeepak — Thu, 16 Jan 2025 17:48:45 GMT

It is possible to create tasks that run on serverless clusters using dbx. Also, please note that Databricks recommends that you use Databricks Asset Bundles instead of dbx by Databricks Labs. See What are Databricks Asset Bundles and Migrate from dbx to bundles