Have you ever wondered how to automate your #databricks jobs and workflows without using the UI? If you want to manage your Databricks resources as co...

[Hubert-Dudek]

Have you ever wondered how to automate your #databricks jobs and workflows without using the UI? If you want to manage your Databricks resources as code, you should check out Terraform.

Here is a simple example of creating a job that runs a notebook on a new cluster using Terraform.

resource "databricks_job" "example" {
 
  name = "example-job"
 
  new_cluster {
    num_workers   = 2
    spark_version = "13.0.x-scala2.12"
    node_type_id  = "Standard_DS3_v2"
  }
 
  notebook_task {
    notebook_path = "/Users/example@example.com/MyNotebook"
  }
}
 

[-werners-]

or use ADF 🙂 Still waiting for actual added value on using Databricks Workflow over ADF.

[Hubert-Dudek]

at least logical if/no is coming 🙂