Have you ever wondered how to automate your #databricks jobs and workflows without using the UI? If you want to manage your Databricks resources as co...
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-03-2023 05:12 AM
Have you ever wondered how to automate your #databricks jobs and workflows without using the UI? If you want to manage your Databricks resources as code, you should check out Terraform.
Here is a simple example of creating a job that runs a notebook on a new cluster using Terraform.
resource "databricks_job" "example" {
name = "example-job"
new_cluster {
num_workers = 2
spark_version = "13.0.x-scala2.12"
node_type_id = "Standard_DS3_v2"
}
notebook_task {
notebook_path = "/Users/example@example.com/MyNotebook"
}
}
Labels:
- Labels:
-
databricks
-
Terraform
2 REPLIES 2
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-03-2023 06:09 AM
or use ADF 🙂 Still waiting for actual added value on using Databricks Workflow over ADF.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-03-2023 09:55 AM
at least logical if/no is coming 🙂
![](/skins/images/B38AF44D4BD6CE643D2A527BE673CCF6/responsive_peak/images/icon_anonymous_message.png)
![](/skins/images/B38AF44D4BD6CE643D2A527BE673CCF6/responsive_peak/images/icon_anonymous_message.png)