cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks cli workflow

Yohannes
New Contributor

Is there a way that I can set up and configure a Databricks workflow job and tasks from Databricks cli or api tools by using python? 

Any help would be appreciated. 

#databricksworkflow #databricks 

1 REPLY 1

steyler-db
New Contributor III
New Contributor III

Hello and yes, you can set up and configure a Databricks workflow job and tasks using Databricks CLI or API tools with Python. Here are some resources and steps to guide you:

 

  1. Create and run Databricks Jobs: This document: ( https://docs.databricks.com/en/workflows/jobs/create-run-jobs.html )  details how to create and run Databricks Jobs using the Jobs UI. It also provides information on how to use the Databricks CLI and the Jobs API to create and run jobs.

  2. Task type options: You can add different task types to your Databricks job. For a Python script, in the Source drop-down menu, select a location for the Python script (Workspace for a script in the local workspace, DBFS for a script located on DBFS, or Git provider for a script located in a Git repository). In the Path textbox, enter the path to the Python script.

  3. Run jobs interactively, continuously, or using job triggers: You can run your jobs interactively from the Jobs UI, API, or CLI. You can also create a schedule to run your job periodically or run your job when new files arrive in an external location.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.