cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks cli workflow

Yohannes
New Contributor

Is there a way that I can set up and configure a Databricks workflow job and tasks from Databricks cli or api tools by using python? 

Any help would be appreciated. 

#databricksworkflow #databricks 

1 REPLY 1

steyler-db
Databricks Employee
Databricks Employee

Hello and yes, you can set up and configure a Databricks workflow job and tasks using Databricks CLI or API tools with Python. Here are some resources and steps to guide you:

 

  1. Create and run Databricks Jobs: This document: ( https://docs.databricks.com/en/workflows/jobs/create-run-jobs.html )  details how to create and run Databricks Jobs using the Jobs UI. It also provides information on how to use the Databricks CLI and the Jobs API to create and run jobs.

  2. Task type options: You can add different task types to your Databricks job. For a Python script, in the Source drop-down menu, select a location for the Python script (Workspace for a script in the local workspace, DBFS for a script located on DBFS, or Git provider for a script located in a Git repository). In the Path textbox, enter the path to the Python script.

  3. Run jobs interactively, continuously, or using job triggers: You can run your jobs interactively from the Jobs UI, API, or CLI. You can also create a schedule to run your job periodically or run your job when new files arrive in an external location.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group