- 7413 Views
- 14 replies
- 9 kudos
Suggestion Needed for a Orchestrator/Scheduler to schedule and execute Jobs in an automated way
Hello Friends,We have an application which extracts dat from various tables in Azure Databricks and we extract it to postgres tables (postgres installed on top of Azure VMs). After extraction we apply transformation on those datasets in postgres tabl...
- 7413 Views
- 14 replies
- 9 kudos
Latest Reply
- 9 kudos
You can leverage Airflow, which provides a connector for databricks jobs API, or can use databricks workflow to orchestrate your jobs where you can define several tasks and set dependencies accordingly.
- 9 kudos