One of my clients has been orchestration Databricks notebooks using Airflow + REST API. They're curious about the pros/cons of switching these jobs to Databricks jobs with Task Orchestration.I know there are all sorts of considerations - for example,...
@Kaniz Fatma​ Hello Kaniz, I'm currently working with a major Enterprise Client looking to make the choice between the Airflow vs Databricks for Jobs scheduling. Our Entire code base is in Databricks and we are trying to figure out the complexities t...
I've created a new job with the new UI / feature enabled.
I managed to create one task with a new job cluster successfully but when adding a second task with a new job cluster and trying to save it I received the following error:
Error updating [Job...
Hi, I have a similar issue.I'm using the 14 day free trial, setting up the default basic-starter cluster.Then I just followed the basic introduction of dbx with python. https://docs.databricks.com/dev-tools/dbx.htmlCommand Execute is working, but whe...
This morning I encountered an issue when trying to create a new job using the Workflows UI (in browser). Never had this issue before.The error message that appears is:"You are not entitled to run this type of task, please contact your Databricks admi...
Enabling of Task Orchestration feature in Jobs via API as wellDatabricks supports the ability to orchestrate multiple tasks within a job. You must enable this feature in the admin console. Once enabled, this feature cannot be disabled. To enable orch...
Hi everybody!
Is it possible to use one job cluster for multiple tasks using new task orchestration tool?
If so then please tell how.
If it's impossible then this new tool looks useless
Regards,
Daniil.
