Databricks

Dipesh · 01-31-2023

Hi All,I have a job/workflow scheduled in Databricks to run after every hour.How can I configure my Job to pause whenever a job run fails? (Pause the job/workflow on first failure)I would want to prevent triggering multiple runs due to the scheduled/...

Dipesh · 01-31-2023

Hi All,I have some data in Delta table with multiple columns and each record has a unique identifier.I want to update some columns as per the new values coming in for each of these unique records. However updating one record at a time is taking a lot...

Dipesh · 01-31-2023

Hi @Hubert Dudek , Thank you for your suggestion.I understand that we can use Jobs API to change the pasue_status of job on errors, but sometimes we observed that the workflow/job fails due to cluster issues (while the job clusters are getting creat...

Dipesh · 01-31-2023

@Shanmugavel Chandrakasu Thank you for your response. We have enabled Databricks alerts but would be a problem during weekends and holidays  Also the data get updated after each runs so repairing the run after we detect the failure would lead to us...

Databricks

User Stats

User Activity

Pausing a scheduled Azure Databricks job after failure

Bulk updating Delta tables in Databricks

Re: Pausing a scheduled Azure Databricks job after failure

Re: Pausing a scheduled Azure Databricks job after failure