topic Re: How to stop a continuous pipeline which is set to RETRY on FAILURE and failing for some reason in Data Engineering

How to stop a continuous pipeline which is set to RETRY on FAILURE and failing for some reason

ziafazal — Wed, 11 Sep 2024 14:52:24 GMT

I have created a pipeline which is continuous and set to RETRY on FAILURE. For some reason it keeps failing and retrying. Is there any way I can stop it. Hitting Stop button throws an error.

Re: How to stop a continuous pipeline which is set to RETRY on FAILURE and failing for some reason

szymon_dybczak — Wed, 11 Sep 2024 15:19:02 GMT

Hi @ziafazal ,

If UI doesn't work you can try use REST API:

https://docs.databricks.com/api/azure/workspace/pipelines/stop

Re: How to stop a continuous pipeline which is set to RETRY on FAILURE and failing for some reason

ziafazal — Thu, 19 Sep 2024 10:44:34 GMT

Hi @szymon_dybczak

I already tried to remove it via REST API but got same error as in the pipeline logs. Eventually, I had to remove workspace to get rid of it.

Re: How to stop a continuous pipeline which is set to RETRY on FAILURE and failing for some reason

szymon_dybczak — Thu, 19 Sep 2024 19:41:51 GMT

Sorry to hear that you needed to remove workspace. Anyway, weird big.