Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
How to stop subsequent iterations in Databricks loop feature? sys.exit(0) or dbutils.notebook.exit() only marks the current task and series of tasks in sequence as failed, but continues with subsequent iterations.
Currently, there is no out of the box feature to achieve that. What you can do, is to try to implement notebook logic that in case of error will cancel for each task run using REST API or python sdk:
