Hubert-Dudek
Databricks MVP

"Script exit status is non-zero," which means that the script returned some error. To debug, you need to identify what exactly.

The output of the bash script is in cluster logs. You can add some echo (timestamp and code executed) in the script to know what was processed.

You can also implement some error handling logic in bash, but it isn't pleasant https://linuxhint.com/bash_error_handling/


My blog: https://databrickster.medium.com/