Hey,

in my application I am using click to facilitate CLI operations. It works locally, in notebooks, when scripts are run locally, but it fails in Databricks. I defined a task that, as an entrypoint, accepts the file where the click-decorated function resides, I pass the required params and everything works just fine up until the end of the task's run. Then the following error is raised:

SystemExit: 0
An exception has occurred, use %tb to see the full traceback.

---------------------------------------------------------------------------
Exit                                      Traceback (most recent call last)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/click/core.py:1088, in BaseCommand.main(self, args, prog_name, complete_var, standalone_mode, windows_expand_args, **extra)
   1080             return rv
   1081         # it's not safe to `ctx.exit(rv)` here!
   1082         # note that `rv` may actually contain data like "1" which
   1083         # has obvious effects
   (...)
   1086         # even always obvious that `rv` indicates success/failure
   1087         # by its truthiness/falsiness
-> 1088         ctx.exit()
   1089 except (EOFError, KeyboardInterrupt) as e:
   1090     echo(file=sys.stderr)

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/click/core.py:692, in Context.exit(self, code)
    690 def exit(self, code: int = 0) -> "te.NoReturn":
    691     """Exits the application with a given exit code."""
--> 692     raise Exit(code)

Exit: 0

During handling of the above exception, another exception occurred:

SystemExit: 0

I want to reiterate that the task is successful in the sense that it executes what is requested and stores the data I want to store. The error appears AFTER the code is executed and it isn't even manifested in the run's logs - they are impeccable. In click's documentation it says: "This will always terminate the application after a call. If this is not wanted, SystemExit needs to be caught." I was trying to catch the execution, but it's all in vain as, most likely, I do not even know where to catch.