I have a notebook that calls dbutils.fs.ls() for some derived file path in azure. Occasionally, this path may not exist, and in general I can't always guarantee that the path exists. When the path doesn't exist it throws an "ExecutionError" which appears to be suppressing the Py4JJavaError. Is there a way to handle this error while letting other exceptions that may occur be raised?

As far as I can tell, the "ExecutionError" class is defined and instantiated from within the function that's suppressing the Py4JJavaError making it locally scoped to that function. I'd rather avoid catching all exceptions if I can only catch this one exception being raised. If it just returned the Py4JJavaError then that could be easily caught and handled.

If there's no way to catch this specific exception being raised then could we request a feature for the class defining this new error be defined under FSHandler so it can be caught?

I've seen other similar questions, but I haven't found answers: https://community.databricks.com/t5/data-engineering/how-to-handle-java-io-exception-in-python-noteb...