wait, but I think this is another problem. We are mounting an S3 bucket into DBFS and my path is then something like this:

S3_BUCKET_PATH = "dbfs:/mnt/mounted_bucket_name/"

df = spark.read.format(format).load(S3_BUCKET_PATH)