yes, I can read from notebook with DBR 6.4, when I specify this path:

wasbs:REDACTED_LOCAL_PART@blobStorageName.blob.core.windows.net/cook/processYear=2021/processMonth=12/processDay=30/processHour=18

but the same using DBR 6.4 from spark-submit, it fails again.. each time complaining of different part files under different partitions.

Also we have the exact same code, with exact same spark configs deployed in several different regions but this is only where we have an issue. Could this be data related, like some part file size limitations for the given spark version?