Hi All, I'm trying to change our Ingestion process to use Autoloader to identify new files landing in a directory on ADLS.
The ADLS directory has access tier enabled to archive files older than a certain time period. When I'm trying to set up Autoloader on this directory - it throws an error saying - "Operation failed: "This operation is not permitted on an archived blob."", even though I'm using '.option("cloudFiles.includeExistingFiles", "false")'. I'm trying to understand if I'm overlooking something. Any help is greatly appreciated. Thank you.
