Autoloader on ADLS blobs with archival enabled

Manjula_Ganesap
Contributor

Hi All, I'm trying to change our Ingestion process to use Autoloader to identify new files landing in a directory on ADLS. 

The ADLS directory has access tier enabled to archive files older than a certain time period. When I'm trying to set up Autoloader on this directory - it throws an error saying - "Operation failed: "This operation is not permitted on an archived blob."", even though I'm using '.option("cloudFiles.includeExistingFiles", "false")'. I'm trying to understand if I'm overlooking something. Any help is greatly appreciated. Thank you.