Hello @szymon_dybczak ,

I have provided the required permissions for the Service principal. The Queue is generated in the Storage Account. We are actually not using writeStream functionalities, we are using Unity Catalog and storing the data via Delta Live Pipelines into Delta Live Tables. So In that context how can we use the checkpoint option. 

One more observation from my side, when queue is cleared or deleted and when run incrementally the pipeline is accepting the new files. 

Thanks for the response.

 

Regards,

Ibrahim