Unity Catalogue and SAS data using spark-sas7dbat

[MKE]

Information in this post Speed Up Data Flow: Databricks and SAS | Databricks Blog led me to using spark-sas7dbat package to read SAS files and save to delta for downstream processes with great results. I was able to load very large files quickly that weren't working with pandas or other methods I'd tried.

Recently made the move to using Unity Catalogue, currently using a cluster with DBR 13.3. I am not an admin, but I have ALL PRIVILEGES. When I try to use the spark-sas7dbat package,  I am now getting the error: "User does not have permission SELECT on any file error message." which doesn't make sense to me in the context of UC permissions model. I'm a bit lost for next steps, any ideas on how to address are welcome

[Kaniz_Fatma]

Hi @MKE, An admin needs to grant you the SELECT permission on the files. Ensure that your cluster is configured correctly with the necessary permissions. Make sure that the access control policies in Unity Catalog are set up correctly. 

I hope these steps help you get back on track! If you have any more questions or need further assistance, feel free to ask.