Information in this post Speed Up Data Flow: Databricks and SAS | Databricks Blog led me to using spark-sas7dbat package to read SAS files and save to delta for downstream processes with great results. I was able to load very large files quickly that weren't working with pandas or other methods I'd tried.
Recently made the move to using Unity Catalogue, currently using a cluster with DBR 13.3. I am not an admin, but I have ALL PRIVILEGES. When I try to use the spark-sas7dbat package, I am now getting the error: "User does not have permission SELECT on any file error message." which doesn't make sense to me in the context of UC permissions model. I'm a bit lost for next steps, any ideas on how to address are welcome
