SparkException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SE

[GANAPATI_HEGDE]

I am trying to build a dlt pipeline which fetches data from fabric onelake tables and stores it in databricks.

To do this, i am using azure service principals and  databricks asset bundles. i have setup authentication configs as per official documentaion from azure. but i am facing below error even though i have all the necessary acceses

SparkException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SELECT on any file. SQLSTATE: 42501

the code works perfectly fine when i run them in personal compute but throws this error when i run them in job clusters.

[nayan_wylde]

In the workflow check the Runas. If the workflow is running using an SPN. Check if the SPN have appropriate permissions to read and select files.

[GANAPATI_HEGDE]

Yes run as is set to SP. SP has permission to onelake, and unity catalog

[nayan_wylde]

Can you please share the grants that is on the table. I highly doubt there is select permission granted on this schema.