SparkException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SE

GANAPATI_HEGDE
New Contributor III

‎07-08-2025 06:41 AM - edited ‎07-08-2025 06:44 AM

I am trying to build a dlt pipeline which fetches data from fabric onelake tables and stores it in databricks.

 

To do this, i am using azure service principals and  databricks asset bundles. i have setup authentication configs as per official documentaion from azure. but i am facing below error even though i have all the necessary acceses

SparkException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SELECT on any file. SQLSTATE: 42501

GANAPATI_HEGDE_0-1751982239407.png

 

the code works perfectly fine when i run them in personal compute but throws this error when i run them in job clusters.

0 Kudos

nayan_wylde
Esteemed Contributor II

‎07-08-2025 07:13 AM

In the workflow check the Runas. If the workflow is running using an SPN. Check if the SPN have appropriate permissions to read and select files.

0 Kudos

GANAPATI_HEGDE
New Contributor III
In response to nayan_wylde

‎07-08-2025 08:22 AM

Yes run as is set to SP. SP has permission to onelake, and unity catalog

0 Kudos

nayan_wylde
Esteemed Contributor II
In response to GANAPATI_HEGDE

‎07-08-2025 10:33 AM

Can you please share the grants that is on the table. I highly doubt there is select permission granted on this schema.

0 Kudos