cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Permission error loading dataframe from azure unity catalog to GCS bucket

kiko_roy
New Contributor III

I am creating a data frame by reading a table's data residing in Azure backed unity catalog. I need to write the df or file to GCS bucket. I have configured the spark cluster config using the GCP service account json values.

on running : 

df1.write.format("parquet").save("gs://dev-XXXX-analyt-XXXXXXXX") getting error :
Insufficient privileges: User does not have permission SELECT on any file.. what could be the reason or resolution. Need help

1 ACCEPTED SOLUTION

Accepted Solutions

Kaniz
Community Manager
Community Manager

Hi @kiko_roy, The error message you’re encountering, “Insufficient privileges: User does not have permission SELECT on any file,” indicates that your user account lacks the necessary permissions to read files.

 

Let’s address this issue:

 

Cause:

  • Table access control is enabled on your Databricks cluster, and you are not an admin.
  • The Databricks SQL query analyzer enforces access control policies at runtime. When table access control is enabled, users must have specific permissions to access tables.

Solution:

  • Admins can bypass table access control, but regular users need explicit permissions.
  • An admin must grant SELECT permission on files so that you can create a table.
  • Run the following command in a notebook (as an admin):%sql GRANT SELECT ON ANY FILE TO <user@domain-name> Replace <user@domain-name> with your actual user identifier.

Warning:

  • Users granted access to ANY FILE can bypass restrictions on the catalog, schemas, tables, and views by reading directly from the filesystem.
  • Review the Data object privileges documentation for more information.

Remember to execute the command as an admin, and ensure that the necessary permissions are granted. This should resolve the issue you’re facing.

 

If you encounter any further difficulties, feel free to ask for additional assistance! 🚀

View solution in original post

3 REPLIES 3

Kaniz
Community Manager
Community Manager

Hi @kiko_roy, The error message you’re encountering, “Insufficient privileges: User does not have permission SELECT on any file,” indicates that your user account lacks the necessary permissions to read files.

 

Let’s address this issue:

 

Cause:

  • Table access control is enabled on your Databricks cluster, and you are not an admin.
  • The Databricks SQL query analyzer enforces access control policies at runtime. When table access control is enabled, users must have specific permissions to access tables.

Solution:

  • Admins can bypass table access control, but regular users need explicit permissions.
  • An admin must grant SELECT permission on files so that you can create a table.
  • Run the following command in a notebook (as an admin):%sql GRANT SELECT ON ANY FILE TO <user@domain-name> Replace <user@domain-name> with your actual user identifier.

Warning:

  • Users granted access to ANY FILE can bypass restrictions on the catalog, schemas, tables, and views by reading directly from the filesystem.
  • Review the Data object privileges documentation for more information.

Remember to execute the command as an admin, and ensure that the necessary permissions are granted. This should resolve the issue you’re facing.

 

If you encounter any further difficulties, feel free to ask for additional assistance! 🚀

kiko_roy
New Contributor III

Thanks @Kaniz . The solution did work !!

Kaniz
Community Manager
Community Manager

I want to express my gratitude for your effort in selecting the most suitable solution. It's great to hear that your query has been successfully resolved. Thank you for your contribution.




 

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.