cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Shallow clone and issue with MODIFY permission to source table

cszczotka
New Contributor III

Hi,

I'm running shallow clone for external delta tables. The shallow clone is failing for source tables where I don't have MODIFY permission. I'm getting below exception. I don't understand why MODIFY permission to source table is required. Is there any other solution than granting  MODIFY permission to source tables ? I would like to avoid this in case of cloning delta tables from production sources.

Thanks,

: Failed to acquire a SAS token for create-directory on /databases/***/_delta_log/_opt due to java.util.concurrent.ExecutionException: com.databricks.sql.managedcatalog.acl.UnauthorizedAccessException: PERMISSION_DENIED: User does not have MODIFY on Table '***'.
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.AbfsClient.appendSASTokenToQuery(AbfsClient.java:1304)
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.AbfsClient.appendSASTokenToQuery(AbfsClient.java:1270)
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.AbfsClient.createPath(AbfsClient.java:467)
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.createDirectory(AzureBlobFileSystemStore.java:821)
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.mkdirs(AzureBlobFileSystem.java:900)
at com.databricks.common.filesystem.LokiFileSystem.mkdirs(LokiFileSystem.scala:279)
at com.databricks.sql.acl.fs.CredentialScopeFileSystem.mkdirs(CredentialScopeFileSystem.scala:260)

4 REPLIES 4

Amit_Dass_Chmp
New Contributor III

shallow (also known as Zero-Copy) clone only duplicates the metadata of the table being cloned; the data files of the table itself are not copied. This type of cloning does not create another physical copy of the data resulting in minimal storage costs.

Thus, if you do not have the MODIFY permission on the source table, you will not be able to create a shallow clone of that table.

You can try either copying with a CTAS command or Deep Clone, and check if its helpful for your use case

 

 

Hi,

Thanks for response. Do you know what kind of MODIFY operations are executed on the source table during the shallow clone operation ?

Thanks

Amit_Dass_Chmp
New Contributor III

Also check this documentation on access mode :Shallow clone for Unity Catalog tables | Databricks on AWS

Working with Unity Catalog shallow clones in Single User access mode, you must have permissions on the resources for the cloned table source as well as the target table

Hi,

This is rather not my case. We are running shallow clone on compute with shared access mode not on single user access mode. So I still don't know why is require MODIFY permission on source table to be able run shallow clone.

Thanks,

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!