topic Re: Shallow clone and issue with MODIFY permission to source table in Data Engineering

Shallow clone and issue with MODIFY permission to source table

cszczotka — Tue, 14 May 2024 14:31:26 GMT

Hi,

I'm running shallow clone for external delta tables. The shallow clone is failing for source tables where I don't have MODIFY permission. I'm getting below exception. I don't understand why MODIFY permission to source table is required. Is there any other solution than granting MODIFY permission to source tables ? I would like to avoid this in case of cloning delta tables from production sources.

Thanks,

: Failed to acquire a SAS token for create-directory on /databases/***/_delta_log/_opt due to java.util.concurrent.ExecutionException: com.databricks.sql.managedcatalog.acl.UnauthorizedAccessException: PERMISSION_DENIED: User does not have MODIFY on Table '***'.
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.AbfsClient.appendSASTokenToQuery(AbfsClient.java:1304)
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.AbfsClient.appendSASTokenToQuery(AbfsClient.java:1270)
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.AbfsClient.createPath(AbfsClient.java:467)
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.createDirectory(AzureBlobFileSystemStore.java:821)
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.mkdirs(AzureBlobFileSystem.java:900)
at com.databricks.common.filesystem.LokiFileSystem.mkdirs(LokiFileSystem.scala:279)
at com.databricks.sql.acl.fs.CredentialScopeFileSystem.mkdirs(CredentialScopeFileSystem.scala:260)

Re: Shallow clone and issue with MODIFY permission to source table

Amit_Dass_Chmp — Tue, 14 May 2024 14:56:11 GMT

A shallow (also known as Zero-Copy) clone only duplicates the metadata of the table being cloned; the data files of the table itself are not copied. This type of cloning does not create another physical copy of the data resulting in minimal storage costs.

Thus, if you do not have the MODIFY permission on the source table, you will not be able to create a shallow clone of that table.

You can try either copying with a CTAS command or Deep Clone, and check if its helpful for your use case

Re: Shallow clone and issue with MODIFY permission to source table

cszczotka — Tue, 14 May 2024 15:21:56 GMT

Hi,

Thanks for response. Do you know what kind of MODIFY operations are executed on the source table during the shallow clone operation ?

Thanks

Re: Shallow clone and issue with MODIFY permission to source table

Amit_Dass_Chmp — Tue, 14 May 2024 15:56:45 GMT

Also check this documentation on access mode :Shallow clone for Unity Catalog tables | Databricks on AWS

Working with Unity Catalog shallow clones in Single User access mode, you must have permissions on the resources for the cloned table source as well as the target table

Re: Shallow clone and issue with MODIFY permission to source table

cszczotka — Tue, 14 May 2024 18:49:33 GMT

Hi,

This is rather not my case. We are running shallow clone on compute with shared access mode not on single user access mode. So I still don't know why is require MODIFY permission on source table to be able run shallow clone.

Thanks,