Error [UNAUTHORIZED_ACCESS]: Merging into a table with masked columns

thedatanerd
Contributor

‎09-17-2025 06:37 AM

Hi,

I'm facing a weird error when trying to incrementally merge into a table with a masked column.

Table is external, but catalog is managed. Using Unity Catalog.

Error is:

[UNAUTHORIZED_ACCESS] Unauthorized access: PERMISSION_DENIED: Path-based access to table catalog.schema.table with row filter or column mask not supported. SQLSTATE: 42501

It is when doing:

merge_stats = (
    275     current_unity_delta_table.alias("current")
    276     .merge(upsert_df.alias("upsert"), merge_condition)
    277     .withSchemaEvolution()
    278     .whenMatchedUpdateAll()
    279     .whenNotMatchedInsertAll()
--> 280     .execute()
    281 )

Checking these posts 1,2 I tried changing the job cluster type to be Standard (formerly Shared), Dedicated (formerly: Single User) and also tried changing runtime to 17.2, 16.4,15.4 but nothing worked. Always throws same error.

Is there a tested and tried solution to this weird limitation? shouldn't Databricks handle this automatically?

 

Path based access not supported for tables with row filters? 

path-based access to a table with row filters or column masks is not supported 

Data
0 Kudos

K_Anudeep
Databricks Employee
Databricks Employee

‎09-18-2025 08:11 AM

Hello @thedatanerd ,

Tables with row filters or column masks can’t be accessed by path. Delta Lake’s object APIs (e.g., DeltaTable + .merge()) aren’t supported for these tables.This is expected by design .Run the merge via SQL against catalog.schema.table so Unity Catalog can enforce policies. Example: register your DataFrame as a temp view and do a MERGE INTO

 

Anudeep

thedatanerd
Contributor

‎09-18-2025 10:02 AM

Thank you @K_Anudeep I was inspired by your comment and I tried to use DeltaTable.forName instead of DeltaTable.forPath and it worked.

Data

View solution in original post

0 Kudos

K_Anudeep
Databricks Employee
Databricks Employee

‎09-18-2025 10:10 AM

Hello @thedatanerd ,

Thanks for the reply. Yes, you can access that by DeltaTable.forName if you know the table name or use MERGE into using SQL. If your question is answered,could you please accept this as a solution, as it would be useful to others in the community?

Thanks.

Anudeep
0 Kudos