Limitations for Unity Catalog on single user access mode clusters

[Nes_Hdr]

Hello! 

According to Databricks documentation on azure :

"On Databricks Runtime 15.3 and below, fine-grained access control on single user compute is not supported. Specifically:

• You cannot access a table that has a row filter or column mask.
• You cannot access dynamic views.
• To read from any view, you must have SELECT on all tables and views that are referenced by the view."

Well, I am using DBR 15.4 LTS and getting the following error message when running an optimize job (OPTIMIZE table name) on some tables that have a row filter:

Py4JJavaError: An error occurred while calling o388.sql. : java.util.concurrent.ExecutionException: com.databricks.sql.managedcatalog.UnityCatalogServiceException: [RequestId=d5be0519-**** ErrorClass=INVALID_PARAMETER_VALUE.ROW_COLUMN_ACCESS_POLICIES_NOT_SUPPORTED_ON_ASSIGNED_CLUSTERS] Query on table **** with row filter or column mask not supported on assigned clusters.

This is my job cluster configuration: 

Can you please help me figure out when could possible cause such an error ? 

Thanks a lot! 

[MuthuLakshmi]

@Nes_Hdr Do you have Serverless enabled for your workspace?

1. Fine-grained access control on single user compute: Databricks Runtime 15.4 LTS does support fine-grained access control on single user compute, but it relies on serverless compute to run data filtering. This means that when a query accesses objects like tables with row filters or column masks, the single user compute resource passes the query to the serverless compute to execute the data filtering.
2. Error Message: The error message you are encountering, ROW_COLUMN_ACCESS_POLICIES_NOT_SUPPORTED_ON_ASSIGNED_CLUSTERS, indicates that the query on the table with row filters or column masks is not supported on assigned clusters. This suggests that the single user cluster you are using might not be configured to leverage serverless compute for these operations.
   
   In the documentation it was mentioned that Serverless needs to be enabled in that workspace. 
   https://learn.microsoft.com/en-us/azure/databricks/compute/access-mode-limitations#assigned-limitati...
   
   Hope this helps!