Does column masking work with job clusters

[Harsha777]

Hi,

We are trying to experiment with the column masking feature.

Here is our use case:

1. We have added a masking function to one of the columns of a table
2. the table is part of a notebook with some transformation logic
3. the notebook is executed as part of a workflow with a job cluster
4. the job cluster configuration is updated to use dbr15.4LTS and also tried dbr16.0
5. the notebook fails while accessing the table with error 
   

We tried accessing the masked table with SQL warehouse, Serverless, All-purpose clusters (DBR15.4) and it worked fine without any issue.

So, the question is - are there any restrictions with the masking concept in using the job clusters? we couldnt find anything in the documentation.
Any inputs would be highly appreciated,

Regards,
Harsha

[Walter_C]

It seems that the issue is that the cluster you are using in single user cluster, can you try to modify the job cluster and make it shared?

[MoJaMa]

For this to work on single-user clusters you need the Filtering Fleet enabled which requires you to enable Serverless Compute (same infra as Serverless Workflows / Notebooks / DLT).

https://docs.databricks.com/en/release-notes/product/2024/october.html#fine-grained-access-control-o...

" In workspaces enabled for serverless compute, if a query is run on supported compute such as single user compute and the query accesses any of the following objects ..."