How to Retrieve Masked Column Information in Unity Catalog

[jv_v]

Hi all,

I’ve been working with Unity Catalog in Databricks and using column masking policies for sensitive fields like SSNs. While trying to retrieve metadata about which columns have masking policies applied, I ran into some challenges and wanted to share my findings + get input from the community.

✅ Tried Approaches

1. DESCRIBE EXTENDED <table>
   This shows masking policies applied on individual columns under the "Comment" or "Metadata" fields — but it’s not structured and hard to automate for large-scale scans.

2. <catalog>.information_schema.column_masks
   Surprisingly, this view is empty in some client environments — even when I know masking policies are in place and functional. I'm assuming it could be due to:

   • Delayed metadata sync

   • Lack of permissions

   • Feature not fully available in that workspace

❓ Looking for a Better Way

Aside from DESCRIBE EXTENDED and the information schema view:

• Is there a more reliable way to programmatically retrieve which columns have masking policies applied?

• Does anyone have experience using Databricks REST API?

• Any internal Unity Catalog system tables or best practices for scanning a catalog/schema and reporting masked fields?

Would appreciate any guidance, alternate approaches, or scripts you’ve found useful in your projects.

Thanks in advance!
—Vyshnavi

[niteshm]

@jv_v 
There are a few more approaches that you can try which are ideal for large scale scanning, 

Try using, Unity Catalog System Tables via system.information_schema Instead of <catalog>.information_schema.column_masks

If that doesnt work, use Databricks REST API, GET /api/2.1/unity-catalog/tables/{full_name}.

Reference.: Get a table | Tables API | REST API reference | Databricks on AWS

[DoTA]

To build on what @niteshm said - the reason <catalog>.information_schema.column_masks came back empty for you is almost certainly not sync delay or permissions, it's the query scope. Per-catalog information_schema only reflects masks defined within that catalog's own metastore context. Query it from the SYSTEM catalog instead:

SELECT * FROM system.information_schema.column_masks WHERE catalog_name = '<your_catalog>'

That's the documented, cross-catalog view and it's the one that's actually reliable for scanning at scale.

One more thing worth checking if that still comes back empty: are your masks applied via ABAC policies (governed tags, catalog/schema-level policies that apply automatically) rather than manual UDF assignment directly on the column? Databricks has two distinct mechanisms now - manual column mask UDFs bound per-column, and newer ABAC policies that apply based on tags across whole catalogs/schemas. column_masks reflects the manual per-column bindings; if your masking is coming from an ABAC policy, it won't show up there at all, since it's not a per-column assignment in the same sense. For that case you'd want to enumerate policies directly - DESCRIBE POLICY on a specific table/policy shows the details, and it's worth checking your workspace's ABAC policies documentation for how to list all active policies rather than assuming column_masks is exhaustive.

So the practical order I'd check in: query system.information_schema.column_masks (not the catalog-scoped one) first, and if it's still empty despite known masking, that's a strong signal you're on ABAC-based masking rather than direct UDF assignment and need the policy-listing path instead.