topic Re: How to Retrieve Masked Column Information in Unity Catalog in Data Governance

How to Retrieve Masked Column Information in Unity Catalog

jv_v — Thu, 05 Jun 2025 12:55:07 GMT

Hi all,

I’ve been working with Unity Catalog in Databricks and using column masking policies for sensitive fields like SSNs. While trying to retrieve metadata about which columns have masking policies applied, I ran into some challenges and wanted to share my findings + get input from the community.

✅ Tried Approaches

DESCRIBE EXTENDED <table>
This shows masking policies applied on individual columns under the "Comment" or "Metadata" fields — but it’s not structured and hard to automate for large-scale scans.
<catalog>.information_schema.column_masks
Surprisingly, this view is empty in some client environments — even when I know masking policies are in place and functional. I'm assuming it could be due to:
- Delayed metadata sync
- Lack of permissions
- Feature not fully available in that workspace

❓ Looking for a Better Way

Aside from DESCRIBE EXTENDED and the information schema view:

Is there a more reliable way to programmatically retrieve which columns have masking policies applied?
Does anyone have experience using Databricks REST API?
Any internal Unity Catalog system tables or best practices for scanning a catalog/schema and reporting masked fields?

Would appreciate any guidance, alternate approaches, or scripts you’ve found useful in your projects.

Thanks in advance!
—Vyshnavi

Re: How to Retrieve Masked Column Information in Unity Catalog

niteshm — Sat, 07 Jun 2025 07:08:03 GMT

@jv_v
There are a few more approaches that you can try which are ideal for large scale scanning,

Try using, Unity Catalog System Tables via system.information_schema Instead of <catalog>.information_schema.column_masks

If that doesnt work, use Databricks REST API, GET /api/2.1/unity-catalog/tables/{full_name}.

Reference.: Get a table | Tables API | REST API reference | Databricks on AWS

Re: How to Retrieve Masked Column Information in Unity Catalog

DoTA — Tue, 22 Sep 2026 13:19:27 GMT

To build on what @niteshm said - the reason <catalog>.information_schema.column_masks came back empty for you is almost certainly not sync delay or permissions, it's the query scope. Per-catalog information_schema only reflects masks defined within that catalog's own metastore context. Query it from the SYSTEM catalog instead:

SELECT * FROM system.information_schema.column_masks WHERE catalog_name = '<your_catalog>'

That's the documented, cross-catalog view and it's the one that's actually reliable for scanning at scale.

One more thing worth checking if that still comes back empty: are your masks applied via ABAC policies (governed tags, catalog/schema-level policies that apply automatically) rather than manual UDF assignment directly on the column? Databricks has two distinct mechanisms now - manual column mask UDFs bound per-column, and newer ABAC policies that apply based on tags across whole catalogs/schemas. column_masks reflects the manual per-column bindings; if your masking is coming from an ABAC policy, it won't show up there at all, since it's not a per-column assignment in the same sense. For that case you'd want to enumerate policies directly - DESCRIBE POLICY on a specific table/policy shows the details, and it's worth checking your workspace's ABAC policies documentation for how to list all active policies rather than assuming column_masks is exhaustive.

So the practical order I'd check in: query system.information_schema.column_masks (not the catalog-scoped one) first, and if it's still empty despite known masking, that's a strong signal you're on ABAC-based masking rather than direct UDF assignment and need the policy-listing path instead.