How to Retrieve Masked Column Information in Unity Catalog
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-05-2025 05:55 AM
Hi all,
I’ve been working with Unity Catalog in Databricks and using column masking policies for sensitive fields like SSNs. While trying to retrieve metadata about which columns have masking policies applied, I ran into some challenges and wanted to share my findings + get input from the community.
✅ Tried Approaches
DESCRIBE EXTENDED <table>
This shows masking policies applied on individual columns under the "Comment" or "Metadata" fields — but it’s not structured and hard to automate for large-scale scans.<catalog>.information_schema.column_masks
Surprisingly, this view is empty in some client environments — even when I know masking policies are in place and functional. I'm assuming it could be due to:Delayed metadata sync
Lack of permissions
Feature not fully available in that workspace
❓ Looking for a Better Way
Aside from DESCRIBE EXTENDED and the information schema view:
Is there a more reliable way to programmatically retrieve which columns have masking policies applied?
Does anyone have experience using Databricks REST API?
Any internal Unity Catalog system tables or best practices for scanning a catalog/schema and reporting masked fields?
Would appreciate any guidance, alternate approaches, or scripts you’ve found useful in your projects.
Thanks in advance!
—Vyshnavi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-07-2025 12:06 AM - edited 06-07-2025 12:08 AM
@jv_v
There are a few more approaches that you can try which are ideal for large scale scanning,
Try using, Unity Catalog System Tables via system.information_schema Instead of <catalog>.information_schema.column_masks
If that doesnt work, use Databricks REST API, GET /api/2.1/unity-catalog/tables/{full_name}.
Reference.: Get a table | Tables API | REST API reference | Databricks on AWS