Governance RiskOps Agent for Unity Catalog
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-31-2026 06:13 PM - edited 05-31-2026 06:31 PM
Body:
Every day, data platforms generate thousands of audit events. But here's the problem: security teams are drowning in noise.
Critical risks hide in plain sight. Manual investigations take hours. Compliance gaps surface too late. And there's no intelligent way to prioritize what matters.
I built a solution to fix this.
🚀 Introducing the Governance RiskOps Agent
An automated risk detection system for Databricks Unity Catalog that transforms raw audit logs into actionable security insights.
How it works:
✅ Continuous Monitoring → Ingests and enriches Unity Catalog audit events in real-time
✅ Smart Risk Scoring → Multi-dimensional algorithm scores every event from 0-100 using 9 risk factors: • Action type & permission level • Data sensitivity classification • After-hours access patterns • Privilege changes & cross-domain access • Failed attempts & external sources
✅ Actionable Findings → Not just alerts. Each finding includes: • Exact risk score & severity (CRITICAL/HIGH/MEDIUM/LOW) • Full context (who, what, when, why) • Specific remediation steps
The Architecture:
🏗️ Medallion pipeline (Bronze → Silver → Gold) • Bronze: Raw audit event ingestion • Silver: Normalization + dimensional enrichment • Risk Engine: 15+ detection rules with sophisticated scoring • Gold: 4 analytical tables ready for consumption
📊 AI/BI Dashboards with executive metrics (Governance Risk Index, critical findings, risky users)
💬 Genie Space integration for natural language investigation (no SQL required)
Real Impact:
In our demo with 327 realistic events, the system detected: • 86 CRITICAL findings (score 75-100) • 106 HIGH risk events (score 50-74) • 105 MEDIUM risk events (score 25-49)
Investigation time: from hours to minutes.
Production-Ready:
✨Deploys with Databricks Asset Bundles in a single command ✨ Open-source and enterprise-ready ✨ Works today with your Unity Catalog audit logs
🎥 Watch the 5-minute demo video to see the full solution in action → [Link to video]
💡This project was built for the DAIS 2026 Community Virtual Contest.
Data Engineer | Machine Learning Engineer
LinkedIn: linkedin.com/in/wiliamrosa