Governance RiskOps Agent for Unity Catalog

[WiliamRosa]

Body:

Every day, data platforms generate thousands of audit events. But here's the problem: security teams are drowning in noise.

Critical risks hide in plain sight. Manual investigations take hours. Compliance gaps surface too late. And there's no intelligent way to prioritize what matters.

I built a solution to fix this.

🚀 Introducing the Governance RiskOps Agent

An automated risk detection system for Databricks Unity Catalog that transforms raw audit logs into actionable security insights.

How it works:

✅ Continuous Monitoring → Ingests and enriches Unity Catalog audit events in real-time

✅ Smart Risk Scoring → Multi-dimensional algorithm scores every event from 0-100 using 9 risk factors: • Action type & permission level • Data sensitivity classification • After-hours access patterns • Privilege changes & cross-domain access • Failed attempts & external sources

✅ Actionable Findings → Not just alerts. Each finding includes: • Exact risk score & severity (CRITICAL/HIGH/MEDIUM/LOW) • Full context (who, what, when, why) • Specific remediation steps

The Architecture:

🏗️ Medallion pipeline (Bronze → Silver → Gold) • Bronze: Raw audit event ingestion • Silver: Normalization + dimensional enrichment • Risk Engine: 15+ detection rules with sophisticated scoring • Gold: 4 analytical tables ready for consumption

📊 AI/BI Dashboards with executive metrics (Governance Risk Index, critical findings, risky users)

💬 Genie Space integration for natural language investigation (no SQL required)

Real Impact:

In our demo with 327 realistic events, the system detected: • 86 CRITICAL findings (score 75-100) • 106 HIGH risk events (score 50-74) • 105 MEDIUM risk events (score 25-49)

Investigation time: from hours to minutes.

Production-Ready:

✨Deploys with Databricks Asset Bundles in a single command ✨ Open-source and enterprise-ready ✨ Works today with your Unity Catalog audit logs

🎥 Watch the 5-minute demo video to see the full solution in action → [Link to video]

💡This project was built for the DAIS 2026 Community Virtual Contest.

[BS_THE_ANALYST]

Hey @WiliamRosa!

Great contribution 😀. I have been so busy with client work the last couple of weeks that I didn't get a chance to get a challenge sorted for this 😴. Very jealous!

I love the concept of " transforms raw audit logs into actionable security insights". So much rich information is sitting in systems. 

From the video, this just looks really really cool 😀..

Can I ask, what was the development & planning process like to create something like this? Personally, I'd love to be able to develop something like this. I wonder if there's a WiliamRosa framework that I could follow 😎.

Thanks a bunch for sharing!

All the best,
BS

[WiliamRosa]

Hi @BS_THE_ANALYST 

Thank you for your comment.

I'd like to share the inspiration behind this scenario. The idea actually came from a real-world data governance initiative I worked on with ANBIMA, one of Brazil's leading financial market associations, responsible for establishing standards, certifications, and best practices across the financial sector.

In that project, we leveraged Databricks Genie Agents and custom Skills to validate whether data ingestion standards and governance policies were being followed correctly. We also created log, metrics, and KPI tables that allowed both dashboards and AI agents to identify inconsistencies, policy violations, and recurring offenders.

The Governance RiskOps Agent presented in this article is an evolution of that concept, applying similar governance principles to Unity Catalog audit logs and transforming technical events into actionable security and governance insights.

Thanks again for taking the time to read and engage with the article!

[Sumit_7]

Hey @WiliamRosa,
Good job creating the Insights out of raw audits.

[WiliamRosa]

@Sumit_7 Thank you! I appreciate your feedback.

[szymon_dybczak]

Hi @WiliamRosa ,

Good job! Really interesting use case. Thanks for sharing, and good luck in the contest!

[WiliamRosa]

@szymon_dybczak Thank you! I appreciate your feedback my friend.

All the best!