If you're catching up on Data + AI Summit 2026, here’s a simple Unity Catalog recap for data governance. Three themes stand out: Control, Context, and Choice. Control: Keep governance consistent as AI moves from access to action. AI governance at Dat...
As part of flawless data governance and data quality implementation, Data Drift metrics play a very important role. I have compiled a few Data Drift metrics like Jensen-Shannon Divergence Score, Correlation Breakdown Detection, Data Lineage Health an...
Hi,I am currently designing a PII governance framework to meet CCPA compliance requirements on Databricks. I understand that Databricks provides mechanisms such as VACUUM and Deletion Vectors combined with REORG … APPLY (PURGE) to permanently remove ...
For GDPR/CCPA deletion requests involving PII, I think the key is having a clearly defined process that identifies all locations where the user’s data may exist, including primary records, backups, logs, integrations, and derived data.It’s also impor...
Databricks introduced several governance capabilities for the agentic era, including Unity Catalog Business Semantics, AI Gateway, and agent-ready governance.My question is:In a large enterprise with 20+ business domains, how are you governing:Agent ...
One thing I’d be interested in hearing from teams running this across multiple business domains is how they handle business ownership and consistency of definitions. Technical access controls can restrict what an agent can see or invoke, but how are ...
There's a lot of talk about "agent-ready data governance" right now, and the mechanics are well covered on this board: service principals per agent, Unity Catalog grants, AI Gateway guardrails, MCP tools as securables. The layer that sits on top of a...
I'm designing an enterprise Unity Catalog RBAC model in Databricks and would like feedback on whether this follows best practices.My current design is:A single Service Principal is used by Terraform to provision all Unity Catalog objects.The Service ...
I agree with Lu, this is a solid enterprise baseline and the two replies above cover the design mechanics well. One thing worth adding from running this pattern at scale is the evidence layer: a single Terraform SP with ownership transfer is clean to...
Hello everyone, We are planning to roll out Genie Spaces as a BI Agent and Genie Code as a Pair Programmer for multiple user groups. Our current Genie guardrails are set by inheriting workspace policies and Unity Catalog permissions with ABAC. Additi...
While creating Databricks Dashboard , can we control the access like a manager can see all employee performances and kpi metrics under it , but an employee can see only its own performance dashboard and its related metrics .Thanks,Pragya Sharma
Yes, you can Dynamic Views for that. Putting an example below:Steps:1. Create an employee-access mapping table containing:viewer_emailemployee_email2. Create a secured view or apply a Unity Catalog row filter: CREATE OR REPLACE VIEW hr.employee_perfo...
Environment:Cloud: AzureDatabricks Account Console (accounts.azuredatabricks.net)Access Connector: system-assigned managed identity (no user-assigned identity involved)Region: East USIssue:Creating a Unity Catalog storage credential via the Account C...
Thanks for pointing me in the right direction! That was indeed the issue.For anyone else who runs into the same problem, here's what worked for me using the Databricks Account CLI.1. Install Databricks CLIwinget install Databricks.DatabricksCLI2. Log...
Hi All, I'm trying to solve two related problems on our lakehouse and I keep arriving at data contracts as the answer, but I can't find much on what a first working version looks like in practice on Databricks:(a) Upstream application teams change th...
A practical MVP is to keep Bronze permissive for ingestion, but enforce contracts from Silver onward. Store the contract in Git, version it alongside pipeline code, validate it in CI using the Data Contract CLI, and publish approved changes with a si...
As a way to minimize storage costs, my team and I want to create views instead of tables in the Gold layer.We always try to improve the experience of our users by adding comments to the columns. the problem is that views do not inherit comments from ...
Why are COMMENT ON statements and inline COMMENT clauses not supported for non-materialized views defined in Lakeflow Declarative Pipelines?Databricks SQL supports view-level and column-level comments, but the same functionality does not appear to be...
We're deploying a Databricks App using Service Principal (SP) mode (not using OBO's user_api_scopes).The sample query "List app actions performed on behalf of users" in Logging and Monitoring for Databricks Apps shows system.access.audit.user_identit...
Hi, Everything @binlogreader has commented above is correct, the app service principal will be what appears in the system tables and if you'd like to get individual user authentication you'll need to use OBO authorisation, or pull somthing from the ...
Hi I have a fabric semantic model. I have created a fabric data agent (published) on top of the semantic model. Now i have created a unity catalog connection along with service. Is MCP connection is true. I can use this connection in AI playground or...
This seems to be a duplicate question, please see my answer on the other thread.
We have a Lakeflow (DLT) pipeline that runs as a task inside a job. A service principal is set as the run-as identity.The pipeline originally created its streaming tables and a metadata Delta table (which gets updated in a separate task in the job) i...
@keshavmonga22 Is there anyone in your team with Admin access that can grant you temporary access or can check the event log?
Hi there,I'm not sure if this is a glitch or something changed, but when trying to setup a new Data Ingestion Custom Connector, I am running into an issue at the connection page due to not having CREATE_CONNECTION permission.The thing is, I don't wan...
Hi ChristianRRL,Most likely a classic UI race condition or state management glitch with the UI which is common as Community Custom Connectors are in beta and under active development. Interfaces and behavior are subject to change.What's likely happen...
Hello Databricks Community,Databricks is making great progress towards becoming a truly governed and unified catalog. When it comes to governing and cataloguing physical data assets, it already offers a strong foundation. With the introduction of met...
HiThis is a really practical explanation of how governance works in the real world. Centralizing the semantic layer and making governed data objects the source of truth seems much more reliable than letting everyone create their own datasets. The rep...
