If you're catching up on Data + AI Summit 2026, here’s a simple Unity Catalog recap for data governance. Three themes stand out: Control, Context, and Choice. Control: Keep governance consistent as AI moves from access to action. AI governance at Dat...
I am excited to share that I recently published a whitepaper on how to leverage Databricks to build a robust data governance framework. This paper explores key strategies and best practices for implementing a strong governance model using Databricks'...
Hey @ArunKhandelwal , Thanks for sharing this with the community! Data governance is top of mind for so many of our customers, and having practical guidance on how to bring policy and control frameworks into Databricks is incredibly valuable. I’m lo...
We have been using the feature MLflow Model Registry Webhooks on Azure Databricks . Webhooks are in public preview, but oddly are deprecated at the same time with the introduction of Unity Catalog. The documentation says "For an alternative, see Can ...
Greetings @bragoarefur , Thanks for laying out the context so clearly—your read of the current state is spot on. Below is what the Databricks docs actually say, and a recommendation with an implementation path that avoids polling while staying cloud...
Hi everyone!I would love to know whether Databricks has the following capabilities and what are the core functionalities of the tool:Data glossary: provides clear definitions of business terms, ensuring that everyone in the organization uses the same...
Hi @DG1 - You can use AI to generate comments for your Unity Catalog Database objects. For the Data glossary, you can have specific roles in your organization to establish those standards. We do have business semantics (UC metrics Views) for a centra...
Hi everyone,I want to apply Row Filters to restrict access to Unity Catalog tables using a dedicated cluster running Databricks Runtime 15.4 LTS (Spark 3.5.0, Scala 2.12).I followed the official documentation which says that row filters are supported...
Hi @ESN, let's first understand why you need to use a dedicated cluster with FGAC (Fine-Grained Access Control). You should ideally use a standard cluster where you do not need the serverless component for filtering. Is there a specific piece of your...
I have been tasked with governing the data within our Databricks instance. A large part of this is adding Comments or Descriptions, and Tags to our Schemas, Tables and Columns in Unity Catalog.For most objects this has been straight-forward, but one ...
It is currently not possible to reliably add or persist comments or descriptions directly to Streaming Tables created via the SQL Server Data Ingestion Wizard in Databricks using the Data Ingestion UI or Jobs & Pipelines UI. All metadata management—i...
Hi Databricks team,Quick heads-up and request for guidance. While collaborating on a notebook in my personal workspace, a colleague who had READ permission on a workspace-backed Secret Scope was able to delete a secret key via terminal/CLI from the s...
Hi @guidotognini - This is not expected behavior if your colleague truly has only READ and their CLI was using their own identity; deletion requires WRITE/MANAGE on the scope. The most probable explanation is that the CLI call was authenticated as an...
Hello Team,Could you please let me know what Databricks recommended way to move a job from lower env to higher , especially when certain parameters will be dynamic with respect to env .DAB support templating ?
@arun_6482 Please accept the solution if your issue is solved, thank you!
The future of data and AI sharing should be open, secure, and seamless not locked into costly proprietary systems. What is Delta Sharing?Delta Sharing is the world’s first open protocol for secure, real-time data sharing at scale, developed by Databr...
Hello! Newbie here, so apologies if this is a super basic question. Trying to figure out if we can connect Purview to an AWS instance of Databricks (vice Azure instance), but I have only seen articles on connecting Azure Databricks to Purview. I r...
@ShaSap wrote:Hello! Newbie here, so apologies if this is a super basic question. Trying to figure out if we can connect Purview to an AWS instance of Databricks (vice Azure instance), but I have only seen articles on connecting Azure Databricks to...
Hi folks,How encryption is handled for UC managed tables on AWS- based Databricks account? I found this post: https://docs.databricks.com/aws/en/security/keys/ , which describes how workspace file system (dbfs) and control panel data is encrypted, bu...
I stored my artifact of model in volume.I can view the object via catalog page.however, when i open from experiment > artifact, i can't access to view my artifact with error failed to fetch.Any idea to solve the problem?
Hello @tana_sakakimiya! Artifacts stored in Volumes can’t be viewed directly in the MLflow experiment UI, as it only supports displaying artifacts saved to DBFS. You can instead access or download artifacts through Catalog Explorer or using mlflow.ar...
Where does the Unity Catalog metastore- metadata actually reside?Is it stored and managed in the Databricks account (control plane)?Or does it get stored in the customer-managed S3 bucket when we create a bucket for Unity Catalog metastore?I want to ...
1. Unity Catalog metadata (schemas, tables, permissions, lineage, etc.) is stored and managed in the Databricks control plane, not in your S3 bucket. This metadata resides in a Databricks-managed database and is not in your customer-managed storage.2...
Hi everyone,I'm running into an issue when requesting permissions on a Unity Catalog table. Specifically, when trying to request SELECT and MODIFY privileges on a specific table, using Request for Access in Unity Catalog. I get the following error:th...
Hi @boitumelodikoko ,Weird, it looks like a bug. MODIFY is a valid permission for table object. You can check it yourself in docs:Unity Catalog privileges and securable objects | Databricks on AWSModify allows user to add, update, and delete data to ...
I'm currently working with Databricks within Azure and encountering an issue with Unity Catalog in a newly created workspace.My colleague initially deployed a Databricks workspace where Unity Catalog was provisioned and enabled by default. I recently...
Hello like szymon suggested just try to check with the admin also as you share vnet injected workspace was it done recently if yes! ThenMight be you were missing two private endpoint of storage account you needed to create 2 private endpoints, both ...
Hi Everyone ,while I am creating sync table in UC, I got an error and failed . then I am not able to recreate the same table.when I checked the catalogue table name is there but not able to open , and I am querying the same in the notebook .it's givi...
@ajai_duraisamy Here are few checks you can do:1. SHOW TABLES IN <catalog>.<schema> LIKE '<table>'; This command will show if the table exists in UC.2. Make sure your compute is UC‑enabled.If you’re on a cluster/SQL Warehouse without UC enabled (or c...
