If you're catching up on Data + AI Summit 2026, here’s a simple Unity Catalog recap for data governance. Three themes stand out: Control, Context, and Choice. Control: Keep governance consistent as AI moves from access to action. AI governance at Dat...
If a user has Can Run permission on a notebook, why are they still able to view the notebook's contents without having Can View permission?Restricting visibility while allowing execution could be useful in scenarios where sensitive configuration deta...
This is by design: Can Run always includes viewing the cells. Notebook access has a single view-only level (labeled CAN VIEW in the UI, CAN READ in the Permissions API) that sits below Can Run, and %run and notebook workflows execute at that view-onl...
I have a table pulled from Snowflake via a foreign catalog that's only returning 100 rows in Databricks even though there's billions of records in the source Snowflake table. What steps should I take to diagnose the limitation of rows?
Hi Community,I'm currently facing an issue with the system.access.column_lineage system table.My goal is to assess the downstream impact of removing a column from a source table. I tried to use the column_lineage table for this purpose, but it seems ...
Hi @antoine_pac, I think this is expected behaviour, and you have already identified the reason in your query. The public docs for lineage system tables explicitly say that both lineage tables represent only a subset of read/write events, and that "...
I have created an Azure data factory pipeline with a copy data function to copy data from adls path to a delta table .In the delta table drop downs i am able to see only the hive metastore database and tables only but the unity catalog tables are not...
Hi, Anybody have an update on this? I also use the managed identity and i receive the same error as the last two people mentioned here.
Hi everyone,I'm currently working on implementing a data dictionary, and I'm struggling to understand what the best approach is when using a Medallion architecture in Databricks.My company is migrating its data to Databricks so that users can consume...
We are implementing Atlan to provide a business glossary to users and agents. We're ingesting multiple sources, primarily Databricks and SQL Server based. For data with a published definition we copy those into table and column comments. We also u...
Hi all, We're hitting a sudden failure with Databrick's built-in python sandbox system.ai.python_exec function and want to check whether others are seeing the same thing, since it looks region-specific.Environment:Cloud: Azure DatabricksRegion: West ...
Experiencing this issue as well, started some time prior to June 1st. ```USE CATALOG system;USE SCHEMA ai;SELECT python_exec('print("hello")');``` Returns:```[UNRESOLVED_ROUTINE] Cannot resolve routine `python_exec` on search path [`system`.`session`...
If you're catching up on Data + AI Summit 2026, here’s a simple Unity Catalog recap for data governance. Three themes stand out: Control, Context, and Choice. Control: Keep governance consistent as AI moves from access to action. AI governance at Dat...
I have one task for you which is related to unity catalog I want to implement row and column level security on databricks for few tables please let me know different kind of approaches and methods.We are using azure cloud and databricks witj unity c...
Hi Rupa,One thing worth adding from running both dynamic views and column masking in production - the table above is a good way to think about the mechanics, but in practice the decision usually comes down to how many things are querying the same dat...
ABAC in Unity Catalog already gave teams a better way to apply row filters and column masks at scale. Instead of rebuilding the same access logic table by table, teams can classify data with governed tags and let policies apply automatically across a...
I am using Azure Databricks DBFS. I have stored some python scripts, notebooks in folders in DBFS root and have mounted ADLS Gen2 to DBFS. Currently I do not have Unity Catalog enabled and this was setup 3-4 years ago. I came across this article http...
In current state, when DBFS is disabled in a workspace, both the root filestore and mount paths become inaccessible.What is deprecated are:DBFS root storage the workspace‑provisioned filestore - /dbfs/FileStore/, /dbfs/root/ etc.DBFS mounts (/dbfs/mn...
I have a Foundation Model provisioned throughput endpoint with 50 provisioned model units. It is READY but has received zero inference requests. Yet system.billing.usage shows DBUs accruing continuously from creation. -- Cost is accruingWITH usage_ag...
Hello @nara_shikamaru , I did some digging and here is what I found. Short answer first: what you're seeing is expected behavior for Provisioned Throughput (PT), not a billing bug. Let me walk through why, because the two queries you ran measure diff...
Hi, I have an Azure Databricks Premium workspace (upgraded from Standard today) and cannot enable Unity Catalog because my Azure subscription was created with a personal Microsoft account (live.com#ushakaleclouddba@outlook.com). Workspace details:- W...
Hi,You can log into the Azure Portal using personal outlook account, navigate to Microsoft Entra ID and create a new cloud user directly within default tenant directory. Because the primary subscription email is an external identity, the directory do...
Is it possible to let a group in databricks to change a delta table, but not the table’s schema in UC? i.e., they can add or remove rows of the delta table, but cannot change the name or the format of the columns.
Granting only MODIFY on the table allows the group to insert, update, and delete rows. MODIFY grants the ability to change the contents of a table without altering its schema or metadata - it does not include schema-altering privileges like ALTER or ...
In the article Manage budgets and cost controls for Genie at Manage budgets and cost controls for Genie - Azure Databricks | Microsoft Learn it mentions setting the resource type of Unity AI gateway from the dropdown. However when using Account Conso...
Hi @AndyRoyle, Per the public docs, the expected flow is that once that preview is enabled, you go to the Budgets page in Account Console, and the budget definition should show a Resource types dropdown where you can choose Unity AI Gateway. You can ...
Hi everyone,Don't know where to put this specific question. I'm working on a reference data mesh implementation to connect and combine datasets to find matching trials for patients with a specific genetic profile. - Do you know anyone that might be i...
Hi @steff_horemans, My guess is that the Free Edition is probably not the best place to demonstrate true external sharing. Free Edition is positioned as a serverless-only, quota-limited, non-commercial environment, with one workspace, one metastore, ...
