If you're catching up on Data + AI Summit 2026, here’s a simple Unity Catalog recap for data governance. Three themes stand out: Control, Context, and Choice. Control: Keep governance consistent as AI moves from access to action. AI governance at Dat...
no information on character_maximum_length and is_identity in unity catalog table system.information_schema.columnstable definition exampleCREATE TABLE tyable_name ( UniqueID bigint not null GENERATED ALWAYS AS IDENTITY , ID varchar(256))looks like ...
if you look at the official document here: https://docs.databricks.com/gcp/en/sql/language-manual/information-schema/columnsmany of them are "Always NULL, reserved for future use."so I think the feature simply isn't available to use yet. It's seems i...
i try to find out SQL for RLS , the databricks geni give this feedback "In Databricks SQL, the CREATE ROW ACCESS POLICY statement is not supported" . it means we can set RLS (role level security) for managed table or external table ?
We have RLS applied on a table and we would like to enable lake house monitoring for a quality profile metrics ,My Questions:Which identity is used by Lakehouse Monitoring and how can I find it, so that I can exclude it for unrestricted access to tab...
Hi @karunakaran_r In Databricks Lakehouse Monitoring, the profiling and drift metric collection runs as a service principal that’s tied to the Databricks system itself, not as your own user account.That means when the monitor queries your table, it w...
Hi,I am working with Databricks Serverless Compute and All-Purpose Compute and have several questions regarding tagging, system tables, and budget policies. Specifically, I need clarification on how these work together in Databricks, and how I can ma...
I am now more inclined towards the serverless and tagging, would be more helpful if you could take a real time use case and elaborate on it. Ty
Unity Catalog: create the first metastoreThe great benefit of the Unity catalog is that data is ours and stored in an open format on cloud storage in our container. To install the unity catalog, we need to create storage and give databricks access to...
We are in the process of implementing Row-Level Security (RLS) on a table in Databricks. As per our architecture, data ingestion is handled via Structured Streaming using an ADF (Azure Data Factory) pipeline.However, we are encountering the following...
@karunakaran_r Are you trying to read or write to a table with RLS/CM enabled using a file path directly instead of specifying the table name? You cannot specify the table path for reading and writing tables with RLS/CM enabled. You would have to rea...
How to Create Cluster that should have access to AWS Glue Catalog in different account
I mean to say I need to allow the role which I am using in my databricks cluster in my account where I want access the glue catalog
I wonder if i use databricks free version not free trail , can connect external location.follow set up the test only self assume role fail
Hello @FanMichelleTW! Databricks Free Edition does support external locations via Unity Catalog.Regarding Self assume role, check this out: https://community.databricks.com/t5/product-platform-updates/update-your-uc-aws-iam-roles-to-include-self-assu...
I have a weird issue with Databricks.I'm using the query history API:https://docs.databricks.com/api/workspace/queryhistory/listWhen I'm using the API with Personal Access Token, it works great.When i'm using the API with Service Principal, i'm getti...
@SP_6721 I don't filter by user ids in both scenarios, it shouldn't matter@nayan_wylde I also tried with the SDK but it's the same API call behind the scenes. I made sure it's not a pagination issue.@jhonm_839 This is so weird. PAT and SP have the sa...
Dear all,We are working on the column masking topic recently, using the column filter feature of Unity Catalog.We recently face the problem of masking a nested column (a sub-column within a STRUCT type column).We just wonder if this is even possible ...
Hi Community,We are working on implementing Databricks cluster policies across our organization and are seeking advice on best practices to enforce governance, security, and cost control across different environments.We have two main teams using Data...
Hi Team,I believe these are a few suggestions that can help!Start with environment-based policies: dev, qa, prodThese policies define the broadest guardrails (security, cost control, stability)Add team-specific variants only if requiredFor example: p...
We have a Delta Share that includes a partitioned table, and we want our recipient to be able to retrieve data only from a specific partition.I reviewed the Delta Sharing server documentation and found that it's possible to use SQL expressions for fi...
Hi @Col1ns ,All the things mentioned by @lingareddy_Alva are correct. Some additional things to consider. Check if your implementation of Delta Sharing supports predicate hints. Also, if your server supports them then your predicate should use restri...
I have nested JSON objects stored as values in some table columns. The table has a few top-level columns, and inside those, there are further nested JSON structures.When I write queries in the new SQL editor, the autocomplete correctly lists the firs...
@yit I could think of these possible reasons.1. Autocomplete relies on metadata to provide suggestions. Ensure the workspace is enabled for Unity Catalog and that the user has the necessary permissions to access the metadata for the tables containing...
Hi everyone, As part of our plans to improve the Discover Page, one of the next items we are looking into is building out Domains. 1. How have are your organizations using Data Domains today? 2. Can you share an example of your domain (and subdomain...
Hi @PeterWang,One of my clients in the financial services space groups their data using a domain-centric approach like customer, deposit, wealth, and credit.We tag tables and views with these domains and use that for RBAC, so access is granted based ...
When I run CREATE OR REPLACE VIEW on an existing view in Unity Catalog, the grants that were made on that object are removed. This seems like it is a bug. Is it on purpose or not?How to replicate:1. Create the viewRun the create or replace statement:...
Would also love any sort of communication back from Databricks on why create or replace view acts like this.I have had to run my CICD once with create (or replace) and then make a code change to swap to "alter view," which per https://docs.databricks...
