Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
When I look at the permissions for a specific catalog in Catalog Explorer, I see the Application ID from Entra ID instead of the Service Principal name defined in the Databricks account for some Service Principals, while for others, I see the Service...
Thank's for your response. I think I resolved it:The Service Principal was added via the Databricks Console with a display name and application ID, but it was not added to the Databricks Workspace that I used to display permissions for that catalog, ...
Hello!Our team wish to implement just-in-time access to the audit log, where we thought we could use row filter for the audit log, such that the different teams can see their own events, and afterwards revoke their right to see it.Upon first inspecti...
I want to try Lakehouse monitoring in Unity Catalog.But it was failed in three table patterns.All of the drift metrics and profile metrics are No Data.I used the sample data "nyc taxi trips"And I configured the details like the following image.
Hi @Yuki The reason you cannot see anything in the result tables is that Lakehouse monitoring only analyzes data from 30 days before its creation. Please try using a source table that has more recent timestamps. When you first create a time series...
Respected Concern,We are currently using unity catalog in our workspace but we still see legacy hive_metastore under catalog section. Is there any way we can remove it. Because the issue we are facing is our cluster still somehow tries to connect wit...
Hi @DSoni, By default hive_metastore uses an RDS in AWS, in the failure you can see: xyz.region.rds.amazonaws.com which I think you redacted, that is the host with the issue. I would require more details to check hive_metastore comes into picture if ...
From my understanding, a user needs to own the object in order to alter the comments of columns/tables. A user would also need ALTER TABLE permissions in order to make the change. Is there a way to permit a user to change comments without giving them...
Hi @mrosman,As far as I understand, you can only provision SELECT / MODIFY / APPLY TAG to a table. And there is no specific privilege as of now in UC where you can assign a permission to alter the table / column comments. If the user have SELECT priv...
Are manual processes slowing down your business? Struggling with data silos and security concerns while scaling AI? Without the right automation and governance, AI adoption can be challenging.Join us for an exclusive live webinar on March 28th, 2025,...
In Unity Catalog, we can specify the managed location for a schema while creation.However, say if we migrate the tables and schemas from Hive metastore to UC without specifying managed location while creating the schema in UC.Is there a way to update...
Hi @bhanu_dp In Databricks Unity Catalog, once a schema is created without specifying a managed location, you cannot directly alter the schema to set or change its managed location. The managed location is typically set at the time of schema creation...
Hi everyone, We're considering bringing a section called "For You" in the catalog tree view into the Catalog Explorer tab. Right now this section surfaces active tables and favorited UC objects in authoring surfaces (SQL Editor, Notebooks, and Dashbo...
That makes a lot of sense! We're considering revamping it to not have the For You pills and to put the favorites up at the top
I am using Azure Databricks DBFS. I have stored some python scripts, notebooks in folders in DBFS root and have mounted ADLS Gen2 to DBFS. Currently I do not have Unity Catalog enabled and this was setup 3-4 years ago. I came across this article http...
Hello @humayun! DBFS mounts and root storage are deprecated, as they are not secure with the Unity Catalog and pose security risks. While there’s no official end-of-life date, Databricks discourages using mounts and root storage. Existing setups will...
We have many suscription with databricks workspaces active, but i cant create a metastore. The button is inactive.
Hi @DiegoRU ,Thank you for providing the details. You also need to have the role of global administrator on the azure subscription To create metastore, please login to this link https://accounts.azuredatabricks.net/login as admin and you should be ab...
Hi,we are trying to move some of our code from a ‘legacy’ cluster to a ‘Multi-node/ Shared' cluster so that we can start using Unity Catalog. However, we have run into an issue with some of our code, which calls Stored Procedures, on the new cluster....
We face the same issue. Our code depends heavily on stored procedures (best practice, no).Unfortunately, when moving to shared clusters and DBR 13.3, this does not work anymore.driver_manager = spark._sc._gateway.jvm.java.sql.DriverManager connection...
Hi,I deployed a Unity Catalog metastore some time back (screenshot below).But when I am trying to create a new one, the create metastore option is disabled. Hovering on the button shows the message: "You must have an active subscription to create a M...
Are you using Azure Databricks ?? have you recently migrated your classic Global Administration role to RBAC global admin in azure we had the same issue we fix it by creating metastore from CLI databricks metastores create {Name} --region {regi...
As written here: Compute access mode limitations for Unity Catalog | Databricks on AWSWhy is R language not supported?
Is there any update on when the timeline will come out? I am unable to use a shared cluster at work because shared computes do not have R as a supported language. Thanks
Hey Guyz, We need to scan the complete schema present at unity catalog to an external user or group of users without onboarding them on to our platform. Is there a way we can expose this details to them. Additionally, can we expose the sample dataset...
Hi @RohitKumar7 , Greetings! Looking at your request, i would like to confirm you that it would be possible to use the Delta sharing feature. Delta sharing feature lets you share data and AI assets with users outside your organization, whether or no...
Hello,I would like to know if there is direct access to the Databricks query history tables. For compliance issues, I would like to be able to create reports for something like: who has accessed a particular column in a table in the past 6 months. T...
Throwing it out there that my team is also looking for a way to easily do this and I don't think the above solutions really fit our need. We are currently in the process of migrating schemas (from one name to a better name, we didn't rename because w...
