Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
I am trying to connect to Unity catalog meta store tables using Presto Based on the presto documentation I need to use the below configuration to connect to delta tables in the unity catalog https://prestodb.io/docs/current/connector/hive.htmlSo from...
Hello @December , You should contact your account team if you need more info on this feature which is not GA for the time being. Another option would be to use Iceberg REST interface: https://medium.com/@flyws1993/oss-trino-to-read-tables-from-databr...
How can I get the unity catalog code base?
https://github.com/unitycatalog/unitycatalogThis is the OSS version of course. The databricks version is not available afaik.
I get an error message saying I Error getting sample data, when I try to view sample data from a table in a schema I created in a Unity Catalog. I dropped the schema and table and got a collague to recreate and still the same message. We are both Uni...
Check your table level ACLs, refresh your session, and verify role inheritance. Even if permissions look the same, there might be a small difference causing the issue. Try comparing your access settings with your colleague’s to spot any discrepancies...
I have created an Azure data factory pipeline with a copy data function to copy data from adls path to a delta table .In the delta table drop downs i am able to see only the hive metastore database and tables only but the unity catalog tables are not...
Any further updates on this? I also have the same problem. I tried above solutions, but no luck. By default, it shows only default and it's tables and I can preview those tables. If I enter the <catalog>.<schema> manually, it populates all the tables...
I tried to create a view in hive_metastore.default which would access a table from a different catalog. Is there any chance to do so?eg.create view myTest as select * from someCatalog.someSchema.someTable
Hi, so I don't know if you still have this problem (seeing as this post is almost 3 years old now haha) but I came across this when I was trying to solve an issue. What worked for me was creating the unity catalog table in the hive_metastore using an...
In the documentation is stated that the identity information is reserved for future use and currently always returns 'NO'.when is this going to be available ? and is there an alternative way to check if a specific column is already identity-defined o...
found an alternative way to check it for the moment : # check if identity column is active or not work around identityValue = 0 try: identityValue = str(spark.sql(f"show create table {table_catalog}.{table_schema}.{tab...
When I look at the permissions for a specific catalog in Catalog Explorer, I see the Application ID from Entra ID instead of the Service Principal name defined in the Databricks account for some Service Principals, while for others, I see the Service...
Thank's for your response. I think I resolved it:The Service Principal was added via the Databricks Console with a display name and application ID, but it was not added to the Databricks Workspace that I used to display permissions for that catalog, ...
Hello!Our team wish to implement just-in-time access to the audit log, where we thought we could use row filter for the audit log, such that the different teams can see their own events, and afterwards revoke their right to see it.Upon first inspecti...
I want to try Lakehouse monitoring in Unity Catalog.But it was failed in three table patterns.All of the drift metrics and profile metrics are No Data.I used the sample data "nyc taxi trips"And I configured the details like the following image.
Hi @Yuki The reason you cannot see anything in the result tables is that Lakehouse monitoring only analyzes data from 30 days before its creation. Please try using a source table that has more recent timestamps. When you first create a time series...
Respected Concern,We are currently using unity catalog in our workspace but we still see legacy hive_metastore under catalog section. Is there any way we can remove it. Because the issue we are facing is our cluster still somehow tries to connect wit...
Hi @DSoni, By default hive_metastore uses an RDS in AWS, in the failure you can see: xyz.region.rds.amazonaws.com which I think you redacted, that is the host with the issue. I would require more details to check hive_metastore comes into picture if ...
From my understanding, a user needs to own the object in order to alter the comments of columns/tables. A user would also need ALTER TABLE permissions in order to make the change. Is there a way to permit a user to change comments without giving them...
Hi @mrosman,As far as I understand, you can only provision SELECT / MODIFY / APPLY TAG to a table. And there is no specific privilege as of now in UC where you can assign a permission to alter the table / column comments. If the user have SELECT priv...
Are manual processes slowing down your business? Struggling with data silos and security concerns while scaling AI? Without the right automation and governance, AI adoption can be challenging.Join us for an exclusive live webinar on March 28th, 2025,...
In Unity Catalog, we can specify the managed location for a schema while creation.However, say if we migrate the tables and schemas from Hive metastore to UC without specifying managed location while creating the schema in UC.Is there a way to update...
Hi @bhanu_dp In Databricks Unity Catalog, once a schema is created without specifying a managed location, you cannot directly alter the schema to set or change its managed location. The managed location is typically set at the time of schema creation...
Hi everyone, We're considering bringing a section called "For You" in the catalog tree view into the Catalog Explorer tab. Right now this section surfaces active tables and favorited UC objects in authoring surfaces (SQL Editor, Notebooks, and Dashbo...
That makes a lot of sense! We're considering revamping it to not have the For You pills and to put the favorites up at the top
I am using Azure Databricks DBFS. I have stored some python scripts, notebooks in folders in DBFS root and have mounted ADLS Gen2 to DBFS. Currently I do not have Unity Catalog enabled and this was setup 3-4 years ago. I came across this article http...
Hello @humayun! DBFS mounts and root storage are deprecated, as they are not secure with the Unity Catalog and pose security risks. While there’s no official end-of-life date, Databricks discourages using mounts and root storage. Existing setups will...
