Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
Hi there,When creating an external table in Unity Catalog using an existing Delta Lake table with a comment on the table itself, the comment isn't imported to the `Comment` key's value in the Unity Catalog table.Could you explain why?
Hi team, Have you tried using SQL command to update the column comment?ALTER TABLE <table_name> ALTER COLUMN <column_name> COMMENT '<comment_text>';I believe this should resolve the issue
Unity Catalog (UC) tracks the metadata and your cloud storage accounts store the your data. This python script will extract the metadata from {catalog}.information_schema into folders in a storage location. Take this and put into a notebook. Make da...
Hi All, Can anyone point me to either documentation or personally tried and tested method of backing up (and restoring) Unity Catalog and its associated managed tables? We're running on Azure and using ADLS Gen2.Regards,Ashley
Unity Catalog (UC) tracks the metadata and your cloud storage accounts store the your data. This python script will extract the metadata from {catalog}.information_schema into folders in a storage location. Take this and put into a notebook.Make data...
Hi,I am working for a large company that is implementing a Databricks solution. We have multiple domains, each responsible for its own data products, following a data mesh approach.As part of a federated governance model, we need a way to communicate...
If a model is registered in Unity Catalog, then what all APIs and SDKs could be used to build the entire lineage for a model? I'm trying to figure out what all do I query to ensure I don't miss any element of the model lineage.Now a model can have be...
@rashij You're tackling an important challenge in ML governance. Building a comprehensive lineage for models registered in Unity Catalog requires piecing together information from multiple APIs. Let me outline a more complete approach based on your i...
I am trying to set up Unity Catalog in Azure Databricks following this documentation Unity Catalog, but I do not find "Create Metastore" and "Create Metastore" on Databricks.Please help with the same, how can I enable Unity Catalog.
This issue is resolved. Thank you @Camilla Lok​ and @Harun for the suggestions. It works fine after enabling Global administrators in Azure Active Directory .
Dear community, I want to configure the unity catalog with databricks, but the Create MetaStore button does not appear in my Azure Databricks workspace.In the community, I have seen that several users present the same situation ... What is the reason...
If I am the owner of the Azure Subcription and databricks premium is in my resource group that makes me owner on that RG as well as Databricks i am still not able to see Metastore setting for unity catalog .. have someone found a resolution to this. ...
I'm working out a terraform script that creates a job that needs to write files in the volume.When the job runs as me (I have full privileges), it encounters the following error if running with job compute: Permission error: operation not permitted...
I think I found the solution, you need to specify the data_security_mode parameter as SINGLE_USER to grant access to the Unity Catalog
Hi there,Please, I am trying to understand if Databricks is able to support master data management capabilities. Particularly, focusing on the following ones:- Integrate and link different data systems: Connect various systems and make sure the data ...
We at Frisco Analytics are a built on partner for Databricks and built and MDM application that is native to Databricks. https://www.lakefusion.ai/ Please contact us at contact@friscoanalytics.com .Thank you!Haritha
Hi all,I'm deploying Unity Catalog into a large enterprise with quite strict controls. I want to give my users some autonomy over creating tables from upstream datasets they have select access on, but also restrict these controls. I've been through a...
@ajbush did you workout your requirements in a way you wanted? appreciate if you can share insights here.
I am trying to connect to Unity catalog meta store tables using Presto Based on the presto documentation I need to use the below configuration to connect to delta tables in the unity catalog https://prestodb.io/docs/current/connector/hive.htmlSo from...
Hello @December , You should contact your account team if you need more info on this feature which is not GA for the time being. Another option would be to use Iceberg REST interface: https://medium.com/@flyws1993/oss-trino-to-read-tables-from-databr...
How can I get the unity catalog code base?
https://github.com/unitycatalog/unitycatalogThis is the OSS version of course. The databricks version is not available afaik.
I get an error message saying I Error getting sample data, when I try to view sample data from a table in a schema I created in a Unity Catalog. I dropped the schema and table and got a collague to recreate and still the same message. We are both Uni...
Check your table level ACLs, refresh your session, and verify role inheritance. Even if permissions look the same, there might be a small difference causing the issue. Try comparing your access settings with your colleague’s to spot any discrepancies...
I have created an Azure data factory pipeline with a copy data function to copy data from adls path to a delta table .In the delta table drop downs i am able to see only the hive metastore database and tables only but the unity catalog tables are not...
Any further updates on this? I also have the same problem. I tried above solutions, but no luck. By default, it shows only default and it's tables and I can preview those tables. If I enter the <catalog>.<schema> manually, it populates all the tables...
I tried to create a view in hive_metastore.default which would access a table from a different catalog. Is there any chance to do so?eg.create view myTest as select * from someCatalog.someSchema.someTable
Hi, so I don't know if you still have this problem (seeing as this post is almost 3 years old now haha) but I came across this when I was trying to solve an issue. What worked for me was creating the unity catalog table in the hive_metastore using an...
