Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
I have enabled Unity Catalog in my AWS Databricks workspace, switching from the standard Hive Metastore. I have a few custom Hive UDFs in a JAR file that I have copied to /databricks/jars/ folder through init script. I want to be able to register tho...
Hi @Franklin George​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us ...
I'm using Azure Databricks.I've been following this guide to create a Unity Catalog metastore using a managed identity.When I create the catalog and the validation step kicks off, I get this screen: What does this mean?If I click "Force create" and t...
Hi @Alex Loddengaard​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...
Hi people, i`m using databricks Runtime 11.2 - Clusters Standard_F4 with Unity Catalog enabled and the service principals working,I literally perform an select in two delta tables that is in the storage X, the display() works, if i try to write an ne...
Hi @Guilherme Cesar​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us ...
The Unity Catalog API 2.1 only supports retrieving the unity catalogs associated with individual workspaces> The listMetastores endpoint does not list all Metstores that exist in the customer account. Instead it restricts the list by what the Workspa...
Hi @Iwan Aucamp​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...
Hi Team,Does Instance Pool supported in Unity Catalog enabled workspace?Cheers,G
Hi @Gil Gonong​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback w...
Feel free to watch a recording I made showing how to use the sync to upgrade tables from Hive metastore to Unity Catalog https://www.youtube.com/watch?v=HTiwJH9nvr8
Hi all,I am trying to write data from external s3 bucket to delta live tables at unity catalog( not dbfs ) from a workflow. I am seeing the following error ( saying catalog namespace is not supported. Please check full error below ) Error Detailspy4...
We have multiple processing chains that uses R notebooks with Sparklyr, and we try to migrate them from runtime 10.4 to 12.2. unfortunately, there seems to be an incompatibility with Sparlyr with runtimes > 10.4:Steps to reproduce:1) Create a noteboo...
Hi @Robin LOCHE​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
Hi,I am trying to setup presto in aws emr to be able to read delta tables. For this, as per my understanding, I need to setup the delta.properties file. Does anybody know how do I find the hive.metastore.uri that is needed for this file? For either u...
To find the hive.metastore.uri needed for the delta.properties file, you can follow the below steps:Login to your AWS Glue consoleClick on the "Crawlers" tab and select the Glue crawler that you are using for your Delta tablesIn the "Crawler details"...
We are following these instructions to link/assign a unity catalog metastore to a workspace using the UI. When we apply the option to enable the unity metastore in the workspace configuration, after hitting "Update", the option gets disabled. When we...
@Guilherme Jordan​ :If the option to enable the Unity Catalog metastore in the workspace configuration is getting disabled after hitting "Update", it could be a sign of an issue with the workspace or the Unity Catalog metastore.Here are some steps yo...
I've been trying to set up a Unity Catalog and play around with it. For that purpose, I created a new workspace with premium tier. As a tutorial, I follow Dustin Vannoy's video (https://www.youtube.com/watch?v=-RwzDRVgjLc) where he goes to his userna...
Hi @Martin Kostadinov​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your fee...
Hello. Despite having Databricks Premium, I cannot find the options to work with Unity Catalog. I login as administrator, go to the Data section, but cannot find the option to create a Unity Metastore. What am I missing?
Hi @Vlad Gheorghe​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so...
Apologies in advance for the soft question, but I'm genuinely struggling with this.We're a small data science unit just setting up in Databricks. While we do run some intensive ETL and analytics jobs, a non-trivial part of the team's BAU is explorato...
Hi @Jason Millburn​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
If I will migrate the Hive_metastore table into Unity Catalog table and subsequently hive metastore table is target of curation process. how I will migrate the delta record from Hive_metastore table to Unity Catalog table
Hi @Vivek Singh Banafar​ Hope everything is going great.Just checking in. Does @Debayan Mukherjee​'s response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to he...
After implementing Unity catalog, assuming a user has access to more than one DB. Can that user access all the DB in a single workspace? Can we restrict any specific DB in a workspace?
Hi @Prashant Lalaiya​,Hope you are well. Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too.Cheers!
