Best Practices for Securing Access to Data in DatabricksUnity Catalog is the unified governance solution for Data & AI assets in Databricks and greatly simplifies and centralized data access control. This guide includes best practices for both the st...
We are planning on migrating to unity catalog but are unable to determine how we can segregate dev, staging and production data from each otherOur plan was to separate catalogs by SLDC Environment scopes (as per description and diagram at https://doc...
@Alex Davies​ :Unity Catalog does not currently support separating data by workspace or Azure subscription. As you noted, data from all catalogs within a region can be accessed by any workspace within that region, and it is up to user permissions to ...
I have managed to create my catalog and schema on unity catalog. Added Storage Blob Contributor role to the unity catalog metastore in ADLS for the Access Connector for Azure Databricks.however, when trying to create a table i received this error:Err...
Hi @adam wong​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback wi...
I click on the Search box on the top bar of the databricks workspace and select "Open advanced search". A dialog pops up with the Tables tab selected and an error below that says: "An error occurred while fetching search results. Please try again lat...
Hi @John Urban​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we...
I have enabled Unity Catalog in my AWS Databricks workspace, switching from the standard Hive Metastore. I have a few custom Hive UDFs in a JAR file that I have copied to /databricks/jars/ folder through init script. I want to be able to register tho...
Hi @Franklin George​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us ...
I'm using Azure Databricks.I've been following this guide to create a Unity Catalog metastore using a managed identity.When I create the catalog and the validation step kicks off, I get this screen: What does this mean?If I click "Force create" and t...
Hi @Alex Loddengaard​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...
Hi people, i`m using databricks Runtime 11.2 - Clusters Standard_F4 with Unity Catalog enabled and the service principals working,I literally perform an select in two delta tables that is in the storage X, the display() works, if i try to write an ne...
Hi @Guilherme Cesar​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us ...
The Unity Catalog API 2.1 only supports retrieving the unity catalogs associated with individual workspaces> The listMetastores endpoint does not list all Metstores that exist in the customer account. Instead it restricts the list by what the Workspa...
Hi @Iwan Aucamp​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...
Hi all,There is no official doc with a step-by-step process to enable Unity Catalog in azure databricks.If anyone has created doc or has the process, please share it here.Thanks, in advance.
Feel free to watch a recording I made showing how to use the sync to upgrade tables from Hive metastore to Unity Catalog https://www.youtube.com/watch?v=HTiwJH9nvr8
no information on character_maximum_length and is_identity in unity catalog table system.information_schema.columnstable definition exampleCREATE TABLE tyable_name ( UniqueID bigint not null GENERATED ALWAYS AS IDENTITY , ID varchar(256))looks like ...
@Werner Stinckens​ @Christy Seto​ @Jose Gonzalez​ can you help with that issue? It is a really interesting bug and a nice feature once it will work correctly.
Hi all,I am trying to write data from external s3 bucket to delta live tables at unity catalog( not dbfs ) from a workflow. I am seeing the following error ( saying catalog namespace is not supported. Please check full error below ) Error Detailspy4...
We have multiple processing chains that uses R notebooks with Sparklyr, and we try to migrate them from runtime 10.4 to 12.2. unfortunately, there seems to be an incompatibility with Sparlyr with runtimes > 10.4:Steps to reproduce:1) Create a noteboo...
Hi @Robin LOCHE​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
Hi,I am trying to setup presto in aws emr to be able to read delta tables. For this, as per my understanding, I need to setup the delta.properties file. Does anybody know how do I find the hive.metastore.uri that is needed for this file? For either u...
To find the hive.metastore.uri needed for the delta.properties file, you can follow the below steps:Login to your AWS Glue consoleClick on the "Crawlers" tab and select the Glue crawler that you are using for your Delta tablesIn the "Crawler details"...
We are following these instructions to link/assign a unity catalog metastore to a workspace using the UI. When we apply the option to enable the unity metastore in the workspace configuration, after hitting "Update", the option gets disabled. When we...
@Guilherme Jordan​ :If the option to enable the Unity Catalog metastore in the workspace configuration is getting disabled after hitting "Update", it could be a sign of an issue with the workspace or the Unity Catalog metastore.Here are some steps yo...
