Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
We are excited to announce the Security Analysis Tool (SAT) for AWS! What is the Security Analysis Tool (SAT)?SAT helps customers monitor the security health of customer account workspaces over time by comparing workspace configurations against speci...
Announcing Terraform support for setting up the Security Analysis Tool (SAT). Now setting up SAT is super easy on all clouds: https://github.com/databricks-industry-solutions/security-analysis-tool/tree/main/terraform
Hi All,I am trying to manage access to objects created in a Unity Catalog.I want the owner/creator of the object(me) to not have access to the object and only the specified users/groups to be able to select/perform other actions on the objects.But th...
Hi @Akash Sivadas​ ,I think you can do it by data explorer tab.Please refer below blog for the same-Manage Unity Catalog permissions in Data Explorer | Databricks on AWS
HiGetting the error message as below: when following the tutorial:Get started using Unity Catalog - Azure Databricks | Microsoft Learnin the section of "Create your first metastore and attach a workspace" with both storage and connector information f...
This is a bug, thanks for reporting it! Databricks engineers would address this soon.
HelloI have created the metastore, linked it to the worskpace, my account is admin in Azure and Databricks, followed these instructions:link1link2link3But when I try to create a table (granting privileges to use catalog, create schemas and create tab...
I finally made it work.It seems that although after configuring the metastore to work with a manged identity, the metastore must be upgraded with this guide.
Hi,we are creating an external table in the Azure databricks but facing this error:org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.meta...
@Priyanka Biswas​ is there a place where we can have updates on metastore upgrades or control plane individual components. usually we can only see control plane region wide status
In my org we've got workspaces with a mixture of SCIM-provisioned and non-SCIM groups. These are all 'workspace local' groups. My identity provider is AAD.I've created a new workspace and want users in this workspace to be provided access only via ac...
Hi, Please refer to https://learn.microsoft.com/en-us/azure/databricks/administration-guide/users-groups/users and https://learn.microsoft.com/en-us/azure/databricks/administration-guide/users-groups/groups. Also, please note if you already have SCIM...
Hello,I started testing Unity catalog and was surprised by the following mention in the documentation about shared clusters not supporting third-party1 libraries. Nevertheless, I tried doing imports and using pip install and both were working correct...
Hi Aviral-Bhardwaj-ji, I have seen a couple of mentions about Whitelisting of libraries on the Account but have struggled to find any detail about how to actually do this. Like RickDB (OP) I have been exploring the idea of 'whitelisting' libraries on...
I have created a cluster which has advance options set for external hive metastore. I created a mount using sp for my data lake and then create a table using the mount . As per my knowledge the metadata of the table will be stored in sql db but i wou...
Hi @Vishal Gupta​ when we create a mount we store the mount info in a file in the DBFS root bucket.
I am trying to set up Unity Catalog in Azure Databricks following this documentation Unity Catalog, but I do not find "Create Metastore" and "Create Metastore" on Databricks.Please help with the same, how can I enable Unity Catalog.
This issue is resolved. Thank you @Camilla Lok​ and @Harun for the suggestions. It works fine after enabling Global administrators in Azure Active Directory .
I am trying to invoke a REST service from Databrcisks notebook using python programming. It is working fine before enabling Unity Catalog but after enabling it (In another environment) it is not working and never giving any response and not showing a...
I suspect the problem could be caused by the cluster mode. What type of cluster or what access mode do you use for your cluster?
we have enabled unity catalog in Azure, we have a requirement to create external table in hive_metastore. we have configured ADLS Gen2 and by using different access methods that ADLS Gen 2 supports we have created external table. we are able to view ...
@karthik p​ Unity Catalog is supported in DBR 11.2 and higher versions. If you attempt to create hive external tables on these DBR versions by default storage credential will be used for authentication to the storage account. You can create storage c...
Hello,im trying to use oracledb on my cluster and having issues using it.An init script on the cluster fixed this issue.Now, im trying to create a shared cluster with unity catalog and have oracledb in it.I encountered issues having this.- Init scrip...
@Chris Joshua Manuel​ The support for init script, library installation, Machine Learning, UDF on shared cluster is in preview. Please submit this form to onboard your workspace for improved shared clusters preview. https://docs.google.com/forms/d/e...
We're trying to use pandas_udf in DBR 10.4 LTS Shared mode but keep running in the below error :AnalysisException: UDF/UDAF/SQL functions is not supported in Unity Catalog;But in Single User mode above code works correctly.
For me, it is like a bug as, in my opinion, that code should be independent of the unity catalog as it sparks code, and you don't register anything in the unity catalog. So please pass it to your databricks representative or as a support ticket.There...
Dear community, I want to configure the unity catalog with databricks, but the Create MetaStore button does not appear in my Azure Databricks workspace.In the community, I have seen that several users present the same situation ... What is the reason...
Hello Databricks Community, good news! I already found the solution: To login as an administrator, it is necessary to have the role of global administrator on the azure subscription, otherwise it will never appear.
Hi,I am not able to access the schemas inside the hive_metastore in the schema browser inside the SQL Editor in Databricks SQL. I tried on 2 databricks instances. One has Unity Catalog enabled and I am able to access the other catalogs/schemas but no...
