Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
Hello, now that a metstore is created by default; I'd like to know where the corresponding storage account is. Is it hosted by Databricks? I'm guessing so; if that the case; from a compliance and security perspective; is it better to move it to a sto...
Hi,I got error [UC_NOT_ENABLED] Unity Catalog is not enabled on this cluster, when run in the notebook to create a catalog. Looks like I followed all the steps to set up workspace, metastore and compute. The steps are:create workspace from UI -> Qu...
Hi @Meghana_Vasavad - I followed these steps below to solve the similar issue I faced today.Evidently, your workspace is not UC enabled. Please find the MS Learn link below to follow the step-by-step instructions to enable UC via account console.http...
We are trying to add Slack as a "Notification destination" under "Workspace Settings" in our Databricks workspace.However, there is no option to actually grant permission on that destination to define who gets to actually use that destination. This i...
Hi @emmalily ,thanks for your answer. I am curious to hear more about your first proposed solution "creating private channels". I am not sure how that would prevent others from using the Slack notification destination since everyone in the Databricks...
In today’s data-driven landscape, organizations increasingly recognize the critical need for unified governance to ensure consistent data security, compliance, and efficiency. AWS Glue Data Catalog users seeking enhanced governance, operational contr...
I have created an Azure data factory pipeline with a copy data function to copy data from adls path to a delta table .In the delta table drop downs i am able to see only the hive metastore database and tables only but the unity catalog tables are not...
Hi all, Please try following this syntax in the database field <catalog_name>`.`<database_name> It is important that you use ` not ' This should force the connect to use a Unity Catalog Database.
Hello community,I got this error, I created a new recipient grant the share data and created a share data with the catalog, how can I proceed in the interested catalog, where do I need to assign USE_PROVIDER?
You need to ask a metastore admin to gran you the USE PROVIDER permission at the metastore level https://docs.databricks.com/en/data-governance/unity-catalog/manage-privileges/privileges.html#use-providerThis can be done through the UI, under Catalog...
I have created an external table in unity catalog and when describing its data getting the following errororg.apache.hadoop.fs.FileAlreadyExistsException: Operation failed: "The specified path, or an element of the path, exists and its resource type ...
Can you confirm you created a Storage Credential and External Location prior to the CREATE TABLE statement. Also can you post the CREATE TABLE syntax? Thanks....
Hi, I have a couple of questions 1) For Unity Catalog can we sync 2 Metastores which are in different regions?2) If latency is not an issue, can a single metastore handle 2 different regions (and their workspaces)
1. There is no way to sync 2 metastores specifically but you can use Delta Sharing to create a read only view of a source metastore on a target metastore. 2. Metastores are associated to the region, only workspaces in the same region can be assigned
In our Production tenant we have majority of our workspaces in the North Europe region. We have disaster recovery set up and these workspaces are in West Europe, so will require 2 MetastoresIf Unity Catalog was to get implemented, is there a way to s...
I dont see the reply form KanizCan you please share what was the reply you received to your question? Thanks in advanceSaurabh
What is the pricing of Unity Catalog? Does it come without additional costs in premium workspace? If I upgrade my standard workspace to a premium workspace, is Unity Catalog added by default?
HI Shivam2, Unity Catalog features are provided to you at no charge once you upgrade to Premium or Enterprise tier. If your Databricks account is created after November 8, 2023, then Unity Catalog is enabled by default. You can use following instru...
I have been using "rdd.flatMap(lambda x:x)" for a while to create lists from columns however after I have changed the cluster to a Shared acess mode (to use unity catalog) I get the following error: py4j.security.Py4JSecurityException: Method public ...
Hi All,I faced the same issue. When i updated the cluster to DBR 15.5 it got resolved, it is not working in DBR 13.3 (my previous cluster version).Well, it is also working in DBR 10.4. I stayed with 15.5 as being the latest. Just try out with differe...
Hi there,Please, I am trying to understand if Databricks is able to support master data management capabilities. Particularly, focusing on the following ones:- Integrate and link different data systems: Connect various systems and make sure the data ...
I failed to mention above, Databricks has several solution accelerators which support MDM/ER types of work. They are meant to be examples of how, not to be used directly out of the box.Customer Entity Resolution | DatabricksEntity Resolution for Pub...
Hi, I am working with a firm that uses Databricks in order to analyze their system tables for cost visualizations and forecasting. I am trying to gain access to just the system tables without being able to view their other sensitive notebooks, artifa...
Hi @szymon_dybczak Thank you very much for the clarification! Turns out, the people I am working with are using the regular plan and not the premium so that might not be an option. Would the only other way be to send the system tables to an S3 bucket...
Hi there,We are using Databricks and are in the early stages of adopting it. Recently, I noticed something in Databricks that caught my attention.I implemented column-level security, which works such that if you are a member of a particular group, yo...
As written here: Compute access mode limitations for Unity Catalog | Databricks on AWSWhy is R language not supported?
@johnmn Please find below the reason for why R is not supported? R is currently not supported in Shared clusters due to the specific limitations and configurations of these clusters. However, Databricks has recognized this gap and has plans to suppor...
