Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
We have many suscription with databricks workspaces active, but i cant create a metastore. The button is inactive.
Hi @DiegoRU ,Thank you for providing the details. You also need to have the role of global administrator on the azure subscription To create metastore, please login to this link https://accounts.azuredatabricks.net/login as admin and you should be ab...
Hi,we are trying to move some of our code from a ‘legacy’ cluster to a ‘Multi-node/ Shared' cluster so that we can start using Unity Catalog. However, we have run into an issue with some of our code, which calls Stored Procedures, on the new cluster....
We face the same issue. Our code depends heavily on stored procedures (best practice, no).Unfortunately, when moving to shared clusters and DBR 13.3, this does not work anymore.driver_manager = spark._sc._gateway.jvm.java.sql.DriverManager connection...
Hi,I deployed a Unity Catalog metastore some time back (screenshot below).But when I am trying to create a new one, the create metastore option is disabled. Hovering on the button shows the message: "You must have an active subscription to create a M...
Are you using Azure Databricks ?? have you recently migrated your classic Global Administration role to RBAC global admin in azure we had the same issue we fix it by creating metastore from CLI databricks metastores create {Name} --region {regi...
As written here: Compute access mode limitations for Unity Catalog | Databricks on AWSWhy is R language not supported?
Is there any update on when the timeline will come out? I am unable to use a shared cluster at work because shared computes do not have R as a supported language. Thanks
Hey Guyz, We need to scan the complete schema present at unity catalog to an external user or group of users without onboarding them on to our platform. Is there a way we can expose this details to them. Additionally, can we expose the sample dataset...
Hi @RohitKumar7 , Greetings! Looking at your request, i would like to confirm you that it would be possible to use the Delta sharing feature. Delta sharing feature lets you share data and AI assets with users outside your organization, whether or no...
When I run CREATE OR REPLACE VIEW on an existing view in Unity Catalog, the grants that were made on that object are removed. This seems like it is a bug. Is it on purpose or not?How to replicate:1. Create the viewRun the create or replace statement:...
I've just encountered this exact same problem where the behaviour of CREATE OR REPLACE for Tables keeps the permissions but loses them for Views. Very frustrating as it creates a race condition within the CI/CD to resolve!@marcell_nagy or @vmpmreist...
Hello,I would like to know if there is direct access to the Databricks query history tables. For compliance issues, I would like to be able to create reports for something like: who has accessed a particular column in a table in the past 6 months. T...
Throwing it out there that my team is also looking for a way to easily do this and I don't think the above solutions really fit our need. We are currently in the process of migrating schemas (from one name to a better name, we didn't rename because w...
Hi, as described in the title, when I try to explore the content of our metastore default location (external location created by default when the metastore is created) I get NO_PARENT_EXTERNAL_LOCATION_FOR_PATH. I've tried using a Serverless SQL ware...
The error NO_PARENT_EXTERNAL_LOCATION_FOR_PATH you are encountering is related to missing permissions for accessing the external location in Unity Catalog. This behavior is expected if the necessary permissions are not correctly set. Here are the ste...
Hi,I am trying to set up Unity-Catalog for my company and ran into a problem today. Basically, for each new source of data we ingest, we create a view-layer on top of the "tables". We do that because we have pseudonymized information in our datalake ...
Databricks actually has pretty good secrets management. If you need to secure a key, do this: https://learn.microsoft.com/en-us/azure/databricks/security/secrets/To use the key in your SQL statements, do this: https://learn.microsoft.com/en-us/azure...
Hi Community,I’ve been working on a data pipeline using Databricks Delta Live Tables (DLT) with Unity Catalog. My setup saves data into Azure Data Lake Storage (ADLS) for streaming tables. However, I’ve noticed that Unity Catalog encrypts various com...
Hi Community,I previously reached out regarding creating shortcuts in Microsoft Fabric for Databricks Delta Live Tables (DLT) managed through Unity Catalog, specifically when the data resides in Azure Data Lake Storage (ADLS) and appears encrypted.Af...
If I grant all privileges in my schema does that automatically give access to users for all underlying objects? Or should I give access seperately for all the objects?
Hi @Harikrish, In Unity Catalog, privileges are hierarchical and inherited downward. This means that granting a privilege on a catalog or schema automatically grants the privilege to all current and future objects within the catalog or schema. Theref...
Setup UCX from Databricks web terminal For cases when your desktop or laptop will not support the databricks labs install ucx technical requirements such as Python 3.10, administrative access, python package access or network access but your Databric...
I’ve found that using DBR 15.4, Personal Compute, and a Single Node is the most effective setup for installing UCX via web terminal. Note: The Databricks CLI is the recommended and most efficient option for managing the UCX installation. Always try t...
Hi,I want to import Databricks metadata information from unity catalog including lineage into GCP Dataplex governance solution. Can you let me know the how this can be achieved.Similar to Purview and Unity Catalog integration, are there APIs to push ...
To import Databricks metadata information from Unity Catalog, including lineage, into GCP Dataplex, you can use the Unity Catalog APIs to extract the necessary metadata and lineage information and then push this data into Dataplex using its APIs. Her...
When I try to interact with the "Browse DBFS" button, I get the following error: Operation failed: "The specified path, or an element of the path, exists and its resource type is invalid for this operation.", 409, PUT, https://dbstoragerrzvpacujrs22....
Hi @learning384e, Could you please share the command you are running including the action? does it have dbfs:/ in the path?
When creating a data share in my Databricks workspace, in the step Add data assets, one of my schemas in Unity Catalog shows 'No data' when expanding. If I expand the same schema in Unity Catalog explorer, it shows lots of tables and all tables conta...
Hello @Anske , This feature is currently in private preview. You should contact your account team to have it enabled in your workspace(s). Requirements Databricks Runtime (DBR) 13.3 or aboveRecipients must use Serverless SQL warehouse to query a shar...
