Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
As written here: Compute access mode limitations for Unity Catalog | Databricks on AWSWhy is R language not supported?
Is there any update on when the timeline will come out? I am unable to use a shared cluster at work because shared computes do not have R as a supported language. Thanks
Hey Guyz, We need to scan the complete schema present at unity catalog to an external user or group of users without onboarding them on to our platform. Is there a way we can expose this details to them. Additionally, can we expose the sample dataset...
Hi @RohitKumar7 , Greetings! Looking at your request, i would like to confirm you that it would be possible to use the Delta sharing feature. Delta sharing feature lets you share data and AI assets with users outside your organization, whether or no...
When I run CREATE OR REPLACE VIEW on an existing view in Unity Catalog, the grants that were made on that object are removed. This seems like it is a bug. Is it on purpose or not?How to replicate:1. Create the viewRun the create or replace statement:...
I've just encountered this exact same problem where the behaviour of CREATE OR REPLACE for Tables keeps the permissions but loses them for Views. Very frustrating as it creates a race condition within the CI/CD to resolve!@marcell_nagy or @vmpmreist...
Hello,I would like to know if there is direct access to the Databricks query history tables. For compliance issues, I would like to be able to create reports for something like: who has accessed a particular column in a table in the past 6 months. T...
Throwing it out there that my team is also looking for a way to easily do this and I don't think the above solutions really fit our need. We are currently in the process of migrating schemas (from one name to a better name, we didn't rename because w...
Hi, as described in the title, when I try to explore the content of our metastore default location (external location created by default when the metastore is created) I get NO_PARENT_EXTERNAL_LOCATION_FOR_PATH. I've tried using a Serverless SQL ware...
The error NO_PARENT_EXTERNAL_LOCATION_FOR_PATH you are encountering is related to missing permissions for accessing the external location in Unity Catalog. This behavior is expected if the necessary permissions are not correctly set. Here are the ste...
Hi,I am trying to set up Unity-Catalog for my company and ran into a problem today. Basically, for each new source of data we ingest, we create a view-layer on top of the "tables". We do that because we have pseudonymized information in our datalake ...
Databricks actually has pretty good secrets management. If you need to secure a key, do this: https://learn.microsoft.com/en-us/azure/databricks/security/secrets/To use the key in your SQL statements, do this: https://learn.microsoft.com/en-us/azure...
Hi Community,I’ve been working on a data pipeline using Databricks Delta Live Tables (DLT) with Unity Catalog. My setup saves data into Azure Data Lake Storage (ADLS) for streaming tables. However, I’ve noticed that Unity Catalog encrypts various com...
Hi Community,I previously reached out regarding creating shortcuts in Microsoft Fabric for Databricks Delta Live Tables (DLT) managed through Unity Catalog, specifically when the data resides in Azure Data Lake Storage (ADLS) and appears encrypted.Af...
If I grant all privileges in my schema does that automatically give access to users for all underlying objects? Or should I give access seperately for all the objects?
Hi @Harikrish, In Unity Catalog, privileges are hierarchical and inherited downward. This means that granting a privilege on a catalog or schema automatically grants the privilege to all current and future objects within the catalog or schema. Theref...
Setup UCX from Databricks web terminal For cases when your desktop or laptop will not support the databricks labs install ucx technical requirements such as Python 3.10, administrative access, python package access or network access but your Databric...
I’ve found that using DBR 15.4, Personal Compute, and a Single Node is the most effective setup for installing UCX via web terminal. Note: The Databricks CLI is the recommended and most efficient option for managing the UCX installation. Always try t...
Hi,I want to import Databricks metadata information from unity catalog including lineage into GCP Dataplex governance solution. Can you let me know the how this can be achieved.Similar to Purview and Unity Catalog integration, are there APIs to push ...
To import Databricks metadata information from Unity Catalog, including lineage, into GCP Dataplex, you can use the Unity Catalog APIs to extract the necessary metadata and lineage information and then push this data into Dataplex using its APIs. Her...
When I try to interact with the "Browse DBFS" button, I get the following error: Operation failed: "The specified path, or an element of the path, exists and its resource type is invalid for this operation.", 409, PUT, https://dbstoragerrzvpacujrs22....
Hi @learning384e, Could you please share the command you are running including the action? does it have dbfs:/ in the path?
When creating a data share in my Databricks workspace, in the step Add data assets, one of my schemas in Unity Catalog shows 'No data' when expanding. If I expand the same schema in Unity Catalog explorer, it shows lots of tables and all tables conta...
Hello @Anske , This feature is currently in private preview. You should contact your account team to have it enabled in your workspace(s). Requirements Databricks Runtime (DBR) 13.3 or aboveRecipients must use Serverless SQL warehouse to query a shar...
Is it OK to provide consent to databricks support person to execute notebook in our workspace...Actually we encountered on technical issue related Mount Exception. Databricks support person is asking for consent to access our workspace in order to ex...
Hello you can indeed provide consent to the run the notebook if needed, but if you have security concerns you can also deny this access and suggest the support person to sync up on a call to execute the process and collect any log that might be neede...
I noticed in the Catalog Explorer of an Unity Catalog integrated workspace that there is a default catalog named 'system' owned by 'System user'. Who is this system user? It is not listed in the admin dashboard of all workspace users.
Is a special user account created by Databricks for managing system-level operations and configurations. This user is not listed in the admin dashboard of all workspace users because it is not a regular user account but rather a system account used i...
Hi, i try to used Unity Catalog in my workspace but i don´t get it!I have followed all the steps in this video and this is my group resource in Azure: After, i change Access control (IAM):Now i have the rol as Administrator and Owner in workspace.But...
Hi Everyone,I’m facing the same issue with Azure Databricks and was hoping someone could help.When I create an Azure Databricks workspace using the Azure subscription provided by my company, the workspace automatically attaches to the company’s Activ...
