Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
Hi Community,I’ve been working on a data pipeline using Databricks Delta Live Tables (DLT) with Unity Catalog. My setup saves data into Azure Data Lake Storage (ADLS) for streaming tables. However, I’ve noticed that Unity Catalog encrypts various com...
Hi Community,I previously reached out regarding creating shortcuts in Microsoft Fabric for Databricks Delta Live Tables (DLT) managed through Unity Catalog, specifically when the data resides in Azure Data Lake Storage (ADLS) and appears encrypted.Af...
If I grant all privileges in my schema does that automatically give access to users for all underlying objects? Or should I give access seperately for all the objects?
Hi @Harikrish, In Unity Catalog, privileges are hierarchical and inherited downward. This means that granting a privilege on a catalog or schema automatically grants the privilege to all current and future objects within the catalog or schema. Theref...
Setup UCX from Databricks web terminal For cases when your desktop or laptop will not support the databricks labs install ucx technical requirements such as Python 3.10, administrative access, python package access or network access but your Databric...
I’ve found that using DBR 15.4, Personal Compute, and a Single Node is the most effective setup for installing UCX via web terminal. Note: The Databricks CLI is the recommended and most efficient option for managing the UCX installation. Always try t...
Hi,I want to import Databricks metadata information from unity catalog including lineage into GCP Dataplex governance solution. Can you let me know the how this can be achieved.Similar to Purview and Unity Catalog integration, are there APIs to push ...
To import Databricks metadata information from Unity Catalog, including lineage, into GCP Dataplex, you can use the Unity Catalog APIs to extract the necessary metadata and lineage information and then push this data into Dataplex using its APIs. Her...
When I try to interact with the "Browse DBFS" button, I get the following error: Operation failed: "The specified path, or an element of the path, exists and its resource type is invalid for this operation.", 409, PUT, https://dbstoragerrzvpacujrs22....
Hi @learning384e, Could you please share the command you are running including the action? does it have dbfs:/ in the path?
When creating a data share in my Databricks workspace, in the step Add data assets, one of my schemas in Unity Catalog shows 'No data' when expanding. If I expand the same schema in Unity Catalog explorer, it shows lots of tables and all tables conta...
Hello @Anske , This feature is currently in private preview. You should contact your account team to have it enabled in your workspace(s). Requirements Databricks Runtime (DBR) 13.3 or aboveRecipients must use Serverless SQL warehouse to query a shar...
Is it OK to provide consent to databricks support person to execute notebook in our workspace...Actually we encountered on technical issue related Mount Exception. Databricks support person is asking for consent to access our workspace in order to ex...
Hello you can indeed provide consent to the run the notebook if needed, but if you have security concerns you can also deny this access and suggest the support person to sync up on a call to execute the process and collect any log that might be neede...
I noticed in the Catalog Explorer of an Unity Catalog integrated workspace that there is a default catalog named 'system' owned by 'System user'. Who is this system user? It is not listed in the admin dashboard of all workspace users.
Is a special user account created by Databricks for managing system-level operations and configurations. This user is not listed in the admin dashboard of all workspace users because it is not a regular user account but rather a system account used i...
Hi, i try to used Unity Catalog in my workspace but i don´t get it!I have followed all the steps in this video and this is my group resource in Azure: After, i change Access control (IAM):Now i have the rol as Administrator and Owner in workspace.But...
Hi Everyone,I’m facing the same issue with Azure Databricks and was hoping someone could help.When I create an Azure Databricks workspace using the Azure subscription provided by my company, the workspace automatically attaches to the company’s Activ...
Hello,Very excited to see the UC federation for hive and aws glue are now in public review. I have a workspace with hive metastore and it integrates with AWS glue (https://docs.databricks.com/en/archive/external-metastores/aws-glue-metastore.html). T...
Hello @hzh, Thanks for your question! UC Federation Connection Type and Metastore Type When creating a connection for UC federation with a Hive metastore integrated with AWS Glue, the connection type should be Hive metastore. Metastore Type If the Hi...
Created a UC volume on the dev lane and, surprisingly, was able to see the same on all the other ADB subscriptions.did i miss anything ?
Then, that is the reason why, the visibility of the UC volume across all your ADB subscriptions is expected behavior if they share the same metastore within the same region and tenant
Hi,In our Azure tenant We have deleted a Unity Catalog registered workspace and ticked the Delete default workspace storage permanentlyThere are 2 catalogs that were created in that workspace that are now greyed out in the catalog explorer when acces...
databricks catalogs delete --name my-catalog --force Thanks for the help ! Needed the force as the catalog was not empty. Also required a change of ownership of catalog to metastore owner.
we are using mount points via service principals approach to connect the storage account. and using the same mount points to create external tables in hive meta store. but now we are trying using only service principals setup ,so we need to change t...
Hi @mysura, Is your workspace Unity Catalog enabled? do you have a metastore assigned? I think legacy hive_metastore will not support service principal to do this change. I think the best approach is to create an external location based on the adfss ...
Is there a feature update request or resolution for adding service endpoints to the worker-vn that databricks created? We are experiencing 403 errors for IP authorizations from the worker-vn and do not have the permissions to update the service endpo...
In my workspace, I'm able to see the unity catalogs my team has created. I'm able to see the schemas and even query data in there and create views using the SQL Editor.When I go to write the same sql queries from a notebook using spark.sql or %sql I ...
i know this is an old topic, but i have the same issue with querying data using the sql editor and a sql warehouse. my sql warehouse does have unity catalog enabled. not sure why it wouldnt be working.
