In databricks course V3 Unity catalog session I could not understand the answer of this question, can someone please enlighten me ? A senior data engineer with workspace identity privileges has created the schema customers within a Unity Catalog name...
The anwser for this question is mentionned here https://community.databricks.com/s/question/0D58Y0000ACa06GSQR/is-there-any-hierarchy-within-schema-permissions
In Unity catalog part we have this question: A data engineer needs to copy an external table from their default Hive metastore to the Unity Catalog metastore. Which of the following are required to upgrade the table to be managed by Unity Catalog? Se...
I think it was a mistake in the mentionned correction in Databricks according to the documentation Upgrade a single external table to Unity CatalogYou can copy an external table from your default Hive metastore to the Unity Catalog metastore using th...
Have been informed by a vendor we are using for Data Observability that the Unity Catalog lineage can be accessed via the systems schema. This is not enabled by default and we will need to execute a PUT request to the endpoint below. I am unable to f...
Hi @Stephen Quarshie​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...
Heres my current setup, dev workspace connected to dev keyvault and a prod workspace connected to a prod keyvault. There's a github repo and action syncing the two environments on pull request and all resources created through terraform. This is my n...
Hi @Jordan Fox​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers yo...
Hi team, Is anyone encounter this error?AnalysisException: [RequestId=d065497e-b88c-4687-9687-d7180663e7ba ErrorClass=METASTORE_DOES_NOT_EXIST] No metastore assigned for the current workspaceWe are using Databricks on AWSThanks!
I did test with different access modes and found that it indeed works with "data_security_mode": "LEGACY_SINGLE_USER_STANDARD". I also found clusters with no entry for "data_security_mode" which worked as expected.
I have started on a text based AI assistant for databricks. The idea is to interface the chatbot with the API of databricks to build a complete chat based interaction. For example intents could be:- a) Open a new notebookb) Edit an existing notebookc...
Hi,I got error [UC_NOT_ENABLED] Unity Catalog is not enabled on this cluster, when run in the notebook to create a catalog. Looks like I followed all the steps to set up workspace, metastore and compute. The steps are:create workspace from UI -> Qu...
Unity Catalog: create the first metastoreThe great benefit of the Unity catalog is that data is ours and stored in an open format on cloud storage in our container. To install the unity catalog, we need to create storage and give databricks access to...
This is very good. Can you explain on how it is providing Data governance across the Organization? With this we create a catalog and Organization can manage can create branches under this catalog, but still there should be governance rules need to be...
Hi,I would like to clone the structure (schema only) of delta table without the content using SHALLOW CLONE command, as recommended in databricks documentation, but it doesn't work. I get an error message that this command is not supported in Unity C...
You can shallow clone the table to an external location after sync the result of the shallow clone to unity catalog. Here is an example https://medium.com/@wahidatoui/sync-delta-tables-stored-in-dbfs-managed-or-external-to-unity-catalog-1fe29a3ae61b
Hi,We have been piloting databricks using unity catalog in development in our azure development subscription. We are looking at plans for moving to production\live which is a completely different azure subscription. I created a workspace in product...
@Andy Skinner​ When you create your workspace that will be tied to your databricks account.your account will be tied to subscription and resource group .from E2 architecture you can create multiple workspaces (in different regions) in one databricks ...
My org is considering a transition from hive metastore to unity catalog. We currently have a workspace for each of dev/uat/production and each of those provide access to their respective blob storage account data. Unity Catalog sits at the account-le...
I did find this document which indicates that you can set the initial catalog on cluster start:: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/hive-metastore#differences-from-table-access-control
I would like to publish data from from delta live table (DLT) to an Azure ADLS Gen2 storage. Checked the documentation but cannot get it to work. Any insights on how to do so?I've tried to specify the "Storage location" with many combinations of abfs...
I am trying to load from ADLS with DLT, after add the spark configurations I got below errors:org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] Query [id = 818323fc-80d5-4833-9f46-7d1afc9c5bf7, runId = 722e9aac-0fdd-4206-9d49-68...
hey i want to know how we can create my delta tables in unity catalog by the use of delta live tables pipelines or any other method by which we can create delta live tables to the unity catalog???
Hi @AlberTad​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we c...
Hi @Janga Reddy​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
