- 14458 Views
- 14 replies
- 4 kudos
In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop down.
I have created an Azure data factory pipeline with a copy data function to copy data from adls path to a delta table .In the delta table drop downs i am able to see only the hive metastore database and tables only but the unity catalog tables are not...
- 14458 Views
- 14 replies
- 4 kudos
- 4 kudos
Hi all, Please try following this syntax in the database field <catalog_name>`.`<database_name> It is important that you use ` not ' This should force the connect to use a Unity Catalog Database.
- 4 kudos
- 9724 Views
- 10 replies
- 4 kudos
Resolved! Step by step process to create Unity Catalog in Azure Databricks
Hi all,There is no official doc with a step-by-step process to enable Unity Catalog in azure databricks.If anyone has created doc or has the process, please share it here.Thanks, in advance.
- 9724 Views
- 10 replies
- 4 kudos
- 4 kudos
Hi everyone!I have an Azure for Students subscription. Since I am unable to be an Azure Active Directory Global Administrator, I tried but it is restricted by the organization to add my user as a Global Administrator, I guess with an Azure for Studen...
- 4 kudos
- 15338 Views
- 9 replies
- 10 kudos
Resolved! Metastore creation - Azure Databricks - Access validation failed
Hi, Getting "Access validation failed" when trying to create a metastore. Anyone noticed anything extra needed to be done except for what's written in the azure guides below? Any help on how to succeed with creating a metastore would be deeply apprec...
- 15338 Views
- 9 replies
- 10 kudos
- 10 kudos
I was told this was a bug that was fixed on Monday. We’re using AWS, and I was able to create the metastore on Tuesday without error.
- 10 kudos
- 4981 Views
- 4 replies
- 2 kudos
Resolved! Databricks-connect 11.3 works with Unity Catalog ?
Databricks-connect 11.3 works for unity catalog enabled cluster or not ? I am trying to utilize the databricks-connect in Azure DevOps pipeline using Microsoft hosted agent. But getting the below error when running the databricks-connect test command...
- 4981 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @Rey Jhon​ Sadly, databricks-connect for DBR versions < 13 does not support UC, as outlined in the limitations section: https://docs.databricks.com/dev-tools/databricks-connect-legacy.html#limitations.To use UC, we recommend using DBR version >= 1...
- 2 kudos
- 6889 Views
- 4 replies
- 19 kudos
Unity Catalog: create the first metastore The great benefit of the Unity catalog is that data is ours and stored in an open format on cloud storage in...
Unity Catalog: create the first metastoreThe great benefit of the Unity catalog is that data is ours and stored in an open format on cloud storage in our container. To install the unity catalog, we need to create storage and give databricks access to...
- 6889 Views
- 4 replies
- 19 kudos
- 19 kudos
This is very good. Can you explain on how it is providing Data governance across the Organization? With this we create a catalog and Organization can manage can create branches under this catalog, but still there should be governance rules need to be...
- 19 kudos
- 11669 Views
- 2 replies
- 1 kudos
Databricks Account in Azure
Hi,We have been piloting databricks using unity catalog in development in our azure development subscription. We are looking at plans for moving to production\live which is a completely different azure subscription. I created a workspace in product...
- 11669 Views
- 2 replies
- 1 kudos
- 1 kudos
@Andy Skinner​ When you create your workspace that will be tied to your databricks account.your account will be tied to subscription and resource group .from E2 architecture you can create multiple workspaces (in different regions) in one databricks ...
- 1 kudos
- 10504 Views
- 8 replies
- 13 kudos
Publish data to Azure ADLS Gen2 from Delta Live Tables pipelines?
I would like to publish data from from delta live table (DLT) to an Azure ADLS Gen2 storage. Checked the documentation but cannot get it to work. Any insights on how to do so?I've tried to specify the "Storage location" with many combinations of abfs...
- 10504 Views
- 8 replies
- 13 kudos
- 13 kudos
I am trying to load from ADLS with DLT, after add the spark configurations I got below errors:org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] Query [id = 818323fc-80d5-4833-9f46-7d1afc9c5bf7, runId = 722e9aac-0fdd-4206-9d49-68...
- 13 kudos
- 6791 Views
- 6 replies
- 4 kudos
Migration of Unity Catalog files with Databricks migration tool
Hi there,I'm currently migrating Databricks (metastores, workspaces, etc.) from Azure to AWS using the Databricks migration tool. During the migration process and after digging in the code, I've come to the conclusion that the tool only supports a mi...
- 6791 Views
- 6 replies
- 4 kudos
- 4 kudos
@Nino Weingart​ :Q1: You are correct that the Databricks migration tool only supports migration of the built-in Hive metastore and not custom metastores in Unity Catalog. If you want to migrate custom metastores, you would need to extend the code and...
- 4 kudos
- 8915 Views
- 5 replies
- 6 kudos
I can´t used Unity Catalog
Hi, i try to used Unity Catalog in my workspace but i don´t get it!I have followed all the steps in this video and this is my group resource in Azure: After, i change Access control (IAM):Now i have the rol as Administrator and Owner in workspace.But...
- 8915 Views
- 5 replies
- 6 kudos
- 6 kudos
Hi @Kaniz Fatma​ i´m owner to the workspace of Databricks but i can´t used Unity Catalog,I tried install dbdemo about Unity Catalog:%pip install dbdemos import dbdemos dbdemos.install('uc-03-data-lineage')it created a notebook and cluster, but the ...
- 6 kudos
- 3942 Views
- 3 replies
- 4 kudos
Advanced Search on Tables getting error
I click on the Search box on the top bar of the databricks workspace and select "Open advanced search". A dialog pops up with the Tables tab selected and an error below that says: "An error occurred while fetching search results. Please try again lat...
- 3942 Views
- 3 replies
- 4 kudos
- 4 kudos
Hi @John Urban​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we...
- 4 kudos
- 7542 Views
- 5 replies
- 0 kudos
Resolved! Unclear validation error message when adding a Unity Catalog metastore in Azure
I'm using Azure Databricks.I've been following this guide to create a Unity Catalog metastore using a managed identity.When I create the catalog and the validation step kicks off, I get this screen: What does this mean?If I click "Force create" and t...
- 7542 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @Alex Loddengaard​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...
- 0 kudos
- 1899 Views
- 2 replies
- 0 kudos
How this is possible? access error in storage, after selecting data from an delta IN THE STORAGE
Hi people, i`m using databricks Runtime 11.2 - Clusters Standard_F4 with Unity Catalog enabled and the service principals working,I literally perform an select in two delta tables that is in the storage X, the display() works, if i try to write an ne...
- 1899 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Guilherme Cesar​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us ...
- 0 kudos
- 6356 Views
- 2 replies
- 3 kudos
dbutils.fs.ls doesn't work on Unity catalog managed location
Hi,dbutils.fs.ls() is a most useful utility to list the files in remote cloud storage. I cannot use this function on top of the location managed by the Unity catalog.I have a managed external location created on `'abfss://xyz@abcd.dfs.core.windows.ne...
- 6356 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Venkadeshwaran K​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answ...
- 3 kudos
- 7074 Views
- 4 replies
- 0 kudos
Resolved! Cant make Unity Catalog work on Azure
HelloI have created the metastore, linked it to the worskpace, my account is admin in Azure and Databricks, followed these instructions:link1link2link3But when I try to create a table (granting privileges to use catalog, create schemas and create tab...
- 7074 Views
- 4 replies
- 0 kudos
- 0 kudos
I finally made it work.It seems that although after configuring the metastore to work with a manged identity, the metastore must be upgraded with this guide.
- 0 kudos
- 2027 Views
- 2 replies
- 0 kudos
Delta table error
Hi,we are creating an external table in the Azure databricks but facing this error:org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.meta...
- 2027 Views
- 2 replies
- 0 kudos
- 0 kudos
@Priyanka Biswas​ is there a place where we can have updates on metastore upgrades or control plane individual components. usually we can only see control plane region wide status
- 0 kudos