cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

RamaTeja
by New Contributor II
  • 8277 Views
  • 6 replies
  • 0 kudos

In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop down.

I have created an Azure data factory pipeline with a copy data function to copy data from adls path to a delta table .In the delta table drop downs i am able to see only the hive metastore database and tables only but the unity catalog tables are not...

  • 8277 Views
  • 6 replies
  • 0 kudos
Latest Reply
latteuro
New Contributor II
  • 0 kudos

Hi, I have the same issue.Additional information: the linked service created in azure data factory using azure databricks deltalake connector is using system-managed-identity rather than Token. Could we have an update? Thank you in advance. 

  • 0 kudos
5 More Replies
Alex006
by Contributor
  • 6195 Views
  • 11 replies
  • 11 kudos

Resolved! Metastore creation - Azure Databricks - Access validation failed

Hi, Getting "Access validation failed" when trying to create a metastore. Anyone noticed anything extra needed to be done except for what's written in the azure guides below? Any help on how to succeed with creating a metastore would be deeply apprec...

Access validation failed when trying to create a metastore Role assignment to azure databricks connector in the storage account
  • 6195 Views
  • 11 replies
  • 11 kudos
Latest Reply
abarber
New Contributor II
  • 11 kudos

I was told this was a bug that was fixed on Monday. We’re using AWS, and I was able to create the metastore on Tuesday without error.

  • 11 kudos
10 More Replies
DataEngineer92
by New Contributor II
  • 2738 Views
  • 4 replies
  • 2 kudos

Resolved! Databricks-connect 11.3 works with Unity Catalog ?

Databricks-connect 11.3 works for unity catalog enabled cluster or not ? I am trying to utilize the databricks-connect in Azure DevOps pipeline using Microsoft hosted agent. But getting the below error when running the databricks-connect test command...

  • 2738 Views
  • 4 replies
  • 2 kudos
Latest Reply
bastih
New Contributor III
  • 2 kudos

Hi @Rey Jhon​ Sadly, databricks-connect for DBR versions < 13 does not support UC, as outlined in the limitations section: https://docs.databricks.com/dev-tools/databricks-connect-legacy.html#limitations.To use UC, we recommend using DBR version >= 1...

  • 2 kudos
3 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 3030 Views
  • 4 replies
  • 19 kudos

Unity Catalog: create the first metastore The great benefit of the Unity catalog is that data is ours and stored in an open format on cloud storage in...

Unity Catalog: create the first metastoreThe great benefit of the Unity catalog is that data is ours and stored in an open format on cloud storage in our container. To install the unity catalog, we need to create storage and give databricks access to...

image.png image.png image.png image.png
  • 3030 Views
  • 4 replies
  • 19 kudos
Latest Reply
RDD1
New Contributor III
  • 19 kudos

This is very good. Can you explain on how it is providing Data governance across the Organization? With this we create a catalog and Organization can manage can create branches under this catalog, but still there should be governance rules need to be...

  • 19 kudos
3 More Replies
DrK
by New Contributor III
  • 2341 Views
  • 2 replies
  • 1 kudos

Databricks Account in Azure

Hi,We have been piloting databricks using unity catalog in development in our azure development subscription. We are looking at plans for moving to production\live which is a completely different azure subscription. I created a workspace in product...

  • 2341 Views
  • 2 replies
  • 1 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 1 kudos

@Andy Skinner​ When you create your workspace that will be tied to your databricks account.your account will be tied to subscription and resource group .from E2 architecture you can create multiple workspaces (in different regions) in one databricks ...

  • 1 kudos
1 More Replies
Alex006
by Contributor
  • 5923 Views
  • 8 replies
  • 13 kudos

Publish data to Azure ADLS Gen2 from Delta Live Tables pipelines?

I would like to publish data from from delta live table (DLT) to an Azure ADLS Gen2 storage. Checked the documentation but cannot get it to work. Any insights on how to do so?I've tried to specify the "Storage location" with many combinations of abfs...

Invalid configuration value detected for fs.azure.account.key Storage location
  • 5923 Views
  • 8 replies
  • 13 kudos
Latest Reply
guostong
New Contributor III
  • 13 kudos

I am trying to load from ADLS with DLT, after add the spark configurations I got below errors:org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] Query [id = 818323fc-80d5-4833-9f46-7d1afc9c5bf7, runId = 722e9aac-0fdd-4206-9d49-68...

  • 13 kudos
7 More Replies
weinino
by New Contributor II
  • 3449 Views
  • 6 replies
  • 2 kudos

Migration of Unity Catalog files with Databricks migration tool

Hi there,I'm currently migrating Databricks (metastores, workspaces, etc.) from Azure to AWS using the Databricks migration tool. During the migration process and after digging in the code, I've come to the conclusion that the tool only supports a mi...

  • 3449 Views
  • 6 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Nino Weingart​ :Q1: You are correct that the Databricks migration tool only supports migration of the built-in Hive metastore and not custom metastores in Unity Catalog. If you want to migrate custom metastores, you would need to extend the code and...

  • 2 kudos
5 More Replies
agallard
by New Contributor III
  • 3819 Views
  • 6 replies
  • 6 kudos

I can´t used Unity Catalog

Hi, i try to used Unity Catalog in my workspace but i don´t get it!I have followed all the steps in this video and this is my group resource in Azure: After, i change Access control (IAM):Now i have the rol as Administrator and Owner in workspace.But...

1 2 3 4
  • 3819 Views
  • 6 replies
  • 6 kudos
Latest Reply
agallard
New Contributor III
  • 6 kudos

Hi @Kaniz Fatma​ i´m owner to the workspace of Databricks but i can´t used Unity Catalog,I tried install dbdemo about Unity Catalog:%pip install dbdemos   import dbdemos dbdemos.install('uc-03-data-lineage')it created a notebook and cluster, but the ...

  • 6 kudos
5 More Replies
JohnUrban
by New Contributor III
  • 1276 Views
  • 4 replies
  • 4 kudos

Advanced Search on Tables getting error

I click on the Search box on the top bar of the databricks workspace and select "Open advanced search". A dialog pops up with the Tables tab selected and an error below that says: "An error occurred while fetching search results. Please try again lat...

  • 1276 Views
  • 4 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @John Urban​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we...

  • 4 kudos
3 More Replies
alexlod
by New Contributor III
  • 4250 Views
  • 5 replies
  • 0 kudos

Resolved! Unclear validation error message when adding a Unity Catalog metastore in Azure

I'm using Azure Databricks.I've been following this guide to create a Unity Catalog metastore using a managed identity.When I create the catalog and the validation step kicks off, I get this screen: What does this mean?If I click "Force create" and t...

Screenshot 2023-02-07 at 1.06.40 PM
  • 4250 Views
  • 5 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Alex Loddengaard​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...

  • 0 kudos
4 More Replies
Guilherme
by New Contributor
  • 908 Views
  • 2 replies
  • 0 kudos

How this is possible? access error in storage, after selecting data from an delta IN THE STORAGE

Hi people, i`m using databricks Runtime 11.2 - Clusters Standard_F4 with Unity Catalog enabled and the service principals working,I literally perform an select in two delta tables that is in the storage X, the display() works, if i try to write an ne...

  • 908 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Guilherme Cesar​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us ...

  • 0 kudos
1 More Replies
Ajay-Pandey
by Esteemed Contributor III
  • 5007 Views
  • 9 replies
  • 4 kudos

Resolved! Step by step process to create Unity Catalog in Azure Databricks

Hi all,There is no official doc with a step-by-step process to enable Unity Catalog in azure databricks.If anyone has created doc or has the process, please share it here.Thanks, in advance.

  • 5007 Views
  • 9 replies
  • 4 kudos
Latest Reply
youssefmrini
Honored Contributor III
  • 4 kudos

I recorded a video showing how to setup Unity Catalog Step by step https://www.youtube.com/watch?v=M2Et5aBj2aw

  • 4 kudos
8 More Replies
venkad
by Contributor
  • 4025 Views
  • 3 replies
  • 3 kudos

dbutils.fs.ls doesn't work on Unity catalog managed location

Hi,dbutils.fs.ls() is a most useful utility to list the files in remote cloud storage. I cannot use this function on top of the location managed by the Unity catalog.I have a managed external location created on `'abfss://xyz@abcd.dfs.core.windows.ne...

  • 4025 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Venkadeshwaran K​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answ...

  • 3 kudos
2 More Replies
Gustavo_Az
by Contributor
  • 2942 Views
  • 4 replies
  • 0 kudos

Resolved! Cant make Unity Catalog work on Azure

HelloI have created the metastore, linked it to the worskpace, my account is admin in Azure and Databricks, followed these instructions:link1link2link3But when I try to create a table (granting privileges to use catalog, create schemas and create tab...

error unity catalog error overlaps with managed storage
  • 2942 Views
  • 4 replies
  • 0 kudos
Latest Reply
Gustavo_Az
Contributor
  • 0 kudos

I finally made it work.It seems that although after configuring the metastore to work with a manged identity, the metastore must be upgraded with this guide.

  • 0 kudos
3 More Replies
Arpi
by New Contributor II
  • 887 Views
  • 2 replies
  • 0 kudos

Delta table error

Hi,we are creating an external table in the Azure databricks but facing this error:org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.meta...

  • 887 Views
  • 2 replies
  • 0 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 0 kudos

@Priyanka Biswas​ is there a place where we can have updates on metastore upgrades or control plane individual components. usually we can only see control plane region wide status

  • 0 kudos
1 More Replies
Labels