topic Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow in Data Governance

In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop down.

RamaTeja — Wed, 26 Apr 2023 09:01:24 GMT

I have created an Azure data factory pipeline with a copy data function to copy data from adls path to a delta table .In the delta table drop downs i am able to see only the hive metastore database and tables only but the unity catalog tables are not listed .Can anyone please help me on this.

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop down.

Anonymous — Thu, 27 Apr 2023 05:07:12 GMT

@Rama Teja Reddy Damireddy :

To see the Unity Catalog databases and tables in the drop-down in Azure Databricks Delta Lake, you need to configure the integration between Unity Catalog and Databricks. Here are the steps to configure the integration:

Navigate to your Databricks workspace and click on the "Admin Console" button on the left-hand side of the screen.
Click on the "Data" tab and then click on "Connect" next to Unity Catalog.
Enter your Unity Catalog credentials and click "Connect".
You should now see your Unity Catalog databases and tables in the drop-downs in Delta Lake.

If you have already followed these steps and are still not seeing the Unity Catalog databases and tables, you may want to check if the user account you are using has the necessary permissions to access the Unity Catalog.

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop down.

Anonymous — Thu, 27 Apr 2023 07:49:35 GMT

Hi @Rama Teja Reddy Damireddy

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop down.

Anonymous — Fri, 28 Apr 2023 13:20:49 GMT

I also have the same question. I don't think you understand what is being asked. Within Azure Data Factory, when creating an Azure Databricks DeltaLake Dataset, there does not appear to be a way to select a database (schema) that is not in the hive_metastore catalog. See attached screenshot. Is UC not currently supported? I've sent the same question to Microsoft.

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

NK_123 — Mon, 10 Jul 2023 11:17:58 GMT

Hi, Were you able to solve the problem? I am also trying to do same

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

cmunteanu — Fri, 05 Jan 2024 11:14:45 GMT

Hi,

Exact same problem here, any updates please ?

Thanks

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

latteuro — Wed, 10 Jan 2024 14:29:17 GMT

Hi,

I have the same issue.

Additional information: the linked service created in azure data factory using azure databricks deltalake connector is using system-managed-identity rather than Token.

Could we have an update?

Thank you in advance.

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

Brammer88 — Mon, 17 Jun 2024 22:01:10 GMT

Dear community,

I am facing the same issue here.

My workaround is to just create an azure databricks activity in my ADF pipeline and use code inside a notebook to copy over data from tables inside unity catalog to ADLS gen2, but this is in my opinion less ideal then just using a azure databricks deltalake connector. Another workaround could be a view in the metastore that references tables in unity catalog, also not a very pretty solution.

is there some other solution that you would prescribe as best-practice for this usecase?

Thanks,

Bram

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

cmunteanu — Tue, 18 Jun 2024 09:46:49 GMT

Hi,

I need to ingest data into Databricks Unity Catalog using ADF, so a workaround is to use an object storage as a Landingzone (ADLS gen2), that is mounted into Databricks under /mnt. I use ADF to copy data (incrementally) to this landingzone first, then use Databricks, to take this data using autoloader into Bronzezone which is under UnityCatalog. Of course, one would expect to perform this data ingestion directly to UnityCatalog schemas, using ADF, but this seems impossible, so far.

Thanks

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

ChrisK — Fri, 05 Jul 2024 17:44:39 GMT

I'm also experiencing the same issue. This article describes a method for getting access to Unity Catalog through an ODBC linked service that connects through DB SQL Warehouse. The article also indicates that direct Unity Catalog access via a Databricks Delta Lake linked service is limited depending on the security setup on the storage accounts used by Delta/DB. This isn't an optimal setup IMO but an acceptable workaround.

Note from the trenches: Securely read or write data from Databricks Unity Catalog using Azure Data Factory | by Ivan Donev | Medium

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

StdyFriend1 — Thu, 11 Jul 2024 21:22:00 GMT

Hi all,

Please try following this syntax in the database field

<catalog_name>`.`<database_name>

It is important that you use ` not '

This should force the connect to use a Unity Catalog Database.

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

ChrisK — Fri, 12 Jul 2024 13:43:05 GMT

Woo hoo! Thank you so much for this suggestion. It is working now.

This seems like a bug in Azure Data Factory when it constructs the query to send to Databricks. The linked service connector must have been written pre-Unity catalog. We should be able to specify <catalog_name>, <database_name>, and <table_name> as separate parameters when configuring the linked service.

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

cmunteanu — Fri, 12 Jul 2024 14:57:33 GMT

That's crazy, but it just works !!!! Thanks so much, Microsoft should include this trick in its documentation.

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

Jaganmalli — Fri, 29 Nov 2024 16:28:54 GMT

@StdyFriend1 I have tried your solution, but no luck. I am using system-assigned -managed identity in the linked service created in Azure data factory using "Azure Databricks delta lake connector".

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

Jaganmalli — Fri, 29 Nov 2024 16:31:53 GMT

Chrisk, Are you using " system-assigned -managed identity" or "access token"?

Re: In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop dow

sijopchalil2023 — Sat, 29 Mar 2025 17:22:47 GMT

Any further updates on this? I also have the same problem. I tried above solutions, but no luck. By default, it shows only default and it's tables and I can preview those tables. If I enter the <catalog>.<schema> manually, it populates all the tables in the given schema. But if I click preview it gives "user does not have use catalog on <catalog> error. If I enter manually the table and run the pipeline, it gives "TABLE_OR_VIEW_NOT_FOUND". I am using system-assigned -managed identity. Any help ?