Unity Catalog view access in Azure Storage account

[prasha123]

Hi,

I have my unity catalog in Azure Storage account and I can able to access table objects but I couldn't find my views that were created on top of those table. 

1. I can can access Delta tables & related views via Databricks SQL and also find the tableId for each table created in the catalog, back to the storage container

2. But I couldn't find the views that were created in the same catalog. The corresponding object storage for view is not visible in the storage account container.

Can you please help to solve this, a critical deployment is going on.

[Walter_C]

In Databricks, views are read-only objects composed from one or more tables and views in a metastore. They reside in the third layer of Unity Catalog’s three-level namespace. A view can be created from tables and other views in multiple schemas and catalogs. However, views might have different execution semantics if they’re backed by data sources other than Delta tables. Databricks recommends that you always define views by referencing data sources using a table or view name.

When it comes to Azure Storage, views are not stored as separate objects in the storage account. Instead, they are metadata constructs within Databricks that reference the underlying tables. This is why you can't see them in the Azure Storage account container. They exist within the Databricks environment and are used to provide a specific view or representation of the underlying data.

[prasha123]

Thank you for the explanation. So in my case I am trying to create a dataflow in azure data factory where I need those views. 
is there any way to get those views in ADF dataflow or I have to recreate in flow?

[artsheiko]

Hi,

Couple of options are possible : 

1. Use Databricks to do the complex SQL queries (joins, unions, etc) and write to a staging Delta Table. Then use DataFlow to read from that staged table. Orchestrate all of this using ADF or even Databricks Workflows. 
2. Use a Copy Activity in ADF to connect to Databricks over a view that has the complex SQL queries (joins, unions, etc) and write to the staged Delta Table that Dataflow can read from. Orchestrate both with ADF pipeline.
3. Use the Rest API inline dataset in ADF DataFlow to connect to the Databricks SQL execution API. This is fairly complex but may be a workaround : https://www.databricks.com/blog/2023/03/07/databricks-sql-statement-execution-api-announcing-public-...

It could be useful to take a closer look at Databricks workflows : https://docs.databricks.com/en/workflows/index.html. Maybe it will be powerful enough to respond to your needs.