cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Cross region delta table creation in Azure

arajshree
New Contributor

Hi Community.

I have a usecase as below.

1. I have a Azure Databricks [ADB] in NCUS which has Unity Catalog [UC] enabled using the NCUS region metastore.

2. The ADLS Gen2 storage is in EUS

I am trying to registers a existing delta table in UC and also create new tables. Both of the operations hang forever. I am using a custom cluster to do so. If I user a serverless compute it succeeds. If the cross region delta table creation not supported using custom cluster?

1 REPLY 1

nayan_wylde
Honored Contributor II


Seems like an networking issue. In a notebook attached to the custom cluster, try dbutils.fs.ls("abfss://<container>@<account>.dfs.core.windows.net/<path>") to confirm reachability before running CREATE EXTERNAL TABLE. (If this hangs or errors, itโ€™s a network/permission issue, not UC SQL.)

Custom cluster runs uns in your VNet. If your ADLS firewall only allows particular VNets/subnets or private endpoints and your NCUS VNet isnโ€™t allowed to reach EUS ADLS, UCโ€™s validation calls from the custom cluster will stall or fail. You must open the network path from that custom cluster to the storage (firewall allowlist, Private Endpoint, DNS, outbound rules).