Is anyone getting up and working ? Federating Snowflake-managed Iceberg tables into Azure Databricks

[ripa1]

I'm federating Snowflake-managed Iceberg tables into Azure Databricks Unity Catalog to query the same data from both platforms without copying it. I am getting weird error message when query table from Databricks and i have tried to put all nicely in place and i can see that Databricks says: Data source Iceberg which is already good. Snowflake and Databricks on Azure both.

Error getting sample data Your request failed with status FAILED: [BAD_REQUEST] [DELTA_UNIFORM_INGRESS_VIOLATION.CONVERT_TO_DELTA_METADATA_FAILED] Read Delta Uniform fails: Metadata conversion from Iceberg to Delta failed, Failure to initialize configuration for storage account XXXX.blob.core.windows.net: Invalid configuration value detected for fs.azure.account.key.

I was so close already when thining it all works and noticed in one point that actually Snowflake compute was used, so databricks was not reading directly Azure stored iceberg files. So i added storage location, auth paths etc. and then i started to get this error message and I am pretty sure no account key is needed, even databricks has access and can see the file, list those etc. something is happening on config that i can not understand why this does not work. Basically i have current setup like this :

• Snowflake (Iceberg table owner + catalog)

• Azure object storage (stores Iceberg data + metadata)

• Databricks Unity Catalog (federates Snowflake catalog + enforces governance)

• Databricks compute (Serverless SQL / SQL Warehouse querying the data)

[Hubert-Dudek]

try this:

1. In Snowflake, run SELECT SYSTEM$GET_ICEBERG_TABLE_INFORMATION('<db>.<schema>.<table>'); to retrieve metadata location.
2. In Databricks, create an external location that matches the metadata URI.

[Hubert-Dudek]

Check also the behavior on both serverless and classic compute - it is catalog federation so it can work like that and needs setup as you read only metadata from catalog and later databricks is just reading data from that metadata location

[Hubert-Dudek]

on classic compute  you can set access to blob by sparkconf:
spark.conf.set(f"fs.azure.account.key.{account_name}.blob.core.windows.net", account_key)

[ripa1]

Thanks Hubert. I did check the Iceberg metadata location and Databricks can list the files, but the issue is that Snowflake’s Iceberg metadata.json contains paths like abfss://…@<acct>.blob.core.windows.net/..., and on UC Serverless Databricks then tries to initialize Blob access and fails with fs.azure.account.key (legacy creds), which Serverless doesn’t allow. Databricks/Azure support confirmed fallback/legacy access cannot be enabled on Serverless, so this seems blocked unless using Pro/Classic compute or Snowflake can publish DFS (…dfs.core.windows.net) paths.I’ll close this thread from my side for now — if someone has this working on Serverless on Azure without account keys, I’d still love to hear how.

[aharsh92]

@Hubert-Dudek  @ripa1 Running into a similar issue. Were you able to get a resolution on this? Assuming spark.conf.set(f"fs.azure.account.key.{account_name}.blob.core.windows.net", account_key) should not be needed as it should authenticate via UC.