Problem with SQL Warehouse (Serverless)

[pesky_chris]

I get the following error message on the attempt to use SQL Warehouse (Serverless) compute with Materialized Views (a simple interaction, e.g. DML, UI sample lookup). The MVs are created off the back of Federated Tables (Postgresql), MVs are created by DLT pipeline.

Your request failed with status FAILED: [BAD_REQUEST] [FAILED_JDBC.UNCLASSIFIED] Failed JDBC jdbc: on the operation: Failed table existence check: [....] SQLSTATE: HV000

Configuration:

The workspace is assigned to Metastore (UC), other objects e.g. Streaming Tables (DLT), stand-alone Delta Tables work with SQL Warehouse (Serverless) compute without problems. The Materialized Views are persisted on the external location (Storage Account) as other objects, so the root storage is the same for catalogs/schemas (it is metastore-level configured) - so network misconfiguration is likely ruled out.

What works?

- any other non-serverless compute does not have problems accessing MVs

Please Advise how to find a solution for this problem.

[RCo]

Hi @pesky_chris!

Are you otherwise able to access your federated Postgresql data via the Serverless SQL Warehouse (e.g. using a SELECT statement)?

If not, your issue may be due to networking/firewall between the Serverless SQL Warehouse and your Postgresql server.

Here is some info on configuring private connectivity for serverless: https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serv...

[pesky_chris]

Hi @RCo 

Thanks for your reply.

SQL Warehouse (Serverless) is set up to access the blob storage (on a storage account) set up with Metastore. Federated Tables come from the database hosted in the private network and are not meant to be accessed by Serverless. They work only with non-serverless compute, it is VNet Injection Databricks deployment.

I would assume that a Materialised View is effectively a persisted table with ability to refresh from the source. Serverless behaviour indicates that access to MV's underlying data source would be required to interact with the MV. This does not make sense, unless I don't understand how the MV does look like in Databricks.