Databricks Community

Jake2 · 01-09-2024

Hey everyone, We're making the move to Unity Catalog from Hive_Metastore and we're running into some issues performing Z-order optimizations on some of our tables. These tables are, in either place, materialized views created with a "create or refres...

Jake2 · 09-15-2023

I'm running into an issue during the "Setting up Tables" phase of our DLT pipelines where I'm told a particular field is unable to be merged due to incompatible datatypes. See this example: org.apache.spark.sql.AnalysisException: Failed to merge fiel...

Jake2 · 01-19-2024

For anyone who's reading this later: You can still Z-order your materialized views, but you can't run it as a SQL command. Instead, you can set it as one of the TBLPROPERTIES when you define the table. Here's an example:create or refresh live table {...

Jake2 · 01-19-2024

Hi shan_chandra, thanks for the response.We found that piece of documentation, but there's a couple points that we find vexing about this.It mentions that MVs are "Unity Catalog managed tables" - we have been using them on hive_metastore for quite so...

Jake2 · 09-18-2023

Hey Kaniz, I appreciate the response. I'm doing a lot of different tables in this pipeline. If explicitly defining the schemas is out of the question due to time constraints, would it work to just split the offending tables off into two pipelines?

Databricks Community

User Stats

User Activity

Z-Ordering a Unity Catalog Materialized View

Failed to Merge Fields Error on Delta Live Tables

Re: Z-Ordering a Unity Catalog Materialized View

Re: Z-Ordering a Unity Catalog Materialized View

Re: Failed to Merge Fields Error on Delta Live Tables