SDP: Your table schema requires manually enablement of the following table feature(s): timestampNtz.

[liquibricks]

Trying to create a Spark Declarative Pipeline with a Materialized View based on a Streaming Table. The Streaming Table works fine, but when trying to add a Materialized View the pipeline fails to initialize with:

Your table schema requires manually enablement of the following table feature(s): timestampNtz.

To do this, run the following command for each of features listed above: ALTER TABLE table_name SET TBLPROPERTIES ('delta.feature.feature_name' = 'supported') Replace "table_name" and "feature_name" with real values. Current supported feature(s): appendOnly, changeDataFeed, deletionVectors, invariants.

The MV does not exist so I can't set the table properties. I found a thread about setting it in the Spark context, however setting 

"spark.databricks.delta.properties.defaults.feature.timestampNtz" to "enabled" in the Spark context doesn't seem to have any effect. Any ideas as to what's wrong?

[stbjelcevic · Accepted solution]

Hi @liquibricks ,

I see you said that the materialized view does not exist, so you can't run the ALTER TABLE command. Try to add TBLPROPERTIES ('delta.feature.timestampNtz'='supported') to the MV CREATE statement (or pass it via table_properties in your pipeline definition). Don’t rely on Spark defaults for materialized views as they won’t be applied.

[liquibricks]

Thanks @stbjelcevic ! Setting it as a table property appears to have worked. My pipeline now has other problems to address so it's not working yet, but that's a different thread 😅

            @dp.materialized_view(

               name=f"{mycatalog}.{myschema}.mv_{mytable}",

               table_properties={"delta.feature.timestampNtz": "supported"}

            )

            def create_dynamic_mv():

                ....